Who here works in tech?

[KittiesKorner]

I like VS a lot, I haven't messed with VS Code too much; and I primarily use it for school. For work however, based on our infrastructure the thing we use the most is PGAdmin, but that's because we use Greenplum, and if we want to take advantage of it's parallelism we have to run things in DB (which is what MADlib is for); and PGAdmin is the common tool. The new version of PGAdmin is beyond awful and we actually have an outstanding ticket with our sys people to install some actual IDEs on our VMs (Toad, DBForge and one other one, forget what it is). I've used Toad a bit, back in the stone age when we were primarily an Oracle shop, and it was fine.

But, we're kind of in a weird spot where we're very much tied to Greenplum for now, with a move to Azure likely in our future, which limits our editor/IDE options. So, outside of PGAdmin we use Jupyter Notebook, which is fine.

EDIT: Oh and of course Notepad++, I know the other dev teams like Ultra Edit, and I messed with it a bit, but I've just gotten so used to Notepad++ that when I just want a basic ***** editor that's what I open.

Haha that was it, notepad++! I always call it textedit because i was raised on macs (or apples, more precisely)

 

[KittiesKorner]

You all have any opinions on jenkins vs bamboo for continuous integration? As far as i can tell, it’s a configurable wash

 

[nvanprooyen]

You all have any opinions on jenkins vs bamboo for continuous integration? As far as i can tell, it’s a configurable wash

I've only used Jenkins. Isn't Bamboo the Atlassian product? (The Jira folks)

 

[KittiesKorner]

I've only used Jenkins. Isn't Bamboo the Atlassian product? (The Jira folks)

It is. I guess that’s a mark in jenkins’s favor haha, i just had never done ci before using bamboo so wasn’t aware. I have had to leverage atlassian’s api for jira before; that was (not) fun

 

[nvanprooyen]

It is. I guess that’s a mark in jenkins’s favor haha, i just had never done ci before using bamboo so wasn’t aware. I have had to leverage atlassian’s api for jira before; that was (not) fun

I've never had to do that, but it sounds terrible. Jira is super bloated IMO. We used it for awhile on a project awhile back.

 

[KittiesKorner]

I've never had to do that, but it sounds terrible. Jira is super bloated IMO. We used it for awhile on a project awhile back.

I hear you; i had to permit a user to open a ticket from a desktop app; ended up being a blob in node essentially to make a post call with umpteen query parameters


Sent from my iPhone using Tapatalk

 

[nvanprooyen]

This thread got real nerdy. Thanks guys.

 

[Ares]

The more I move into leadership, the less I actually work in the weeds on SQL code or data.

But I can see how awful IT leadership is.... at least at my company, so I will keep leading people and squeeze them to pay me for it.

If they choose not to pay me, ill find something else.

Honestly, most IT Managers and above are coders who got promoted into leadership and suck at it.... they are awkward and/or abrasive and have no clue how to actually lead people.

We have an AppDev Director who is leaving, who kept throwing insane timelines on projects and I had to keep pointing out that his teams could never hit any of the targets he set.

His response was that he had to do that because "It's the only way to motivate my teams"

He said that in front of me, my boss, and his own boss..... it was cringe af.

Competent IT leadership is not easy to find.

 

[KittiesKorner]

The more I move into leadership, the less I actually work in the weeds on SQL code or data.

But I can see how awful IT leadership is.... at least at my company, so I will keep leading people and squeeze them to pay me for it.

If they choose not to pay me, ill find something else.

Honestly, most IT Managers and above are coders who got promoted into leadership and suck at it.... they are awkward and/or abrasive and have no clue how to actually lead people.

We have an AppDev Director who is leaving, who kept throwing insane timelines on projects and I had to keep pointing out that his teams could never hit any of the targets he set.

His response was that he had to do that because "It's the only way to motivate my teams"

He said that in front of me, my boss, and his own boss..... it was cringe af.

Competent IT leadership is not easy to find.

That said, you get promoted a lot, so shut up!

 

[ShiftyDevil]

I've never messed with greenplum. What makes it better than vanilla postgres?

Re Jupyter, I know a lot of people like it for tinkering...but I just can't get used to writing stuff in cells in a browser window. I have an extension for VS Code that mimics its behavior in a side window in the editor.

So Vanilla postgres has, basically, no parallelism (at least up to the latest version I have a lot of experience with, which was 9.3). One process gets access to one core, that's it. Greenplum 'fixes' that by having a master and a bunch of slaves. So you have your slaves (segments) and you distribute your data across each one, and then the master balances the work load across them. So, you have your table, and it has 100 rows, your Greenplum environment has, lets say, 10 segments. So your 100 row table exists across each segment, with 10 rows distributed across them. So when you run a query against that table each segment works on the query then serves up the results to the master which munges it all back together.

So it's super fast when it comes to dealing with a metric shitload of data, because you *basically* get 10 Postgres databases working on the problem. And we are mostly focused on text analytics, so we're working tons of big, fat text rows all the time, and Greenplum is pretty good at that. But it's not "wide" parallelism, it's sort of vertically parallel if that makes any sense? We've actually been running into concurrency problems since it's a beast when it comes to working on huge data sets, so running a pretty complex Neural Net against millions documents is no problem, but if we try to run a bunch of significantly simpler models, like a decision tree or something, at the same time and we start hitting performance issues.

It's also forked off a very old version of Postgres, 8.2 I think. So all those nice features that started coming in version of Postgres 9 and later, like JSON being a straight up data type, don't exist in Greenplum. Pivotol has introduced some of those features that newer versions of PG have, but we're at the mercy of when Pivotol develops them. It's also, from what I understand, turbo expensive.

 

[KittiesKorner]

also, thanks for not hiring me, @ares!

Speaking of the management thing, Anyone ever heard of SAFe, or the Scaled Agile Framework? Run away, as quickly as you can. At my last job, I somehow ended up splitting time as a developer as well as a SAFe scrum master (which is actually against the SAFe Code of Conduct), and they offered to make me a full-time SAFe scrum master for more money and stocks, and I said **** your couch and quit.

I'm a lone wolf.

 

[number51]

Fucking software guys, the bain of my existence.

Who hurt you?

Software and IT guys are even worse than us EE's, that's right other nerds think you guys are too fucking nerdy.

The lines are pretty blurred now anyway, all engineers need to be able to write code now, so congratulations you ultra nerds win.

 

[KittiesKorner]

point of order: IT !== developer!!!!!!!

 

[ShiftyDevil]

I've never had to do that, but it sounds terrible. Jira is super bloated IMO. We used it for awhile on a project awhile back.

Ha, just saw this. We use Jira, and I got appointed to be the Jira guy. So I run our "scrums/standups/trying to keep the data scientists from talking for 20 minutes" and our Kanban board. And holy shit, Jira is a slow, clunky annoyance.

 

[nvanprooyen]

So Vanilla postgres has, basically, no parallelism (at least up to the latest version I have a lot of experience with, which was 9.3). One process gets access to one core, that's it. Greenplum 'fixes' that by having a master and a bunch of slaves. So you have your slaves (segments) and you distribute your data across each one, and then the master balances the work load across them. So, you have your table, and it has 100 rows, your Greenplum environment has, lets say, 10 segments. So your 100 row table exists across each segment, with 10 rows distributed across them. So when you run a query against that table each segment works on the query then serves up the results to the master which munges it all back together.

So it's super fast when it comes to dealing with a metric shitload of data, because you *basically* get 10 Postgres databases working on the problem. And we are mostly focused on text analytics, so we're working tons of big, fat text rows all the time, and Greenplum is pretty good at that. But it's not "wide" parallelism, it's sort of vertically parallel if that makes any sense? We've actually been running into concurrency problems since it's a beast when it comes to working on huge data sets, so running a pretty complex Neural Net against millions documents is no problem, but if we try to run a bunch of significantly simpler models, like a decision tree or something, at the same time and we start hitting performance issues.

It's also forked off a very old version of Postgres, 8.2 I think. So all those nice features that started coming in version of Postgres 9 and later, like JSON being a straight up data type, don't exist in Greenplum. Pivotol has introduced some of those features that newer versions of PG have, but we're at the mercy of when Pivotol develops them. It's also, from what I understand, turbo expensive.

Thank you for the awesome explanation. I am in the middle of trying to make some infrastructure decisions, and this was very helpful. I think I understand what you mean by vertical vs horizontal. When dealing with large data (e.g. big strings for analysis) thing work well, but dealing with a bunch of smaller operations throughput becomes an issue because of the master trying to put it all together because things are fast so you hit concurrency issues? Is that close? Or did I **** it up?

 

[ShiftyDevil]

Thank you for the awesome explanation. I am in the middle of trying to make some infrastructure decisions, and this was very helpful.

Yeah, if you have questions about what it's like to live with, let me know. Back in the dark ages we were all Oracle all the time, then switched to Vanilla Postgres. That was some time after Postgres 8.4 but before 9. And we were largely Postgres (with a few one off SQL server DBs floating around) for several years before moving to Greenplum.

Vanilla Postgres is probably my favorite relational DB, it's open source, really well supported, has nice syntax, easy to manage, all that good stuff. But it always had some real drawbacks (ie, not offering parallelism), but I'm pretty far out of the vanilla Postgres loop anymore.

 

[nvanprooyen]

The more I move into leadership, the less I actually work in the weeds on SQL code or data.

But I can see how awful IT leadership is.... at least at my company, so I will keep leading people and squeeze them to pay me for it.

If they choose not to pay me, ill find something else.

Honestly, most IT Managers and above are coders who got promoted into leadership and suck at it.... they are awkward and/or abrasive and have no clue how to actually lead people.

We have an AppDev Director who is leaving, who kept throwing insane timelines on projects and I had to keep pointing out that his teams could never hit any of the targets he set.

His response was that he had to do that because "It's the only way to motivate my teams"

He said that in front of me, my boss, and his own boss..... it was cringe af.

Competent IT leadership is not easy to find.

The inverse of that are pure PMs who don't understand technology at all. I'm not sure who is worse. It is a delicate balance usually to be able to manage technical teams well.

 

[ShiftyDevil]

Software and IT guys are even worse than us EE's, that's right other nerds think you guys are too fucking nerdy.

The lines are pretty blurred now anyway, all engineers need to be able to write code now, so congratulations you ultra nerds win.

We always do.

 

[Crystallas]

I guess I am of the mind that if someone understands the core fundamentals of software development, it is relatively easy to transition from on language to another. Of course there is some learning curve / nuances involved, but having a solid base makes it relatively easy. I've written stuff in C#, Java, PHP, Ruby, Python, etc along with different frameworks and moving from one to another hasn't been that hard.

That said, I don't consider myself to be a "real developer", even though I've written a lot of stuff that works. I'm more of a marketing / analytics / strategy guy that leads teams. Having a solid understanding of development certainly helps though in recruiting talent and managing people / projects.

Yes, it's easy to transition if you are of the right mindset. The issue I personally have is that we push these easy to start object languages that are riddled with bad habit forming techniques and teachers/professors that merely human themselves or not qualified to really do their job. That's why it's so important that users learn on a language that helps them form good habits. If you used Rust, you would totes get what I mean. It solves so many problems and it's a real language for both now and many years to come, as well as one of those few languages that can act as a C-type replacement, where Java, Python-class languages all say they *can* be C-type replacements(not wholly, just at certain levels of the stack), their instability(by comparison) and performance usage are simply unrealistic. IMO, Python is a great perl replacement and that's pretty much it.

And I know this goes too deep into the rabbit hole. But the consequence of pushing so much resource heavy code(again, by comparison to something that is cleaner), means computing devices of all sorts now need *more* resources, which means they need more power, they use more electricity/battery, etc. A snowball of bad habits and direction that will require decades to correct unless the hardcore for no reason C/ASM guys and the user level WYSIWYG IDE whatever guys can meet in the middle. Think about it as a logical conclusion. If everything was merely 15% more efficient overall, calculate how many resources we save, how much the environmental impact changes. It's not a drop in the bucket.

 

[Omeletpants]

You tech nitwits will appreciate this:

My granddaughter is 6 and brilliant. She was accepted into the Northwestern accelerated program for kids at 4, fluent in Spanish, founding member of the Naperville Toastmasters club for kids, and recently was accepted into a programming class for 12 year old at 6. My daughter has her in some extracurricular activity 7 days a week which I think is too much

Lets just hope when she is 16 that she doesnt come home and say she is marrying timmy