I found a disconnect between a fellow bit shifter the other day. This programmer and I go way back. Great person. He is struggling to relate to me on some programming topics though. We are in separate worlds…
Algorithms + Data Structures = Programs
Wirth died recently. His book of the same name as the header here, is quite brilliant. He is the father of most modern languages and he kinda nailed it with this title er, equation. I have a hardcopy of the book that a cherish. One day I will give it the attention it deserves.
Data Structures
variables
structs/classes
arrays
stacks
queues
dictionaries
graphs…
Algorithms
Loops
Depth First Search
Breadth First Search
Rob First Search
…
Programs
Input, (loop), output
command line
gui
batch
To teach the above first without a very strong mathematical background maybe a mistake. There is a reason there is a pedagogy of tons of math before programming class. It maybe mistake, but I know why it is this way. Programming is math all the way down to the metal and glass (sand)
One Language to Rule Them All
metaphor with music and languages
Often when you see a musician, and you ask them what instrument they play, often they will respond with a list. But they will probably start with the instrument they learned on: Piano or Guitar.
As an aspiring string plucker you may ask, how can this person be such a hardcore bard and I can’t play mary had a little lamb in every key on neck of guitar. The skills transfer. There is Transfer Learning.
Often we learn to learn by learning other things.
Learn to crawl, before you walk, etc. Let us go on a journey…
I implore you to grasp that math is the language of the universe, and doing math you are centering yourself with it.
Me teaching the youth how to wire up lasers in 2014
If you think programming is hard now. You are listening to the wrong gurus. Programming computers has never been easier. I will not justify that statement in a single breathe, and this is kind of a venting post, so please take with a grain of salt.
Exponential Complexity
Where are we in this sigmoid? What is a sigmoid? Lets ask our AI brethren.
A sigmoid is a special type of mathematical function that is often used in machine learning and data analysis. It looks like an “S” shape, with the middle part being steep and the two ends being flat. Sigmoids have a really useful property: they can take any number as input, but they always output a value between 0 and 1.
Here’s an example of a sigmoid function:
f(x) = 1 / (1 + e^-x)
This formula might look scary, but it’s actually pretty simple. The “e” in the formula is just a special number called Euler’s number (it’s about 2.71828). The “-x” inside the parentheses means that we take the opposite of whatever number we plug in for x. So if x is 5, then -x would be -5.
e^-x == 1/e^x
-rob
When we plug a negative number into this formula, we get a number very close to 0. When we plug in a positive number, we get a number very close to 1. And when we plug in zero, we get 0.5. So you can see how the sigmoid function maps any input number to a value between 0 and 1.
I ❤ functions. Sigmoids are very cool. I actually used a sigmoid back when I did my Genetic Algorithm Caterpillar thing for undergrad a decade ago. I used a slightly different but same effect.
Creepture circa 2013 – sigmoid activation for network between “muscles”
Excerpt from Creepture paper (appendix)
Ok so why is this useful [late update]
In neural networks in order for a neuron to become excited and “fire” the signals into the neuron through a “synapse” must be added and the fed to a sigmoid function. This will fire a zero or one to the other connected neurons. And then each layer this happens and is forward or backward propagated through the network to the output/input.
In this simple caterpillar example each joint represents a “motor” or “muscle” that is activated by the sigmoid. Zero or One. Move + Stay
So what does this have to do with the exponential complexity topic. Lets talk e^x
Put them into your graphing calculators. play around with sweep.
The Natural Number
e has this special feature. It is often called “Euler’s Number” approximately 2.718…
Deriving it links to compound interest. So you know it’s good. It is irrational meaning you cannot represent it fully with a decimal or fraction. Go take a look at your precalculus books 🙂
(e^x) != (1 / (1 + e^-x))
You can be tricked you are in an exponential function. You might be in a sigmoid 😦
With the amount of documentation, tutorials, influencers, companies shilling products…. I get it. Your signal to noise ratio is approaching zero. But let me give you some signal in that numerator.
Go back to first principles when possible (the math!!!)
Avoid new language (versions), frameworks, fads/trends
Unless you understand this could be completely a waste of time and you are ok with it. schedule your treks.
Remember programming has a fashion sine curve. If you stick around long enough it will become advant-garde again 🙂
Focus on product, use your product, programmers that are NOT project/product driven are doomed to make buttons that nobody clicks. Make something so valuable and ship it that so that if it goes down, PEOPLE CARE.
Tried out GitHub’s Copilot. I was coding my way through the Ray Tracer Challenge. Turn on copilot. Wow. Just tab to complete. Microsoft (M$) shipped this thing? The lawyers let them!?
The Crutch
Of all the dumb things I’ve talked about in interviews, talking about integrated development environments as a crutch, is probably the dumbest. I think the special mix of jet lag, booze, and delays in travel really set me up for anti-success for my Microsoft interview right out of undergrad. It couldnt have been a skillissue.org.
Twasn’t good but here is how it went.
Ready to Move to Washington State
So how do you land a job interview at M$? Well… they come to your campus, if you are going to a college with a decent programming reputation. Once or twice a year they will be there. At the career fair, or just sometimes they setup in the Harris Building at UCF. Then look at your resume and pass you to a very, very, very easy interview on campus.
If you pass, you get free trip to Washington! At this point I bought a copy of cracking the coding interview and pretty much made plans how I’d slice up the company when I was running it. I was surely hired. I’m best programmer I know. Besides Arup maybe.
Hiring Pipeline
Microsoft has one of these impressive hiring pipelines. They send out their jesters into the universities of the world. And send them all back to the mothership. Right away I knew something was wrong, I was overdressed. Though my scrum master look was on point with corduroy jacket. There was at-least a HUNDRED other people interviewing at same time as me. Of course they bussed us all around Redmond. Washington is beautiful.
The Lead Interview
Let’s just skip to the main dude for the coding. I got bussed to a building that was pretty nice and old. It reminds me of a 90s mall. Looks legit though, I could work here.
Meet guy, guy is nice. Indian (not that it matters) — impeccable skills/english/etc a pro. I think he is the lead of his team. Ok, turns out he is running some division of Onenote. Due diligence has not been done, but I do know Onenote, well some of it.
Pleasant meeting start. He is obviously passionate about the tool he creates. We talk multi-player editors and how Onenote was first before google docs in collaborative editing. I point out you can actually see the cursor change in google’s implementation 🙂
Anyways. Leet code challenge.
Order and remove duplicates in a list. Can only pass through list one time, and You cannot allocate any new memory?
Leet Code Challenge 1 – M$ Onenote 2014
Calculating the hash is too expensive (he says), can’t use dictionaries… ugh. I struggle through and think i get a decent whiteboard of flow. But I pretty much try and get more help than the Lead is willing to give me. This is my worst nightmare. Score 5/10 (my scoring) — I feel like there was another gotcha, but I blocked it out.
Coding on Demand
Coding on a whiteboard with no auto-complete, google (at the time), and now no co-pilot. It is a different skill than anything you normally do for the job. Except when you teach people how to code, of course. This is what you are doing when you run code in your head with 2 people, 1 marker, and a whiteboard: The Mind Compiler
This is the opposite of what I do when I try to hire a programmer BTW :*
The Nerd Interview
I get the sense that I failed the first interview, but I have 2 more. He is moving me again to another person on the team. I am in the Onenote enclave i’m realizing.
Ok so I get dropped off with another team member, immigrant from western Europe. Nice older fella. This interview is more personal and we talk about my projects and OpenCV and he is shocked that it has is a C++ API. Dude is living under a rock. But really nice and we both learned something from each other. The coding challenge was tough, but I can’t remember it now. I probably got 9/10 on it.
The Yeoman Interview
Ok so now i get dropped off into an office with no windows. This dude is cookin’ something up in here. Probably building serious shit. He worked in somewhat of a lab. He was closest to my age, although I may have been older than him. The other two were def 10+ years my senior.
He hits me how to see if there is a cycle in a linked list. I wink at him and regurgitate it. We goto lunch. In this lunch I open up about some philosophies.
“I love IDEs but they are crutch” he nods in agreement, but you can see the look in his eyes. I will not fit in here. I can remember it turning quickly. Maybe it was my shit performance in the beginning with the lead. But I only had 3 interviews that day.
I went back to central FL with a story and a fire in my belly.
Back to Copilot
Please understand that I’m not a acc/decc but this technology is not to be taken lightly. Ok so what is Copilot?
GitHub Copilot has multiple offerings for organizations and an offering for individual developers. All the offerings include both code completion and chat assistance. The primary differences between the organization offerings and the individual offering are license management, policy management, and IP indemnity.
Organizations can choose between GitHub Copilot Business and GitHub Copilot Enterprise. GitHub Copilot Business primarily features GitHub Copilot in the coding environment - that is the IDE and CLI. In early-2024, it will also include GitHub Copilot in GitHub Mobile. GitHub Copilot Enterprise includes everything in GitHub Copilot Business and adds an additional layer of customization for organizations as well as GitHub Copilot integrated into GitHub.com as a chat interface to allow developers to converse about their codebase and action buttons throughout the platform. GitHub Copilot Enterprise can index an organization’s codebase for a deeper understanding of the customer’s knowledge for more tailored suggestions and will offer customers access to fine-tuned custom, private models for code completion.
GitHub Copilot Individual is designed for individual developers, freelancers, students, educators, and open source maintainers. The plan includes all the features of GitHub Copilot Business except organizational license management, policy management, and IP indemnity.
I turned it on fully leaded at first. Is this fair use? Hell nah. GPL is viral. You can’t spoon feed off entire codebase like this. This isn’t clean roomed reverse engineering for sure.
Ok so lets try turning on “don’t produce public code” whatever that means.
I was going to make videos, and talk and aand and… just go use it.
Beware.
Admonitions
This copy pasta machine will need to be severely neutered to be useful in a codebase you want to maintain ownership of. The virality of some of the licenses, you will lose control. For instance at the job, we cannot use copilot. Legal will not allow it!
copy and pasting code is generally frowned on, you should go through the motions to actually type. think and internalize it. just like you are playing a song on a guitar 🙂
remember: most of us old timers learned by copy pasta from a 1980s tech magazine basic.
I’m debating to get the new meta rayban face computer glasses. Not sure I want to go even semi-gargoyle though. Huh? Gather round weebs and let us take a look back in time to 1992.
There is a book that influenced many of us, this novel is “Snow Crash”
Using Elong’s chatbot we have the topic of this post summarized nicely for us.
“Gargoyles represent the embarrassing side of the Central Intelligence Corporation. Instead of using laptops, they wear their computers on their bodies, broken up into separate modules that hang on the waist, on the back, on the headset. They serve as human surveillance devices, recording everything that happens around them. Nothing looks stupider; these getups are the modern-day equivalent of the slide-rule scabbard or the calculator pouch on the belt, marking the user as belonging to a class that is at once above and far below human society. ”
Neal Stephenson – Snow Crash ’92
“Hello! Hello! Hello!” she’s shouting. Then she hears the honk of a car horn. Coming out of the telephone.
“Hello?”
“It’s Y.T.”
“How are you doing?” This guy always seems a little too laid back in his personal dealings. She doesn’t really want to talk about how she’s doing. She hears another honking horn in the background, behind Hiro’s voice.
“Where the hell are you, Hiro?”
“Walking down a street in L.A.”
“How can you be goggled in if you’re walking down a street?” Then the terrible reality sinks in: “Oh, my God, you didn’t turn into a gargoyle, did you?”
“Well,” Hiro says. He is hesitant, embarrassed, like it hadn’t occurred to him yet that this was what he was doing. “It’s not exactly like being a gargoyle. Remember when you gave me shit about spending all my money on computer stuff?”
“Yeah.”
“I decided I wasn’t spending enough. So I got a belt-pack machine. Smallest ever made. I’m walking down the street with this thing strapped to my belly. It’s really cool.”
“You’re a gargoyle.”
“Yeah, but it’s not like having all this clunky shit strapped all over your body—”
“You’re a gargoyle. Listen, I talked[…]”
Excerpt From Snow Crash Neal Stephenson This material may be protected by copyright.
Snow Crash Again 🙂
Facecomputers… unfortunately they are here. Neuralink can’t come fast enough
Consistency is the most important part of any process that requires a low-latency high-throughput system. There is something about having conventions rather than rules, and I remember people saying this 10 years ago. I didn’t understand it then, and I barely understand it now. Hopefully this post will guide the reader where I’ve been lost.
I will do a generative ai image post soon!
Be strict when sending and tolerant when receiving
I like the idea of being flexible in what you receive and strict in what you send. This is a guideline Tim Lee had when he invented the web. It is a great philosophy or “code to live by”
So is this just bike shedding?
There is it this term and kind of a story around this concept of a committee that has an endowment and wants to build a park… (sorry fellas)
In the world of project management and software development, a peculiar phenomenon known as “bike shedding” can have significant implications on productivity and team dynamics. Derived from an analogy coined by philosopher P.J. Plauger, bike shedding refers to the tendency for groups to spend a disproportionate amount of time and energy discussing and debating minor or trivial issues, often at the expense of more important, complex matters.
Mixtral24
What color we want boys? — This will be used in in-painting post 🙂
Big Picture
Where do we host repository?
What is main branch, is it svn-trunk style, patch email
collab rules
syntax rules…
… welcome to giant corp or crazy small corp
Devil is in the Details
Budget
Timeline
Lawyers
Technology Stack (truly this might be hell discussion but it should be agreed on)
Focus on details that matter, ignore sticky stuff until you can’t. Don’t lock yourself into a bad design. Keep an open mind. For me this is the hardest part.
Scaling Scopes
To be able to jump between the two views. System Level vs. The Weeds. I will continue this after it marinates for a bit. But the gist is being able to jump between parts of the graph of the system at various zoom levels. This is the key to being a successful engineer. The “application” needs to stay in “production”
Remarks
I will start a running list of style guides for repositories that I think are reasonable. Feel free to submit more.
If it isn’t clear I have a GPU addiction. This post will render away all doubt about my technolust. I love hardware. I love exotic hardware that I’ve never seen before. I love to program or configure directly against this hardware to squeeze every last bit of performance out of it. Think how some (even old) XBox or Playstation games still are amazing on the platform that is a decade old. They programmed down to the metal.
Where do you host your website? R730 running stacked on an old P40 desktop-hack with converted blowers
>>> tell the story of saddam huessein and the playstation super computer he built
Saddam Hussein, the former President of Iraq, was known for many things, but building a PlayStation “supercomputer” is not one of them. There have been rumors and conspiracy theories that Hussein’s regime had developed a powerful gaming console-based computer for military purposes, but these claims have been widely debunked.
The story likely originated from a 2000 article in the Iraqi newspaper Al-Nasiriya Al-Gadida, which claimed that Iraqi scientists had built a supercomputer using 3,000 PlayStation 2 consoles. The computer was said to be used for weather forecasting and oil exploration, not military applications. However, there is no evidence that such a machine ever existed, and the claim has been dismissed as propaganda by many experts.
It’s worth noting that while the PlayStation 2 did have impressive technical specifications for its time, it was not designed to function as a supercomputer. Building a true supercomputer requires specialized hardware and software, as well as a deep understanding of computer science and engineering principles. It is highly unlikely that such a machine could be built using off-the-shelf gaming consoles.
In summary, while Saddam Hussein was known for his military ambitions and authoritarian rule, there is no evidence to support the claim that he built a supercomputer using PlayStation consoles. The story appears to have been exaggerated or fabricated for propaganda purposes.
Mixtral24
I guess I’ve been snoped. Moving on. If you want ME to build a super computer out of playstation2’s let me know.
Building a Better Large Language Model System
If you have been following along, I have a consumer level machine learning rig and a Poweredge r720. Things are going great but I figured I’d work my way towards the newer GPUs for servers. I needed a new server and, why not let’s try the r730. It is a little more expensive on e-bay but it looks a bit better spec wise in some areas I find desirable.
What area’s are those?
>>> what is difference from a PCIe bus between a r720 and an r730?
Both the Dell PowerEdge R720 and R730 servers use Peripheral Component Interconnect Express (PCIe) slots to connect various expansion cards, such as graphics cards, network interface cards, and storage controllers. However, there are some differences in the PCIe bus architecture between these two server models.
The Dell PowerEdge R720 supports up to eight PCIe 3.0 slots with a maximum bandwidth of 8 GT/s per lane. The server has a total of 40 PCIe lanes available for expansion cards. The R720 also supports the use of mezzanine cards, which are smaller expansion cards that can be mounted directly onto the motherboard.
On the other hand, the Dell PowerEdge R730 supports up to twelve PCIe 3.0 slots with a maximum bandwidth of 8 GT/s per lane. The server has a total of 64 PCIe lanes available for expansion cards. The R730 also introduces support for PCIe Gen 3 x16 slot, which provides higher bandwidth than the previous generation.
In summary, while both servers use PCIe 3.0 slots with similar maximum bandwidth per lane, the R730 supports more PCIe lanes and slots compared to the R720. Additionally, the R730 introduces support for a higher bandwidth PCIe Gen 3 x16 slot, which is not available on the R720. These differences can affect the overall performance and expandability of the server depending on the specific workload requirements and expansion cards being used.
Mixtral24
Poweredge R730
Really struggled this time to get ollama, you can see in the history (setup) below
So I figured I’d push something out this year to justify the bill of hosting this site…
Let’s just say I’ve lost something over the past few years in programming computers. The magic is gone. I know how it works, I know why it works, I’m not impressed.
Every time I use a piece of kit from someone, I feel I’m being funneled into something. Save on OneDrive, save on iCloud… Why can’t I find my actual hard drive in Finder anymore?
End key doesn’t work in editor on WordPress to go to end of line…
I’m grumpy.
Nobody respects the user. You know who calls their customers users? Drug Dealers.
Ok enough complaining.
What are you up to?
So my last post was written in an Airbnb in Pensacola, FL while I was selling my house in Seattle. That sale went well, and now I’m back in the panhandle.
from a narrator perspective this was right before we shipped the final touch of “the rewrite” of the BAW PDK. I feel quite different about the above post than when I wrote it. I have my mojo back…
CommandPattern.org 