The Tool Maker’s Dilemma

A humorous illustration entitled "The Tool Maker's Dilemma" with the main person not so freaked out, replace speech bubbles with nonsense slogans, make it higher resolution if possible, make it appear like an old style 70s poster, and make it more technology based

Programmers love their tools. It can be a fetish. How many tools is too many tools between your software and your customer?

Sometimes having more tools in your toolbox is a hinderance not an asset. Organizational breakdowns. Lost tools. Normally there are a few key tools you always have close by if needed.

As engineers or programmers it is often our bench software: Matlab, python, various editors and programming environments. Everyone loves good tools.

Editors and IDEs

Typing is so important. The speed of thought into your editor, the interface between man and machine. Keyboard shortcuts, specialized editors, beyond qwerty… they all tie you to a configuration. It is not a deal-breaker, but its important to reduce the friction between you and your data entry point. But the more you configure the harder it is to work on another’s system.

Do I have to mention emacs and vim? Yes. These ancient editors require months of training to use effectively and perhaps years to master.

VIM and EMACS fighting in a ring in a hellscape outside in a post-apocalyptic setting with a mortal kombat style

1970s computing was defined by big iron machines or mainframes. Big servers, dumb clients. PDP-10’s main storage was tape machines and TECO was the editor of choice. A single page editor. This was great for editing LISP programs and eventually an Editor of Macros (Emacs) was developed mid-70s. Somehow it is still the preferred editor of many programmers.

On the other end of the editor spectrum was “line editors” which were more attuned to working on teletypes. Teletypes are typewriters with serial links. Could it be over a phone line? Vi is a visual line editor created shortly after Emacs. Its modern version was rebooted in the 1980s as Vim. Again, the editor is still used at large (in 2024) and there is a large key-binding extension/plugin for most Integrated Development Environments (IDEs).

Is it worth it to learn emacs or vim? Yes, but that is my opinion and I wouldn’t suggest learning to ship a product and learn vim at same time. It is a lifestyle change that seems less and less important these days. It’s a fun distraction from real work.

Programming outside of a UNIX environment is largely done in a IDE. Visual Studio being the de-facto standard for C++ development. As opposed to makefiles and shell scripts, the idea of an IDE is everything is in one spot. Editor, Compiler, Debugger. Very convenient, you can setup IDE like features in emacs and vim, you could say emacs is a IDE for LISP. Borland IDEs have a special place in my heart as this is where it all started for me, outside of QBasic of course.

The point I’m trying to make is, you can approximate one with the other. Your customer doesn’t care which IDE or text editor you use. The language, framework, etc is a builder’s artifact. A tool mark.

Custom Keybindings & Hardware

If you listen intently you can hear the crackle of mechanical keyboards chording and creating arpeggiations of clicks echoing. Mechanical Keyboards are nice, but I think if you wanted to become inept on the other peoples machines. Just learn DVORAK or COLMAN instead of QWERTY. Still awaiting learning how to use my ErgoDox.

The command interface or how you move around the file are what I would say splits emacs and vim editors. Modal (vim) vs normal. Let’s leave that for another post. This is all to say the mouse is not the most important tool of a programmer when writing code. Your hands re on your keyboard.

Some people (ahem) have even added a foot pedal for shift or other keyboard strokes.

Vendor Software

On a unix you are almost guaranteed to have a C compiler. I think that is POSIX standard. Your distribution will provide it. Microsoft provides the Visual Studio. One you thing you can do is make these IDEs emulate emacs or vim. I do this for vim in both Visual Studio (Proper) and VS Code. Its pretty close and I get some breakpoints for free. On Linux I’ve heard good things about Qt Creator as an IDE.

You may have other vendors outside of your main development tool kit provided by vendor 1. Every piece of software you add beyond this point is normally business related. Libraries are the most important consideration when writing software in a reasonable amount of time. I would vet your libraries thoroughly and “vendor them in” to your main source control.

Then there is the subscription as a service (SaaS). This means you normally don’t host it yourself. Github is a perfect example of a legit reason to include a SaaS product into your development pipeline. But the real thing here is with a remote service, you run risk to disruptions you have no control over. Sometimes the website goes bankrupt. I would avoid these in your flows if possible.

Of course there is our own customization of your tools on top of this. Configurations, Plugins, Extensions, our own shell scripts. Often it’s the “dotfiles”. These build up over the years.

3 Levels

It is important for you to customize your development environment so that you can build and ship your product to customer. Often these setups can be cumbersome and fragile, but that is ok, the product doesn’t suffer from this directly. Now developer tool fatigue is a real thing. Developer ergonomics are important, but taken to the extreme, turns into a naval gazing exercise.

  • Rolling Stock – Minimum settings changed, As it was intended
  • Custom Setup – Tinkering deeper, voiding warranty
  • Fired or Getting Paid – How many apps/scripts are between you and your product or customer?

I like to roll stock when I can. Just because you CAN customize and extend, doesn’t mean you should. Is it worth it to add this new complication to my codebase?

Fired or Getting Paid? — I think this is an important question, will you create an unmaintainable mess while getting paid and then get fired because the business cannot support the liability you have created, or will you keep it running as best you can and improve the system reliability slowly over time?

endcap

I may or may have not written a kernel extension for windows that maps caps lock to escape. The autohotkey hack wasn’t fast enough. I love the idea of “jig’s” I love making tools, but sometimes it gets me into deep trouble…

Going Solar in 2023

In December of 2022 I got solar panels above my shop. The cost of energy has been so high and I use a bunch, I figured why not invest in a system. Total system cost installed was $36,066.36. Uncle Sam took care of a credit of 26% of total cost of system. So final price for me after doing taxes is…

$26,689.47 is the Final Cost of System after US Tax rebate

Calculating the return on investment (ROI), Let’s just take the number from the screenshot above which is $2,000 output for entire year. Also assume that its all same rate/fuel charge etc.

Annual percentage rate for 2023 is approximately 7.5% return. Estimated time until system is fully paid for by energy production: 10-12 years. Let’s hope she lasts.

This is a bit pessimistic ROI and depending on energy prices over next decade I predict I will break even in 2030. Mark your calendars.

2023 Wrap-Up

It is hard to believe that I only have 3 classes left to finish up my Masters. One elective (Power Electronics) and Project 1, 2. Going to stretch it out over 2024 and really try and present an impressive project as my capstone. I think this maybe where my university education ends.

So if you aren’t aware. I am now a YouTube star…

I am really proud of the work the team of students did. I was team lead and guided the team on creating a fairly extensive software suite for the Ghost Robotic’s dog. We only had about 9 weeks for the challenge and what we have presented is quite impressive. Android Application, ROS2 Node, Threat Detection in OpenCV and Python. It runs the gamut.

Will be up to the AFRL in 2024 to see if we build off this knowledge/team. Looking forward to the follow-up.

Python Again

So with all the hype in the world around AI and Deep Learning I finally decided to dig back in. I took Machine Learning in Fall of 2022, but this was very much the “old” machine learning. Meaning. Linear Algebra review, Support Vector Machines, etc. So I’ve enrolled in Andrew Ng’s Deep Learning Course: https://www.coursera.org/specializations/deep-learning?

This is excellent course and I’m 2/5 of the way through it now. The first class is “easy” because I’m familiar with it a bit. But the second was quite the slog and I had to really study and take time. I’m going to take a few more weeks before I start on the convolutional neural networks and then I’m excited to get to sequence models. I don’t think it gets to the transformer architecture in this course, but it should be good platform to learn the new hotness (ChatGPT etc)

I’m trying to switch all programming projects that make sense to python. To keep everything straight. To give myself a refresher, as I’ve not done python really for serious projects since Python2. I’ve decided to enroll in ExecuteProgram.com’s python course which is quite good. Almost done with it: https://www.executeprogram.com/courses/python-for-programmers/lessons/wrapping-functions (edit: now done with it 12/29/2023 in perpetual review?)

Very impressed with the method Gary uses at Execute Program. Looking forward to the next class he will release which will deal more with classes and advanced python stuff.

That being said the current non-work programming task I’m working on is… https://devtalk.com/books/the-ray-tracer-challenge which I’m writing in python and hope to have some pictures rendering in 2024. Wish me luck.

Happy New Year!

End of Summer Update 2023

Finished up the semester a few weeks ago now and am recovering from the sprint of creating a UART transmitter in VLSI class. The software was quite buggy and old, but it ended up working.

It was quite fun designing everything from scratch with NPN and PNP transistors. I actually punted on laying this beast out by hand. I did do some layouts by hand though…

I also wrote a paper and a bit skeptical including it in the public, but was a solid week of work. I actually used Word for this instead of LaTeX, because life is too short. My undergrad self would be disappointed. Professor gave me a ninety on it, no notes back 😦

This next semester is Aerial Robotics and some cyber security class. Looking forward to finishing the year and starting on the capstone project for my masters.

Over the past month or two I’ve also been involved with a robotic dog project. Learning ROS2 and fighting with simulators and docker. I’ll post more when there is something. Here is a short video of it walking some way points.

About halfway done with this ROS2 book. Is very good, love that it is in color. GUS pictured above with lidar and thermal cameras

Finally my guitar pedal addiction has been traded for eurorack.

Spending it as fast as I’m making it. Living the dream. See you next time!

User Hostile

Software companies and drug dealers both call their customers the same thing. Users.

unknown
  • Forced Upgrades (with no functional benefit/change)
  • Requiring Network or Lookup on Network on startup
  • Telemetry
  • Dependencies
  • Blackholes

Of the above, I think the forced upgrades is the worst. Like most things in the list, they make developers lives easier, but it is terrible for the customer.

Shipping often has become an ethos, but I’m afraid that most of your users would rather stability over new releases. I look at software the same way I look at restaurants. I want the same burger I got last time. I may glance at the specials and think maybe I will try that another time.

If they really were so special, they’d be on the menu

Seinfeld

If they really were so special, they’d be on the menu, to quote Mr. Seinfeld. It is quite disturbing to my flow of actually getting work done to launch an app and be presented with a question if I’d like to upgrade. No, I’d actually like to do the thing I started the app for. Edit the file. Get off my lawn.

Not sure if it is to justify the developers own existence, but to me, more versions signal more problems. Once a quarter should be enough. Pushing betas out to users and letting them test it for you is lazy. Be your own first user and find the rough edges first.

I’ll just end with the concept of a Blackhole. Features like inboxes, random feeds, etc that only serve to boost engagement are the death of product.

Please click next page to continue reading…

The image in this post is: By Ute Kraus, Physics education group Kraus, Universität Hildesheim, Space Time Travel, (background image of the milky way: Axel Mellinger) - Gallery of Space Time Travel, CC BY-SA 2.5, https://commons.wikimedia.org/w/index.php?curid=370240

April 2023 Update

Another few drafts drafted, still no posts. I figured it was time just to write something.

It is finals week and I’m watching the final lecture of my Digital Signal Processing (DSP) class and it is on doing Discrete Fourier Transform in Matlab.

I still have to write a few audio filters and do a test for my Principles of Engineering Mathematics course. One more week…

This summer I am taking it “easy” and only taking one class Very Large Scale Integration (VLSI) class which is something I’ve always wanted to take. This is the book: https://www.amazon.com/Chip-Design-Submicron-VLSI-Simulation/dp/053446629X although we are using the 2nd edition which I had to get through the university bookstore.

I’m still working while doing this Masters and I have had to do serious time management to not get fired and still get A’s 🙂

Hopefully I come out unscathed from the incoming layoffs.

Around the house, the garden is booming and I am still planning to build an outdoor kitchen. The shop is still a mess and the dust collection system is half-way done. More to come on this.

I’m developing quite the guitar pedal habit also… still trying to figure out my signal chain.

AR4 Robotic Arm Build

In this post I will be summarizing the build of the Annin Robotics AR4 Robotic Arm. Last semester I started my Masters in Science from University of West Florida and my focus is in Robotics. In the Introduction to Robotics course the student must build or procure an arm and perform some tasks with it.

I’ve built a toy arm before and I decided it was time to build a “real” arm. The AR4 is quite an investment, but the results of millimeter repeatability and a 2kg payload make it an excellent kit if you have the skills and time to build it.

Unsurprising the build of this robot took the most effort. Currently the arm is sitting in garage awaiting me adding another axis and building a CNC table…

Sorting M&Ms with Vision

Pipe Cleaner 2023

Hello. Long time, rest assured I am well.

My drafts keep growing, but my public posts stagnate. Let me just push something out.

It has been about 2 years since I moved back to Florida from Seattle, quit posting on social media, and seriously tried to pickup some hobbies outside of computers.

Last fall I enrolled in a Masters of Science program at UWF and after this semester I will be about half-way done. I’ve been quite happy with it so far. Relearning and applying all that calculus I nearly forgot. Still have the day job and we have finally shipped the software that inspired many of the posts in the past.

Time will tell if we made the right decisions, but so far everything is faster and we are able to extend and build confidently.

Many books have been added to my bookshelf. Most have been read but some still await.

Next post I plan to write a bit about building the AR4 robotic arm…

AddRoom Code Breakdown for AOL

In a previous post, I teased that I would explain exactly what is happening in what I believe was the most copied function of the AOL Proggie era. Adding the list of people in a chatroom to a list in your program. This is that post. Fire up your BASIC editor and open up your Win32 API manual and follow along.

Let us start with the include from the Win32 API. Visual Basic (VB) is somewhat of an interpreted language, but it has the ability to include function calls to Dynamically Linked Libraries (DLLs) written in C or another system language.

Key Win32 APIs that are imported and used in dos32.bas from various .DLLs

VB “compiles” to P-CODE and then is interpreted by the runtime or virtual machine of sorts. Similar to Java’s JVM and its bytecode. In VB5 there was a native compiler for x86 machine code, but before VB5, and maybe after, you could “decompile” the P-CODE and get actual source of any program. People got upset at this and there was an entire obfuscation scene dedicated to preventing decompiling of VB programs…

Image result for fate x aol
A good programming team

Interacting with AOL by reading dialogs, clicking buttons and other automations was fairly straight forward. The general flow would be: identify the window, studying its hierarchy, find the control, and send it messages.

To do this, even today in 2021, I like to use Patorjk’s Tool for Window Spying, but you could always use spy++.exe included in the tools from Microsoft. Automation of this style is similar to scraping websites now. You are reverse engineering and poking into stuff you normally shouldn’t. With most scriptable apps you have an official API. For progs, you needed to invent your own API or SDK… by essentially inspecting the window “DOM”..

Image result for spy++

Most controls could be read and written too. But there was one control that alluded us. The list of people in the chat room was some type of special control. There was some formatting that would happen and other things that prevented a standard LB_GETTEXT from working.

Image result for aol chat room
TOXIC GAMER BROS – AOL 4.0? Running on Windows 98?

In the earlier Sk8er.bas there was a call to a “311.dll” which took care of the AOL16 bit. But when 3.0 came out, which was 32bit we had to reinvent the wheel. Not sure who wrote the 311.dll, maybe we could decompile it. It probably was reverse engineered… But I know it held something that is similar to the code in question. Food for thought: Why was it in a .dll?

Don’t make globals, always use Option Explicit 🙂

Ok onto the 32 bit version of this code. There are 2 main variants, but they all do the same thing. Tap into the process, read some memory and do some magic.

dos32.bas’s AddRoom Implementation

This dos32.bas was claimed to be completely written from scratch, it was… except for the above bit. Still searching for the original author. There is actually another variant that I saw in alpha32.bas, it is shorter and simpler, but I will focus on the tear down of dos’s copy pasta.

Code Review

1122-1132

  • Turn off error handling with On Error Resume Next (think of it as a blind try/catch on all exceptions and keep on trucking… not great, but hey)
  • Find Chat Room Window, Get Thread and follow to Process ID (seems roundabout way, was there multiple threads in Aol32.exe?)
  • You can see there are old-school postfix variable identifiers, I think this kept code smaller w/ Option Explicit.
  • VB4 straddled both 16bit and 32bit so I am guessing that is why the integers are defined as “long” here at 32 bit. Long to me is 64bit, but 1995…
Some Copy Pasta’d Knowledge. Thanks David!

1133-1137

  • Biggest mistake I think in this entire snippet, Iterating over a list of items while the list can change.
  • It cannot be atomic I guess, we cannot lock it, but it seems very strange to call LB_GETCOUNT every loop iteration.
  • Send message and get the LB_GETITEMDATA, then pad by 24… No idea perhaps that was formatting data? This listbox had embolden and italic modes for enter/leaving.
  • ReadMemory from ScreenName string which is only 4 characters, must be an address/pointer
  • psnHold now holds the value stored at the address stored in ScreenName
Source is Pointer to Address in ScreenName String of 4 Chars so 32bit Addy

1138-1142

  • psnHold now holds an address. Pad Address by 6… again no idea why.
  • Resize ScreenName to 16 characters (largest screen name possible on AOL)
  • Read Memory into ScreenName
  • Last Step: Trim to null character, remember we use c-strings in VB!

Epilog

Wow, we got through it. Still a lot of unanswered questions, but I think we solved most of the important bits. When I originally read this code in the 90s, it blew my mind. I just chalked it up to magic and copy and pasta’d it. It always stuck in the back of my mind though, and after I learned about memory and pointers in college it seemed to make more sense.

I promised myself I would eventually write this post to shed some light on one of the most important years of my life. What lessons are learned while you are having fun and sharing your code with friends. Thank you for reading and remember… If you don’t understand it, you don’t always need to! Copy Pasta!

The War Room

It keeps going…

Cosmic Rays – Extragalactic Objects

Determinism is an important part of computing.

2 + 2 = 4

Knowing that the computation of the above never changes gives the programmer the ability to reason about there machine in a scientific way. Two plus two is always equal to four!

Or is it?

With our molten iron core spinning within the earth, it creates a giant electromagnetic (EM) field that shields the planet. Very high up there, there is a cloud of stuff that is rejected and deflected from entering our atmosphere and ultimately us and our devices.

Mostly sunburn is the only problem coming from the heavens toward us, but there are other things to “look” for as well. Cosmic Rays are really charged particles believed to come from the stars exploding…

 In 1929, Bothe and Kolhörster discovered charged cosmic-ray particles that could penetrate 4.1 cm of gold.

Wikipedia 2020 🙂

So what happens when one of these particles hits a semi-conductor like your RAM, CPU, SWITCH?

Bad stuff. It could flip a bit.

2 + 2 = 5

This could be a problem. Flip the right bit, you have a blue screen of death. Not a big deal to an entertainment device or non-mission critical systems. Very big deal for a self-driving car, or a power plant control and monitoring system.

Shielding can help, but shielding is expensive. I wonder how well those NeXT cubes with the magnesium cases do for shielding.

Lockstep Computing…