Make Yourself

Owning your whole tool chain is important part of becoming a seasoned developer of quality software.

Building the software is the first step. We all love to use Integrated Development Environments (IDE) but what happens if you lose license to that suite of tools?

Your developer ergonomics are important and it is why we use those heavy weight tools like Visual Studio, the best IDE ever invented.

How would you even start rolling your own development toolchain with “free as in beer” tools?

Little Black Books

When you are cook aspiring to become a chef, you are expected to write down notes.

These are your notes, they go home with you.

When you quit or get fired. You keep the book.

In tech, it is not like that ūüė¶

We have one way to exfiltrate some of the sauce. “Open Source”

The post that inspired this post: https://web.archive.org/web/20091121122743/http://www.onehungrychef.com/2009/11/on-hand-me-downs.html#more

I miss that blog.

I ‚̧ @internetarchive

The Speed of Thought

There are two types of software.

Those that you choose to use, and those that your boss chooses for you.

hmm

The “only way to do it” is sometimes painful. But if there are dollars to be made between those button clicks and some frustration, I will happily click them. Hopefully in the right order.

If I have a choice though, I will choose the program that keeps me in the “zone” for as long as possible. Every second I feel like I am waiting on the machine; is one more second I’m likely to change tasks or get distracted from what I’m actually in your program for.

There is some number… of millisecond lag tolerance. This is the most important metric and a balancing act with the value you are creating. How much of the program’s feature plays in your cash generating mechanisms? How long am I willing to wait? Remember I’m not just a user of a novel feature in your app. It is part of an entire flow that you will never see.

In the craft of turning dirt to dollars, I focus mostly on Electronic Design Automation (EDA). The vast amount of tooling surrounding modern Semi-Conductors and PCBs are staggering (and expensive). Unifying these tools for the usage of mere mortals has been my task for the past 5 years.

To me the task of the next 10 years is to optimize this flow to take seconds or minutes… Rather than years. Who will be a blackbox and who will be sherlocked?

The Mind Compiler

Reading code is the most important activity of a computer programmer. Ideally the code is then interpreted and ran as a thought experiment inside the programmer’s brain. This sounds hard, but as you learn an environment, the mind will eventually reach this level of tooling.

The ability to reason about code without running it, is the key to failing fast and often. Which you always want to do. Failing slow is painful. Just as Intellisense is a huge help for catching syntax errors early, not having to actually build & run your program, you can iterate much quicker.

Of course what happens if your internal transpiling mechanism and logic simulation units fail as you bite off more than your meat computer can handle?

Just press F5.

I recommend picking your favorite open source project and start reading. If you get stuck, just start dropping some breakpoints and calibrating your internal toolkit.

Big Idea Fridays

There are few speeches that cause me to read them multiple times. Dr. Hamming’s “You and Your Research” resonates deeply with me and is one of my favorites. Hamming, defines a work ethic and sprinkles the talk with antedotes that are both kind and relatable.

However you define great things

Dick Hamming

He gave this speech many times and prefer this written version: https://www.cs.virginia.edu/~robins/YouAndYourResearch.html

I like the idea of reserving Friday’s for thinking about deep problems.

Have a good weekend.

Building Your Own Lightsaber

If you can find a piece of software that does what you need in a timely manner and is within your budget, then by all means use it and move on. ¬†But sometimes you need a feature or a flow that can’t be bought for any cost. ¬†You must roll your own.

I’d run for the hills if you can’t stand computers, because the only thing worse than computers is programming computers. ¬†But if you have the muster and the gumption you to can make the computer go beep.

Chances are there are ways to make it go beep without you writing your own software but what is the fun in doing that?

Automation

The Tool Makers Dilemma…

Sometimes though, you just build to build. ¬†This is very, very dangerous. ¬†It is the state that I am in right now with the Session Link Engine. ¬†I’ve learned and forgotten the web stuff a few times since I started with it in the late 90s.

The purpose of this build is to relearn the web and ship a web app.  So what do we make?  A modern wiki: https://sessionlink.herokuapp.com/

 

 

Session Link 1.0

Update Summer 2020:

This was never meant to be a single player game… Perhaps the biggest change in humanity over the past 200 years is the ability to communicate over the wire or the air thousands of miles away instantly.

The Hive Mind is a real thing and is buzzing with all of your data.  The aggregate.

I’ve been working on a proof of concept prototype for multi-user editing.  Co-design if you will.  Essentially a google docs shared cursor:

session_link_cursor_1p0.gif

This has been my side project for a few months and it uses Typescript, React, and WebSockets.  Beware that there is no privacy features yet. So be good.

https://sessionlink.herokuapp.com/?id=05efce93-4840-4c27-beac-bf8b181b9875

Open it up in two+ different browsers/machines etc ūüôā

Standback google docs.

I will do a walk-through of the source code soon.  Here are some shots during development.

We still have callback hell here with my commando function.  It is passed into every block and is the way that everything updates.  Not sure if I will keep this but it works.

Screen Shot 2019-12-13 at 5.41.11 AM.png

Main render from “Playbox” the root component of app.  I have a file drop that isn’t being used yet.

Screen Shot 2019-12-13 at 5.45.21 AM.png

The kids these days call it hydration/dehydration instead of deserialize/serialize:Screen Shot 2019-12-13 at 5.42.03 AM.png

On the todo list:

  • Private Channels w/ XOR Key
  • JSON Deltas not full blobs for transmission of edits
  • Throttling server/client
  • Usernames/Colors not random
  • Actually Reseeding the data to new users
  • Collision Avoidance Schemes w/ Time travel
  • File Sharing
  • Markdown Editing
  • DataTable Editing

Stay tuned.

Red Dawn or Blue Sky 4

Breathing life into an old unknown codebase is one of the most exhilarating and most often a frustrating endeavor a software engineer can partake in.  Duct-tape programmers as someone once said, are a necessity.  The question I always ask myself:

Is it easier or faster just to rewrite all this shit?

Depends on where you are in your journey.  Knowing when to add another layer of lasagna vs replacing 5k lines of code with 3 python functions and no dependencies is the real key.

Baggage or Luggage?  Baggage you wish you could leave behind.  Only bring the luggage.

rm -rf

 

When GUIDs Collide

What happens when 2 identifiers or keys collide?  Bad stuff.  Databases originally used integers for their record identifiers.  If you wanted to add a new record you must query or ask database what was the last integer and add one to it.

This is fine in isolation, but what happens if you want to insert a bunch of records into a remote database that you can only connect to once per hour? ¬†Oh and there are multiple users. ¬†You can’t use Integers anymore you must use something like a UUID/GUID.

Globals Unique Identifiers (GUID) or Universally Unique Identifiers provide a “random” 128bit value that looks like the following:

https://sessionlink.herokuapp.com/?id=c42db039-ba38-4ce8-a5c0-c997894bcade

They work great, and once you switch from Integers to GUIDs your database is truly distributed and can be updated on the moon!  They are a bit ugly though.  Also they take a huge amount of storage compared to the 64 bits of an integer.  Data size matters sometimes.  Wait, GUIDs are 128 bits!

But still we have a slight probability of generating or guessing the same ID with the birthday problem.

Screen Shot 2019-10-27 at 6.42.43 AM.png

If you are FAANG then you have to start worrying about stuff like this.  There will be collisions.  What happens if you add another GUID or two, increasing the bits to 512?

So there is still the GUID Soup¬†to consider… ¬†Start putting multiple GUIDs in an URL and it is pretty gross.

Enter CUID, which has some interesting objectives:

Collision-resistant ids optimized for horizontal scaling and binary search lookup performance.

Screen Shot 2019-10-27 at 6.29.57 AM.png

How do they do this?  with a string 25 characters.  Timestamping is critical to its binary search capabilities.  Ordering by timestamp allows the lookup to be logarithmic.  So if you need to find one record in a billion.  It would take about 30 tries to find it.  Sorting is key to speed here.

If you need speed and no creation time privacy CUIDs may be a viable option.  Anonymous systems must never use CUIDs.

Only slightly larger in storage size than a GUID…¬†25 Characters * 8 Bits/Character = 200 Bits. ¬†Guessing CUIDs I feel would be easier…

Screen Shot 2019-10-27 at 6.50.42 AM.png

I’ll stick with my GUIDs soup for now.

Until Next Time. Happy Merging Distributed Datasets,

Rob

 

P.S. Microsoft trying to brand it, er recapitulate:

Screen Shot 2019-10-27 at 7.12.13 AM.png