Building Your Own Lightsaber

If you can find a piece of software that does what you need in a timely manner and is within your budget, then by all means use it and move on.  But sometimes you need a feature or a flow that can’t be bought for any cost.  You must roll your own.

I’d run for the hills if you can’t stand computers, because the only thing worse than computers is programming computers.  But if you have the muster and the gumption you to can make the computer go beep.

Chances are there are ways to make it go beep without you writing your own software but what is the fun in doing that?

Automation

The Tool Makers Dilemma…

Sometimes though, you just build to build.  This is very, very dangerous.  It is the state that I am in right now with the Session Link Engine.  I’ve learned and forgotten the web stuff a few times since I started with it in the late 90s.

The purpose of this build is to relearn the web and ship a web app.  So what do we make?  A modern wiki: https://sessionlink.herokuapp.com/

 

 

Session Link 1.0

This was never meant to be a single player game… Perhaps the biggest change in humanity over the past 200 years is the ability to communicate over the wire or the air thousands of miles away instantly.

The Hive Mind is a real thing and is buzzing with all of your data.  The aggregate.

I’ve been working on a proof of concept prototype for multi-user editing.  Co-design if you will.  Essentially a google docs shared cursor:

 

session_link_cursor_1p0.gif

This has been my side project for a few months and it uses Typescript, React, and WebSockets.  Beware that there is no privacy features yet. So be good.

https://sessionlink.herokuapp.com/?id=05efce93-4840-4c27-beac-bf8b181b9875

Open it up in two+ different browsers/machines etc 🙂

Standback google docs.

I will do a walk-through of the source code soon.  Here are some shots during development.

We still have callback hell here with my commando function.  It is passed into every block and is the way that everything updates.  Not sure if I will keep this but it works.

Screen Shot 2019-12-13 at 5.41.11 AM.png

Main render from “Playbox” the root component of app.  I have a file drop that isn’t being used yet.

Screen Shot 2019-12-13 at 5.45.21 AM.png

The kids these days call it hydration/dehydration instead of deserialize/serialize:Screen Shot 2019-12-13 at 5.42.03 AM.png

 

On the todo list:

  • Private Channels w/ XOR Key
  • JSON Deltas not full blobs for transmission of edits
  • Throttling server/client
  • Usernames/Colors not random
  • Actually Reseeding the data to new users
  • Collision Avoidance Schemes w/ Time travel
  • File Sharing
  • Markdown Editing
  • DataTable Editing

 

Stay tuned.

 

 

Red Dawn or Blue Sky 4

Breathing life into an old unknown codebase is one of the most exhilarating and most often a frustrating endeavor a software engineer can partake in.  Duct-tape programmers as someone once said, are a necessity.  The question I always ask myself:

Is it easier or faster just to rewrite all this shit?

Depends on where you are in your journey.  Knowing when to add another layer of lasagna vs replacing 5k lines of code with 3 python functions and no dependencies is the real key.

Baggage or Luggage?  Baggage you wish you could leave behind.  Only bring the luggage.

rm -rf

 

When GUIDs Collide

What happens when 2 identifiers or keys collide?  Bad stuff.  Databases originally used integers for their record identifiers.  If you wanted to add a new record you must query or ask database what was the last integer and add one to it.

This is fine in isolation, but what happens if you want to insert a bunch of records into a remote database that you can only connect to once per hour?  Oh and there are multiple users.  You can’t use Integers anymore you must use something like a UUID/GUID.

Globals Unique Identifiers (GUID) or Universally Unique Identifiers provide a “random” 128bit value that looks like the following:

https://sessionlink.herokuapp.com/?id=c42db039-ba38-4ce8-a5c0-c997894bcade

They work great, and once you switch from Integers to GUIDs your database is truly distributed and can be updated on the moon!  They are a bit ugly though.  Also they take a huge amount of storage compared to the 64 bits of an integer.  Data size matters sometimes.  Wait, GUIDs are 128 bits!

But still we have a slight probability of generating or guessing the same ID with the birthday problem.

Screen Shot 2019-10-27 at 6.42.43 AM.png

If you are FAANG then you have to start worrying about stuff like this.  There will be collisions.  What happens if you add another GUID or two, increasing the bits to 512?

So there is still the GUID Soup to consider…  Start putting multiple GUIDs in an URL and it is pretty gross.

Enter CUID, which has some interesting objectives:

Collision-resistant ids optimized for horizontal scaling and binary search lookup performance.

Screen Shot 2019-10-27 at 6.29.57 AM.png

How do they do this?  with a string 25 characters.  Timestamping is critical to its binary search capabilities.  Ordering by timestamp allows the lookup to be logarithmic.  So if you need to find one record in a billion.  It would take about 30 tries to find it.  Sorting is key to speed here.

If you need speed and no creation time privacy CUIDs may be a viable option.  Anonymous systems must never use CUIDs.

Only slightly larger in storage size than a GUID… 25 Characters * 8 Bits/Character = 200 Bits.  Guessing CUIDs I feel would be easier…

Screen Shot 2019-10-27 at 6.50.42 AM.png

I’ll stick with my GUIDs soup for now.

Until Next Time. Happy Merging Distributed Datasets,

Rob

 

P.S. Microsoft trying to brand it, er recapitulate:

Screen Shot 2019-10-27 at 7.12.13 AM.png

Duck Typing

If it walks like a duck and talks like a duck it must be a duck.

Recently I started a project with Pure “Vanilla” Javascript, and it was quick going.  With the React as the starter pack, I had a basic prototype with proof of concept multi-user editing: https://twitter.com/robmurrer/status/1169118440122015744

That was a month ago and I ported it to Typescript which is a superset of Javascript (JS) that “compiles” to JS.  It plays well with React’s JSX becoming TSX.  Tip to cast in Typescript:

 blockchain = await this.state.store.getItem(id) as BlockProps;

To me, having to do the work of a compiler, is maddening and that is why I actually really love Typescript (TS).  Coming from any traditional language it is a nice safety cushion under you.

TS has excellent tooling to enable confident refactors, which is the number one issue with my JS codebase.  Refactoring “blind” in pure JS is nightmare.  Refactoring is by far the most important task, so I’ve gone full TS.  But I can see if you are just learning JS you should absolutely not start with TS.

Start pure Vanilla JS and build something. Then when you get stuck trying to refactor.  Revert and start fresh with TS.  I like the quick rewrite and throwaway approach for rapid prototyping.  JS is the best rapid prototyping environment I’ve ever seen.

Ok on to code… The most important part of an editor is the data the user enters.  You cannot lose it.  We are using the Local Forage package from the folks at Mozilla for the data storage layer.  Under the hood it will use IndexedDB on modern browsers.

Screen Shot 2019-10-21 at 6.59.49 AM.png

Ok so what are we looking at here?  The genesis for the post.  What the heck is Hydrate and Dehydrate?  It’s just the cool term for serialize and deserialize.  I like it. The whole point of these names is to communicate what it is they do.

Update:  Um it writes the data in a format it can send over wire or store in database.

But there is something about naming it what it has been called in the past.  Hence this post.

The tour de force of programming design patterns:

  • Finite State Machine (FSM)
    • React’s fundamental concept
    • Start here!
  • Command Pattern
    • Ahem…
  • Classes
    • Only to hold state!!!
    • Of course React uses inheritance 😀
  • Composite
    • Just a tree, or
    • Now with more Blockchain

I’ll write/link up some more about the above at some point.

Until next time, Happy Coding!

 

Screen Shot 2019-10-21 at 7.38.09 AM

Coded into a Corner

Creating software is easy, creating software that is sustainable to maintain and extend is hard.  There are not many mediums in which the consequences of a simple change or design decision can render the complete system useless.  The way in which the software is built is just as critical as to who is building it.

Honey why is the toilet flushing now when I flip the light switch?

Whoa! Cool! I didn’t realize changing the light bulb would do that.

Poorly designed systems are interconnected in ways that are not necessary.  It is often easier to write the prototype in this manner, but products often need to be refactored to use time tested design patterns.

If you ever find yourself scared to change something or can’t figure out how to add the next killer feature.  It is certainly time you reevaluate what you have and don’t be afraid to delete your code.   You may be surprised how much of it is no longer needed.

More code, more bugs!

Recommended Reading

61iOSX4WNiL._SX404_BO1,204,203,200_.jpg

 

The Weekend Thrash

So you think you are almost there. You think one or two more hacking sessions could “do it” and you start the weekend thrash.

What is the difference between a clever hack and an unmaintainable pit of tar?

If you are in the Zone, then you of course nothing can stop you.  But I say stop and rethink your life.  You probably are building the wrong thing anyway.

I don’t roll on Shabbos!

 

walter-sobchak.jpg