Month: August 2024

Perpetual Beta

Welcome to the world. You are a beta tester. No, you do not get an employee discount. Thanks, keep moving. How dare you not run the latest version of my software.

Gold Masters

We used to have to have real discipline with shipping software. We didn’t ship docker containers, we shipped real containers! If you missed a date, it wouldn’t hit Toys-R-Us by Christmas. You missed the wave.

The Mediums Available Now

  • HARD COPY OLDEST TO NEWEST: FLOPPY, DVD, USB STICK
  • APP STORES
  • Web Apps
  • Website Download: Native Software (.exe,.app, .msi)
  • jithub dot com
  • tar.gz (zip files)
  • Dockerfile

As a developer we are already on the hub so we go there. But for mere mortals, App Stores have taken the cake. I will not talk about Docker in this post. As a consumer I prefer web downloads native to my platform or web apps.

App Stores

The walled garden is appealing. It provides you a platform and some level of competence or use to. They are gated by the who own the store. But this is reversing in trend I think***

The assumed blessing gets you eyeballs and is the only way to exist and use the hardware to its full potential on most platforms. You think the billions of iPhones in the world belong to us? They belong to Apple. We are leasing them!!!

The Trap

Apple, Google, etc now control when your users get your update. It is a friction source between you and your customer. They can just flat out refuse. Imagine trying to contact IT for a company will millions of customers. I think this problem is just now being identified. This type of relationship has never existed outside of government census style forced participation.

But it is still the ONLY WAY for some things.

If you would like to save actual data for sure on someones phone. Create a native app. Web apps will lose data cause Safari is a brutal target.

APT

Advanced Package Tool is how we manage our software on Debian based Linux systems (ubuntu). It is quite good and Homebrew on Mac is a poor imitation**

I'll provide you with information about the invention of the APT package system and its initial deployment. However, I want to note that I don't have access to a live database or search function, so I may not be able to provide completely accurate citations. I'll give you the information to the best of my knowledge, but I recommend double-checking these details for full accuracy.

The APT (Advanced Package Tool) package management system was primarily developed by the Debian project. It was first introduced in Debian GNU/Linux.

The primary developers of APT were:

1. Jason Gunthorpe
2. Brian White
3. Ian Jackson

APT was first released in 1998 as part of Debian 2.1 (codenamed "Slink").

While I can't provide specific citations due to the limitations mentioned earlier, you can find more detailed information about APT's history and development in the following sources:

1. The Debian Project's official documentation and release notes for Debian 2.1.
2. Historical mailing list archives of the Debian project from the late 1990s.
3. The APT project's own documentation and changelog.

For the most accurate and up-to-date information, I recommend checking these sources directly or consulting more recent literature on the history of Linux package management systems.

-CLAUDE VAN DAMME

Sometimes the links get broken and it can’t even find a python3.11-venv 😀

I started with Red Hat Linux 5 in 1996 and I think I can say with a lot of confidence. APT is what made Debian and then Ubuntu into a player in the Linux distribution game. APT can be your own app store because you can add repositories to it. If you are ever having a problem finding a package in apt. make sure you do this a few times:

sudo apt upgrade && sudo apt update

Building from Source Aside

::Arch, Gentoo has entered teh chat::

So one of the rights of passage in old linux days was recompiling your kernel. It used to be necessary before they added json parsing for device trees and configurations. Let’s just say you’d make a small change…

Well, first download the source of the kernel. Hopefully it came with your distro. make, make install… a few sudos. I wouldn’t be surprised if it was still about the same process. Seems like a pain though. Hours to compile on a 386sx with no floating point coprocessor or whatever. Cycles were sparse and bits where short.

Why?

for fun, for curiosity, for hopefully profit because YOU can do something no one else can unless they make your modifications.

CPU Platforms

Shipping fat binaries is annoying sometimes so that is why if you are bespoke hardware you build from source. You can still build vim for the Amiga!

  • x86
  • AMD64
  • ARM64
  • POWERPC*

Building from source is often triggered by moving to a new platform or wanting to use cutting edge compiler technologies.

Web Downloads & Native Software

This is my preferred method of distribution.

This is the only way to truly control your destiny. Ideally you have a web-version of the application and then a downloadable offline version. Going native is always the best solution for platform integration. But if you have multi-platform dreams you again put another vendor between you and your customer.

I’m looking at you Xamarin, Flutter, Qt, React Native

Continuous Improvement (BUILD, TEST, DEPLOY)

Servers have always been the method in which we prepare our software for distribution. Producing reproducible build artifacts based on time and version is critical when tracking down hard to find bugs that get introduced to the system.

To me the largest benefit of running a “build server” is that every commit to master or main will get built. And if it doesn’t compile on the server. The developer forgot to commit something!!!

It is a great test of “it works on my machine, does it work on yours?”

Testing

  • Optically (not automated but ideally a 4 eye check)
  • Unit
  • Integration
  • Performance

Release Strategies

I just want to talk about versions here. Releasing software is complicated. Who is this update for? Is it a test that you can’t test yourself so you need custom release for 1-2 people only?

ALPHA

  • Developer only and maybe a short loop customer that needs to test something.
  • This should not be used in production of possible…

BETA

  • Ready enough for general use for testing before a “release”
  • Easily generated through Jenkins or some CI/CD server

Date Based Versioning

Preferred method.

  • 2024.08.30.BETA
  • 2024.07.20.ALPHA
  • 2024.04.19
  • 2023.12.31.BETA

Semantic Versioning (SemVer)

I would be remiss not to mention this. But yes there is this scheme out there that hopes to encode version numbers in W.X.Y.Z format where these letters mean something. I don’t care to go into this, you are pushing the problem to someone who can make mistakes and SemVer’s LIE ALL THE TIME.

Pokemon Versioning

  • With a solid BETA version you tag it with a cute name
  • 2 words normally chosen from a 2 bags
  • Fitting a theme

Look at ROS, Ubuntu, etc

Examples:

  • Humble Hawksbill
  • Icecream Sandwich
  • Stroppy Titan

Stuck in Beta

You can call the version whatever you want. Users know when it is beta, you can feel it.

With your Jenkins about to build, test, and deploy your software with a single click. You get stuck in this release every-week.

You can do this. But unless you are web app where the software updates automagically. Don’t do this. Users do not like their sofware to change too often.

Try to release a beta once a quarter. And do a solid release 2 times a year if possible. Write up a nice changelog and provide a reason for your users to update. Don’t force them!

Give your customer a choice which version of the software to run!

The Command Pattern [BETA]

You would figure that the namesake of this website would have a post about it. Welcome to the Command Pattern. I first read about it in the famous Gang of Four book [Gamma et al]

Notes

This design pattern is useful to implement scripting and undo/redo behavior.

Updates:

Prerequisites

Explanations & Definitions

If I had to describe it to another programmer or technically adept person, I would say its just functions. Functions all the way down. But the special part is how you handle state and the ability to undo state changes. Or, script it!

Let us let the machine try and explain it:

Imagine you’re playing with a toy robot. Instead of controlling the robot directly, you have a special remote control. Each button on the remote is like a “command” that tells the robot what to do.

When you press a button, the remote doesn’t actually make the robot move. Instead, it sends a message to the robot saying “do this action”. The robot then follows that instruction.

This is kind of like how the Command pattern works in computer programs. Instead of one part of the program directly telling another part what to do, it creates a “command” that can be sent and followed later. This makes it easier to add new commands, undo actions, or even save a list of commands to do later.

– Claude3.5 Sonnet

Implementation Motivation

This is my implementation, there are many like this but this what I would consider an easy to teach and implement solution. You can always add features, ERRRR… complications, later 🙂

State or “World”

State is part of most developer’s lives, but really you can think about this as the data you care about in the program. It could be your air-line reservation, forum post, baby photos. You don’t want your data to be lost. State is your data or world, and often like the real-world the state is hierarchical.

Mutable World or Immutable World?

I’ll skip right past the mutable world. It is useful for large simulations, but that is another topic and I will leave this for another moment. Sometimes the memory overhead actually matters, because you cannot afford a machine with that much RAM. If you have limited hardware, consider a hybrid-mutable-immutable-world.

I consider Immutable World the cleanest approach and most ideal for scripting.

The Immutable World

new_world = get_next_state(world, command)

So it is an agreement that any time the “World” will be mutated, we make a copy and return new state. We never mutate state.

We will also keep all of our state objects in a stack linked to their commands that have been performed.

all_states = [] # list of all worlds ever

Function Interfaces

Finite State Machines (FSM) are very important for system design. In this scenario we will ensure to call our functions on our state/world objects to maintain consistent design. From this point on we will refer to world as state as it is the preferred term of the author.

It helps to have an actual problem to solve to teach the command pattern. We will start with the humble calculator.

Command Processor

# CRL LAB - COMMAND PATTERN - A
# COPYRIGHT 2024 CRYPTIDE RESEARCH - ALL RIGHTS RESERVED
# LICENSED UNDER GPL3

# generic processor that is slightly tuned for us to teach
class CommandProcessor:
    def __init__(self, config):
        self.config = config
        self.history = [config.get_default_cmd()]
        self._value = config.get_default_value()

    def exec(self, op, a, b=None):
        self._value, cmd_link = self.config.exec(op, a, b)
        self.history.append((self._value, cmd_link))

        return self._value

    def clear(self):
        self.history.clear()
        self.config.clear()

    def undo(self):
        if len(self.history) == 0:
            raise ValueError("No operations to undo")

        undo_value, undo_cmd = self.history.pop()

        new_value, _ = self.history[-1] #look at last one and get value
        self._value = new_value

        #this following approach is half working for side-effect based systems example
        #new_value = self.config.undo(self.history[-1])
        #but what else do we need?

        return self._value


    def value(self):
        return self._value

Note that this processor implementation is just a starting point. The core ingredients. As we develop our application we may dream up new features of the interfaces.

You can note that there is no specific implementation baked into the design. It is meant to operate as a shim or a harness to run specific state/command patterns.

We have a configuration object passed in which contains the domain specific code we will be wrapping in the façade. A history list and helper value_history to aid in debugging.

This is the core function, this executes a command. a, b are arguments to the command. op is the operation to be performed.

History is preserved and we return the result of the execution is returned for easy of use of the API.

Calculator Commando

from enum import Enum

# simple calculator command processor
class Calculator:
    def __init__(self):
        self._value = 0

    # for referencing the payload
    class CALC_ROW(Enum):
        OP = 0
        A = 1
        B = 2
        RESULT = 3

    # public interface
    def exec(self, op, a, b=None):
      print(f"executing {op} {a} {b}")
      coms = self._get_command_map()
      if op not in coms:
        raise ValueError(f"Invalid operation: {op}")
      self._value = coms[op](a, b)

      return (self._value, (op, a, b, self._value))

    def value(self):
        return self._value

    def undo(self, tuple4):
        op, b, a, _ = tuple4 # pull arguments in reverse order!!!

        if a is None:
            a = self._value

        self._value = self._get_undo_map(op)(a, b)
        return (self._value, (op, a, b, self._value))

    def reset(self):
        self._value = self.get_default_value()

    def get_default_value(self):
        return 0
        
    def get_default_cmd(self):
        return (self.get_default_value(), ('+', 0, 0, 0))

    def clear(self):
        self._value = 0

    # private implementation

    def _add(self, a, b):
        if b is None:
            b = self._value
        return a + b

    def _multiply(self, a, b):
        if b is None:
            b = self._value
        return a * b

    def _subtract(self, a, b):
        if b is None:
            b = self._value
        return a - b

    def _divide(self, a, b):
        if b is None:
            b = self._value
        if b == 0:
            raise ValueError("Division by zero is not allowed")
        return a / b

    def _get_command_map(self):
      return {
          '+': self._add,
          '*': self._multiply,
          '-': self._subtract,
          '/': self._divide,
      }

    def _get_undo_map(self, op:str):

          switch = {
              '+': self._subtract,
              '*': self._divide,
              '-': self._add,
              '/': self._multiply
          }
          return switch[op]

This is the domain specific part of the code. It would change depending on the task that the application programmer might need. Example usage of the latest code:

base_calc = Calculator()
x0 = base_calc.exec('+', 1, 2)
print('using implementation itself')
print(x0) # (3, ('+', 1, 2, 3))
print(base_calc.value()) # 3
x1 = base_calc.exec('+', 1) 
print(x1) # (4, ('+', 1, None, 4))
print(base_calc.value()) # 4

At this point you might be wondering why we’ve done all this boiler plate. The final details:

# you can see the (value, (op, a, b, value))
# data structure here as an artifact to help with undo
# clear up and do the real Command Pattern

base_calc.clear()
print(base_calc.value())

print('wrap it in command processor')
calc = CommandProcessor(base_calc)
calc.exec('-', 10, 4)
print(calc.history[-1])
print(calc.value())

calc.exec('+', 1)
print(calc.history[-1])
print(calc.value())

print('undo last calc...')
calc.undo()
print(calc.value())
print('undo last calc...')

calc.undo()
print(calc.value())

0
wrap it in command processor
executing - 10 4
(6, ('-', 10, 4, 6))
6
executing + 1 None
(7, ('+', 1, None, 7))
7
undo last calc...
6
undo last calc...
0

Discussion

With the encoding of the inverse mapping of the operations between add/subtract and divide/multiply we can safely undo any operation but applying it in reverse. Not all commando’s will have this luxury. Think if there is a function that creates a file on disk. The undo of that command should do what?

DELETE THE FILE

Ok. we can end here for this section for the lab.

There will be an on-going discussion, but part 2 of this lab will transition to image manipulation and scripting support.

Thanks for reading!