Category Archives: Uncategorized

Cloud Enabled Commodore 64: Part V – Do It Yourself

By now many people saw the demo of the Cloud Enabled Commodore 64 project, read posts discussing implementation and the retrospective and some commented that they would like to try it out themselves. This post describes how to do that.

We will start from listing required hardware and then will move to the required software.

Hardware

There are two hardware options to try the project out – you can either use an emulator or use a real Commodore 64. The emulator route is a bit easier as it does not require a working Commodore 64 and additional peripherals. You will still need a Node MCU board like this:

Node MCU Board
Node MCU board

which you can get on ebay for below $5. If you decide to try it out on a real Commodore 64 you will need a C64 WiFi modem. Make sure it is using the NodeMCU module and that the module is accessible. This is how mine looks like:

C-64 WiFi Modem

For the real C-64, you will also need to be able to load the cross-compiled program to your C-64. There are a few possibilities here – I used an SD2IEC floppy drive emulator, and it worked great for my needs.

Software

Before moving to software I would like to start with a disclaimer. I did all the work on MacOS. I will try my best to provide instructions for Windows and Linux, but they might be lacking.

Git

You will need git to clone the project repo. It is very likely that you already have git installed on your machine but if not follow instructions from here: https://git-scm.com/book/en/v2/Getting-Started-Installing-Git

make

You will need make to build the project and the cc65 toolchain. On MacOS you can get it by installing Apple developer tools. On Linux, you likely already have it. On Windows you would need to install either Cygwin or use WSL (Windows Subsystem for Linux).

cc65

cc65 link is a “cross development package for 6502 systems”. To get it, follow instructions listed on https://cc65.github.io/getting-started.html. Please make sure the tools can be resolved (e.g. run sudo make avail or add them to the path). You can test your installation by running cc65 from command line and verifying that it printed cc65: No input files.

Arduino IDE

We will need to update the NodeMCU board for which we will use the Arduino IDE. It can be downloaded from https://www.arduino.cc/en/software.

VICE emulator

If you are going the emulator route (which I recommend even if you eventually want to use the real C-64) you will need the VICE emulator which you can download from: https://vice-emu.sourceforge.io/index.html#download

Dotnet

You will also need the .NET SDK. It will be used to run the server locally. This makes it easier to test and troubleshoot, if necessary. It is will also be needed if you decide to publish the server to Azure. You can get the .NET SDK from https://dotnet.microsoft.com/en-us/download

Node and npm
The server contains a web client which depends on a few node packages (most notably the @microsoft/signalr package) so you will need npm to install these packages.

Preparing and running the application

With all pre-requisites installed we can get down to business and try to start the application. Here are the steps:

  1. Clone the project repo
    Run git clone https://github.com/moozzyk/SignalR-C64

  2. Start the TestServer
    The test server is the chat server our chat application will be talking to. Note, that the server registers an https endpoint which on will use a developer TLS certificate when running locally. This may result in showing a warning or asking to register the certificate (which you can do by executing dotnet dev-certs https --trust). If you don’t want to see the warning you can remove the https url from this line and just use HTTP. This will work fine for local runs but is not recommended (perhaps not event possible) when running the server on Azure. You will also want to make sure that the server is accessible from outside of your machine (i.e. make sure that other devices on your network can access the application).
    To start the TestServer you need to go (cd) to the TestServer directory and run:
    npm install
    dotnet run

  3. Verify that the test server works
    Connect to the server using a browser. Ideally you would want to connect using an external IP or the name of your machine (i.e. avoid 127.0.0.1 or localhost) or use another device connected to the same network. Once connected try to send a message – if you receive the message you typed, the server is set up correctly. (Note that if you try connecting to the server with HTTPS you may see warnings caused by using the local (dev) TLS certificate.)


  1. Backup the C64 WiFi Modem firmware (optional)
    If you are using the C64 WiFi Modem for this project you may want to back the currently installed firmware up as the next step will overwrite the firmware effectively removing the original functionality provided with the modem (i.e. connect to BBSes). One way to do this is to use the esptool to download the existing firmware and then upload it later to bring back the original functionality. You can install the esptool by running:
    pip install esptool

    To download the firmware connect the modem to your computer and run (remember to update the port to point to your serial device):
    esptool.py --baud 115200 --port /dev/cu.usbserial-1420 read_flash 0x0 0x400000 ~/tmp/C64WiFi-backup-4M.bin

    To upload the firmware back to the board run:
    esptool.py --baud 115200 --port /dev/cu.usbserial-1420 write_flash 0x00000 ~/tmp/C64WiFi-backup-4M.bin

    You can use screen (or Putty on Windows) to test that the firmware has been uploaded correctly. First run:
    screen screen /dev/cu.usbserial-1420 300
    and then type AT? You should see something like this:
    AT?
         cOMMODORE4EVER V2.3 wIFI mODEM
    ...

  1. Upload the firmware to the NodeMCU board
    – Start Arduino IDE and open the EspWs.ino sketch and set the default credentials on this line. (For simplicity, the code running on C-64 does not allow setting credentials – it assumes that the credentials are properly configured and will just initiate WiFi connection.)
    – If you are planing to use real C-64 set the transfer speed to 600 bauds here. Leave 1200 if using the emulator as Vice-64 does not seem to support 600 bauds.
    – Connect your NodeMCU board (or the C64 WiFi Modem) to your computer.
    – Make sure to select the NodeMCU 1.0 (ESP-12E Module) board (If you can’t see this board you may need to add it first using the Board Manager Tools -> Board Manager, search for “esp8266” and then install)

– Select the device

– Upload the firmware to the board

  1. Verify firmware was deployed successfully
    Go to the EspWs directory and run the following command (make sure to provide correct values for the server, device and transfer rate):
    python3 prototype.py 192.168.86.250:5000 /dev/cu.usbserial-1420 1200

    If the firmware has been uploaded correctly you should see the following output:
    b'\x03\x00'
    OK
    b'\x05\x1dws://192.168.86.250:5000/chat'
    WS
    b'Connected'
    b'\x06*{"protocol": "messagepack", "version": 1}\x1e'
    OK
    DATA
    b'{}\x1e'
    DATA
    b'\x02\x91\x06'
    DATA
    b'\x02\x91\x06'

    Note there will be some delay before you will be able to see most of the output as it takes about 10 seconds for the board to connect to WiFi. Another, important thing is that if you stop the script and want to try again, you’ll need to reset the board (press the RST button on the NodeMCU module and wait a few seconds before trying again).
  1. Configure Vice
    This step is only needed if you want to run the app using the Vice emulator.
    Open Vice and go to Settings -> Peripheral devices -> RS232
    Make sure to “Enable Userport RS232 Emulation” and select the device that you want to use. In the RS232 devices you need to provide the device filename and the transfer speed. For the emulator you want to 1200 bauds. Here is how this is configured in my case:

  1. Configure chat server URL
    You will need to set the correct URL to be able to connect to your chat server by modifying the value here.
  2. Build and run
    The application should now be ready to run. Go to the App directory where you will be able to build the application with make. The makefile supports a few targets. The default target (i.e. running make without any arguments) will compile the app to a.prg file. make clean will delete temporary files. make d64 creates a .d64 (disk image) file you can either to attach to the emulator or use to run on a real C-64 (e.g. using SD2IEC). The fastest way to build and run the app on the emulator is to invoke the following command:
    make clean && make && x64sc --autoload signalrdemo.prg
    It will clean temporary files, create a prg file, start the emulator and automatically load the prg. Then you can just type run in the emulator to run the app.
  3. Deploy the server to Azure (or a cloud provider of your choice)
    If the application is working correctly in the local environment you can deploy the server to a cloud provider. You will need to update the URL accordingly and compile with the new settings (steps 8 and 9).

This post concludes the Cloud Enabled Commodore-64 mini series. I hope this project brought back some good memories for you as it did for me.

Cloud enabled Commodore 64: Part II – Development Environment

After devising a high-level design for my idea, I needed to take care of more mundane things, starting with setting up the development environment. First, I had to find out if using a Commodore 64 emulator for testing and development was an option. I wanted to rely on the emulator as much as possible to speed up the development time. The main two concerns I had were:

  • would the emulator even support external devices?
  • if, the above was true and my idea worked on the emulator, would it work on real hardware

Vice-64 was the most advanced and mature Commodore 64 emulator I knew of. I used it previously in some of my projects (e.g. Vintage Studio) and to occasionally play some of my favorite games but never with any external hardware. Looking at Settings gave me hope – I found that there were at least controls allowing to configure serial communication. After playing with the settings for a bit I was able to make Nova Term connect to a BBS via the C64-WIFI modem connected to my laptop via USB. This answered my first concern and assured that I would be able to code and test the entire solution on my laptop with maybe some additional debugging on the real C-64 at the very end. Another benefit of using Vice was access to the built-in debugging tools. They allow setting breakpoints and stepping through the code and the support for labels makes debugging much easier.

Next, I had to decide on which 6502 assembler to use. I found ca65 (a part of cc65 suite https://github.com/cc65) to be the best option. It has a lot of great features, has been around for a long time and has a lot of documentation as well as source code available on github. I was almost sure that VS Code was the safest bet to serve as my 6502 assembly editor as it has a ton of extensions. The main thing I was looking for was syntax highlighting and I quickly found the CA65 extension which did exactly what I was after. The extension also provides build tasks but I have not used them as I decided to go with make. Overall, the combination of Vice, VS Code, ca65 and make ended up making quite a productive environment where I could quickly build, launch and debug my project.

To handle the NodeMCU part I decided to stick to Arduino Studio. I knew that VSCode had an Arduino extension (heck, I even created one long time ago before Microsoft decided to occupy this space) but it tried to reformat my code the way I did not like and I didn’t want to spend the time on adjusting it to my liking. I also think I had some issues with uploading my sketch to the board while everything worked fine from Arduino Studio.

At the beginning I was able to test my NodeMCU code using screen. It was possible because the protocol I implemented was text based. Later, I switched to a fully binary protocol and using any general purpose solution was not an option, so I created a simple terminal in Python that was able to interpret and translate my commands and the responses from the module.

I used a standalone NodeMCU module for most development but occasionally tested my code against the C-64 WiFi’s NodeMCU module. Surprisingly, I found some slight differences between how the boards behaved immediately after booting and had to implement a simple workaround which ignored first few bytes received from the module.

For the SignalR part I just took the chat server I created for my other project. I ran it locally during the implementation and then deployed to Microsoft Azure for final testing and demo.

The final missing piece of my environment was actual hardware. I had a Commodore 64 that I know was working the last time I turned it on because it was when I fixed it. I also had a 1084S-D2 monitor which stopped working when I was testing my C-64. Fortunately, it turned out to be only the power switch. Replacing the switch brought the monitor back to life. I decided to go fully retro and had to acquire a 1541 disk drive – luckily I found a working one in decent condition on craigslist. I received a bunch of disks used with Commodore 64 from a colleague many years ago. Despite all these years in a closet almost all of them worked just fine. The only thing remaining was to transfer the compiled program to a disk which I did using a SD2IEC module.

Looking back I am really amazed how many technologies – both hardware and software – were involved in putting my solution together. Four programming languages, vintage and modern hardware, embedded programing, Cloud technologies and a variety of, mostly open source, tools. All this made this project a lot of fun. Next time we will take a closer look at the implementation.

Hello, OCaml!

I am a huge fan of the Advent of Code. I eagerly wait for it every year and once it starts I try to solve the problems the same day they appear. The nice thing about the problems is that they are initially relatively simple and get more difficult over time (and if you ever get stuck there is a subreddit for each problem that you can use to unblock yourself). This makes it a fantastic opportunity to try new languages. (Note, I intentionally said “try” and not “learn” because the vast majority of the problems can be solved in less than 100 lines of code and almost never require advanced data structures and/or language features). And this is what I do – each year I pick a language I have never used before and solve all the problems using this language. Two years ago – the first edition of the Advent of Code – I tried Scala, last year I tried Go and this year I ended up picking OCaml (with Rust and D being other contenders). One thing I decided to do differently this year however, was to write down my observations and things I struggled with to share them, hoping that it will help people trying to learn OCaml.

Preparing Development Environment

Before I could start solving the problems I needed to setup my dev environment and figure out how to build and run programs written in OCaml. I used Visual Studio Code as my editor because I was sure it would have an extension for OCaml (I used OCaml & Reason IDE). To build my programs I ended up using ocamlbuild although probably it was an overkill – I never had more than one file to compile and used at most one additional library (Str). With my dev environment set up I was ready to start solving Advent of Code problems and learning OCaml the hard (i.e. trial-and-error) way.

Common errors

When you try to pick a new language, you will initially make a lot of simple, syntactic mistakes. I think this was one of the biggest barriers to me at the beginning. I would try to compile my program and the compilation would fail with an error I could only stare at. Understanding the error would initially take me a lot of time and experimenting (like commenting out code etc.). After a few days I figured what errors my most common mistakes resulted in and things got much easier, but I would still occasionally encounter an error which took a relatively long time to resolve. (Note, some of the mistakes could probably have been avoided if I had read more on the language before I started coding but this is not the way I learn – I prefer reading just enough to be able to do simple things and then figure out things as I go).
Before I dive deeper into errors I encountered I would like to give a few hints that can help in locating and understanding the error:

  • The location of the error contains the line and the column where the error occurs. The column can be very helpful – especially when you invoke a function with multiple parameters or inline a function invocation
  • Oftentimes the mistake is not in the line the error message points to. If you can’t find anything wrong with the line the error message points to check the line(s) the function is invoked from
  • Learn how to read function signatures (e.g. int -> int -> int) to easier understand errors caused by passing values of incorrect types. See the Types of Functions in the OCaml tutorial.

During my adventure with OCaml I compiled a list of errors I encountered.

Syntax error

This was initially the most common error I saw. It can have many causes and I am sure that the list below is not exhaustive. Most common causes:

  • missing in after variable declaration:
    let sum a b =
    let s = a + b
    print_int
    (* Error: Syntax error *)
    let sum a b =
    let s = a + b in
    print_int s
    (* val sum : int -> int -> unit = <fun> *)
    view raw missing_in.ml hosted with ❤ by GitHub
  • using => instead of -> (likely specific to developers who mostly use C# or JavaScript/TypeScript)
  • using reserved words as variable names (I fell a few times for val and match I wanted to use respectively for a variable storing a temporary value and a result of regular expression match)
  • missing -> in pattern matching (the exact error is Syntax error: pattern expected

Unbound value

This error means that the function you are trying to call cannot be found. Most common causes:

  • You have a typo in the function name
  • Your recursive function is not marked rec:
    let factorial n =
    if n = 0 then 1
    else n * factorial(n - 1)
    (* Error: Unbound value factorial *)
    let rec factorial n =
    if n = 0 then 1
    else n * factorial(n - 1)
    (* val factorial : int -> int = <fun> *)
    view raw missing_rec.ml hosted with ❤ by GitHub

This function has type X It is applied to too many arguments; maybe you forgot a `;'.

Similarly to Syntax error there can be multiple causes for this error:

  • You actually forgot a ; to separate your statements:
    print_string "a"
    print_string "b"
    (* Error: This function has type string -> unit
    It is applied to too many arguments; maybe you forgot a `;'. *)
    print_string "a";
    print_string "b"
    (* Result: ab *)
  • You passed too many arguments to a function:
    let sum list = List.fold_left (fun v a -> a + v) 0 list
    sum [1;2;3] "a"
    (* Error: This function has type int list -> int
    It is applied to too many arguments; maybe you forgot a `;'. *)
    sum [1;2;3]
    (* Result: 6 *)
  • You forgot parenthesis when inlining a function (similar to a previous case but sometimes harder to notice):
    print_int max 4 5
    (* Error: This function has type int -> unit
    It is applied to too many arguments; maybe you forgot a `;'. *)
    print_int (max 4 5)
    (* Result: 5 *)

WTF errors

The following errors fall into one of the categories above but can be very hard to spot a beginner (like I was):

  • Arithmetical operations, string or list concatenation must be in parentheses when used as an argument to a function :
    print_int 2 + 3
    (* Error: This expression has type unit but an expression was expected of type
    int *)
    print_int (2 + 3)
    (* Result: 5 *)
    print_string "a"^"b"
    (* Error: This expression has type unit but an expression was expected of type
    string *)
    print_string ("a"^"b")
    (* Result: ab *)
  • Negative values also should use parentheses:
    print_int -5
    (* Error: This expression has type int -> unit
    but an expression was expected of type int *)
    print_int (-5)
    (* Result: -5 *)
  • Sometimes the error points to a line that does not seem to be the cause of the problem. For instance in the following example I did not convert the argument of the sqrt function to the float type but the error points to a totally different line:
    let is_prime n =
    let rec is_prime_aux n div =
    if div > int_of_float (sqrt n) then true
    else if n mod div = 0 then false
    else is_prime_aux n (div + 1)
    in
    is_prime_aux n 2
    (* Error: This expression has type float but an expression was expected of type
    int
    Points to `n` in `if n mod div...` *)
    let is_prime n =
    let rec is_prime_aux n div =
    if div > int_of_float(sqrt (float_of_int n)) then true
    else if n mod div = 0 then false
    else is_prime_aux n (div + 1)
    in
    is_prime_aux n 2
  • Errors caused by partial application triggered by unintentionally passing fewer parameters that the function requires.

Observations

If you don’t want a biased opinion about OCaml you can stop reading now 🙂

Overall, I did not enjoy OCaml as a programming language. Initially, I struggled with errors that did not have much meaning to me. Once I got past this phase I found that I was not very productive – even conceptually simple problems required too much code for my taste. Maybe part of it was me learning the language but there were other people doing Advent of Code in OCaml and I found their solutions rarely required less code than mine. I also thought that it might be because I don’t use functional programming in my day-to-day work (except for quasi-functional features of C# like LINQ) and can’t switch to the functional paradigm but I briefly looked at my solutions in Scala from 2015 and they are generally much more compact.

Because I wanted to learn the language and not the libraries I wanted to get away as much as possible with just the basic language features (i.e. without using external libraries). It turned out to be hard. One of the most basic operation is writing a formatted string. OCaml offers a number of print_* functions. Unfortunately, these functions are very basic and even writing a number followed by a new line requires two print statements. Printing a formatted string with these functions was so cumbersome that I eventually decided to use the Printf module.

Another thing that baffled me at the very beginning was reading file contents. Most Advent of Code problems require reading the puzzle input from a file and I could not find any simple method to read lines from a file. It took me more time than I wanted to spend on this basic problem. All solutions seemed overly complicated. I eventually found this function on stackoverflow:

let read_lines name : string list =
let ic = open_in name in
let try_read () =
try Some (input_line ic) with End_of_file -> None in
let rec loop acc = match try_read () with
| Some s -> loop (s :: acc)
| None -> close_in ic; List.rev acc in
loop []

I have to admit that being a novice I initially did not fully understand how it worked but hey, it did work. The other alternatives I found were either even more complicated or required understanding concepts I did not event want to learn at the time (e.g. channels). All in all, I have to say there is a heck of complexity to achieve a very basic task that in many languages is just a single, self-explanatory line of code – e.g. File.ReadAllLines("myFile.txt").

I was also annoyed but by the lack of functions for simple string processing. To treat a string to as a sequence of characters you need to do something like this (again, I am not the author of this function):

let explode s =
let rec exp i l =
if i < 0 then l else exp (i - 1) (s.[i] :: l) in
exp (String.length s - 1) []

In reality, it’s hard to do string processing without using the Str module (btw. compare the name with the String module that contains basic string operations and is part of the standard library and tell me how you would not be confused which one is which) which supports regular expressions.

Speaking of regular expressions I found them weird to use. The first weird thing was that you could not just get all matched groups (e.g. as a list). Rather, you need to enumerate them one by one without knowing how many groups where matched (so you potentially enumerate until you hit an error). The other weird thing was that it seems that matched groups (or maybe some state that allows calculating them) are stored in the library and are keyed by the original string that was processed. Each time you want to get a matched group you need to provide this original string.

API inconsistencies are also irritating. For instance, both List and Hashtbl have the fold_left function. However, while List takes the function, the initial value and the list as parameters, Hashtbl takes the function, the hash table and the initial value (i.e. the initial value and the container are swapped comparing to List). It seems like a small thing, but I hit this many, many times.

Similarly, if you have multiple statements you need to separate them with a semicolon (;). However, if you use multiple statements in the then or else clause they also need to be wrapped with begin/end (or parentheses). On the other hand, you don’t need begin/end if you have multiple statements inside a loop (yes, I know, I should use recursion but sometimes using a loop is, you know, just simpler).

Runtime errors are a nightmare. You don’t get any details about the error except for the message. This might work if an exception is thrown from your own code but how you are supposed to find the bug effectively if all you get is:
Fatal error: exception Invalid_argument("index out of bounds")?

The last thing that I was surprised by was how scarce the documentation for OCaml is. When I decided to use OCaml to solve Advent of Code problems I knew it was not one of the mainstream programming languages, but I did not consider it completely niche. As soon as I started I realized that there is only a very limited number of resources at my disposal. Ironically, one of the best turned out to be the library reference on the https://caml.inria.fr website whose main page says:
This site is updated infrequently. For up-to-date information, please visit the new OCaml website at ocaml.org.
The library reference looks a bit raw and dated (e.g. see: https://caml.inria.fr/pub/docs/manual-ocaml/libref/String.html) and at the beginning I had a hard time digesting the information it provided but once I got more familiar with OCaml I would visit it all the time (yes, 50% of my visits were to check if the initial value for the fold_left function should go before or after the container).

To sum up – the next time I start a new project and have freedom to choose the language for the project I don’t think OCaml will be on the list. I am glad I tried it but I hoped for a more pleasant ride.

Coincidentally, HackerRank released their 2018 Developer Skill Report recently and OCaml was one of only two languages with negative sentiment among developers of all age groups. I will just say that the other language was Perl.

P.S. My solutions to Advent of Code 2017 can be found on github.