• 201509.05

    Nginx returns error on file upload

    I love Nginx and have never had a problem with it. Until now.

    Turtl, the private Evernote alternative, allows uploading files securely. However, after switching to a new server on Linode, uploads broke for files over 10K. The server was returning a 404.

    I finally managed to reproduce the problem in cURL, and to my surprise, the requests were getting stopped by Nginx. All other requests were going through fine, and the error only happened when uploading a file of 10240 bytes or more.

    First thing I though was that Nginx v1.8.0 had a bug. Nobody on the internet seemed to have this problem. So I installed v1.9.4. Now the server returned a 500 error instead of a 404. Still no answer to why.

    I finally found it: playing with client_body_buffer_size seemed to change the threshold for which files would trigger the error and which wouldn’t, but ultimately the error was still there. Then I read about how Nginx uses temporary files to store body data. I checked that folder (in my case /var/lib/nginx/client_body) and the folder was writeable by the nginx user, however the parent folder /var/lib/nginx was owned by root:root and was set to 0700. I set /var/lib/nginx to be readable/writable by user nginx, and it all started working.

    Check your permissions

    So, check your folder permissions. Nginx wasn’t returning any useful errors (first a 404, which I’m assuming was a bug fixed in a later version) then a 500 error. It’s important to note that after switching to v1.9.4, the Permission Denied error did show up in the error log, but at that point I had already decided the logs were useless (v1.8.0 silently ignored the problem).

    Another problem

    This is an edit! Shortly after I applied the above fix, I started getting another error. My backend was getting the requests, but the entire request was being buffered by Nginx before being proxied. This is annoying to me because the backend is async and is made to stream large uploads.

    After some research, I found the fix (I put this in the backend proxy’s location block:

    proxy_request_buffering off;

    This tells Nginx to just stream the request to the backend (exactly what I want).

  • 201507.29

    Turtl's new syncing architecture

    For those of you just joining us, I’m working on an app called Turtl, a secure Evernote alternative. Turtl is an open-source note taking app with client-side encryption which also allows private collaboration. Think like a private Evernote with a self-hosted option (sorry, no OCR yet =]).

    Turtl’s version 0.5 (the current version) has syncing, but it was never designed to support offline mode, and requires clients to be online to use Turtl. The newest upcoming release supports fully offline mode (except for a few things like login, password changes, etc). This post will attempt to describe how syncing in the new version of Turtl works.

    Let’s jump right in.

    Client IDs (or the “cid”)

    Each object having a globally unique ID that can be client-generated makes syncing painless. We do this using a few methods, some of which are actually borrowed from MongoDB’s Object ID schema.

    Every client that runs the Turtl app creates and saves a client hash if it doesn’t have one. This hash is a SHA256 hash of some (cryptographically secure) random data (current time + random uuid).

    This client hash is then baked into every id of every object created from then on. Turtl uses the composer.js framework (somewhat similar to Backbone) which gives every object a unique ID (“cid”) when created. Turtl replaces Composer’s cid generator with its own that creates IDs like so:

    12 bytes hex timestamp | 64 bytes client hash | 4 bytes hex counter

    For example, the cid


    breaks down as:

     timestamp    client hash                                                      counter
    014edc2d6580 b57a77385cbd40673483b27964658af1204fcf3b7b859adfcb90f8b895521597 0012
     |                                    |                                        |
     |- 1438213039488                     |- unique hash                           |- 18

    The timestamp is a new Date().getTime() value (with leading 0s to support longer times eventually). The client hash we already went over, and the counter is a value tracked in-memory that increments each time a cid is generated. The counter has a max value of 65535, meaning that the only way a client can produce a duplicate cid is by creating 65,535,001 objects in one second. We have some devoted users, but even for them creating 65M notes in a second would be difficult.

    So, the timestamp, client hash, and counter ensure that each cid created is unique not just to the client, but globally within the app as well (unless two clients create the same client hash somehow, but this is implausible).

    What this means is that we can create objects endlessly in any client, each with a unique cid, use those cids as primary keys in our database, and never have a collision.

    This is important because we can create data in the client, and not need server intervention or creation of IDs. A client can be offline for two weeks and then sync all of its changes the next time it connects without problems and without needing a server to validate its object’s IDs.

    Using this scheme for generating client-side IDs has not only made offline mode possible, but has greatly simplified the syncing codebase in general. Also, having a timestamp at the beginning of the cid makes it sortable by order of creation, a nice perk.

    Queuing and bulk syncing

    Let’s say you add a note in Turtl. First, the note data is encrypted (serialized). The result of that encryption is shoved into the local DB (IndexedDB) and the encrypted note data is also saved into an outgoing sync table (also IndexedDB). The sync system is alerted “hey, there are outgoing changes in the sync table” and if, after a short period, no more outgoing sync events are triggered, the sync system takes all pending outgoing sync records and sends them to a bulk sync API endpoint (in order).

    The API processes each one, going down the list of items and updating the changed data. It’s important to note that Turtl doesn’t support deltas! It only passes full objects, and replaces those objects when any one piece has changed.

    For each successful outgoing sync item that the API processes, it returns a success entry in the response, with the corresponding local outgoing sync ID (which was passed in). This allows the client to say “this one succeeded, remove it from the outgoing sync table” on a granular basis, retrying entries that failed automatically on the next outgoing sync.

    Here’s an example of a sync sent to the API:

        {id: 3, type: 'note', action: 'add', data: { <encrypted note data> }}

    and a response:

        success: [
            {id: 3, sync_ids: ['5c219', '5c218']}

    We can see that sync item “3” was successfully updated in the API, which allows us to remove that entry from our local outgoing sync table. The API also returns server-side generate sync IDs for the records it creates in its syncing log. We use these IDs passed back to ignore incoming changes from the API when incoming syncs come in later so we don’t double-apply data changes.

    Why not use deltas?

    Wouldn’t it be better to pass diffs/deltas around than full objects? If two people edit the same note in a shared board at the same time, then the last-write-wins architecture would overwrite data!

    Yes, diffs would be wonderful. However, consider this: at some point, an object would be an original, and a set of diffs. It would have to be collapsed back into the main object, and because the main object and the diffs would be client-encrypted, the server has no way of doing this.

    What this means is that the clients would not only have to sync notes/boards/etc but also the diffs for all those objects, and collapse the diffs into the main object then save the full object back to the server.

    To be clear, this is entirely possible. However, I’d much rather get the whole-object syncing working perfectly before adding additional complexity of diff collapsing as well.

    Polling for changes

    Whenever data changes in the API, a log entry is created in the API’s “sync” table, describing what was changed and who it affects. This is also the place where, in the future, we might store diffs/deltas for changes.

    When the client asks for changes, at does so using a sequential ID, saying “hey, get me everything affecting my profile that happened after <last sync id>”.

    The client uses long-polling to check for incoming changes (either to one’s own profile or to shared resources). This means that the API call used holds the connection open until either a) a certain amount of time passes or b) new sync records come in.

    The API uses RethinkDB’s changefeeds to detect new data by watching the API’s sync table. This means that changes coming in are very fast (usually within a second of being logged in the API). RethinkDB’s changefeeds are terrific, and eliminate the need to poll your database endlessly. They collapse changes up to one second, meaning it doesn’t return immediately after a new sync record comes in, it waits a second for more records. This is mainly because syncs happen in bulk and it’s easier to wait a bit for a few of them than make five API calls.

    For each sync record that comes in, it’s linked against the actual data stored in the corresponding table (so a sync record describing an edited note will pull out that note, in its current form, from the “notes” table). Each sync record is then handed back to the client, in order of occurence, so it can be applied to the local profile.

    The result is that changes to a local profile are applied to all connected clients within a few seconds. This also works for shared boards, which are included in the sync record searches when polling for changes.

    File handling

    Files are synced separately from everything else. This is mainly because they can’t just be shoved into the incoming/outgoing sync records due to their potential size.

    Instead, the following happens:

    Outgoing syncs (client -> API)

    Then a new file is attached to a note and saved, a “file” sync item is created and passed into the ougoing sync queue without the content body. Keep in mind that at this point, the file contents are already safe (in encrypted binary form) in the files table of the local DB. The sync system notices the outgoing file sync record (sans file body) and pulls it aside. Once the normal sync has completed, the sync system adds the file record(s) it found to a file upload queue (after which the outgoing “file” sync record is removed). The upload queue (using Hustle) grabs the encrypted file contents from the local files table uploads it to the API’s attachement endpoint.

    Attaching a file to a note creates a “file” sync record in the API, which alerts clients that there’s a file change on that note they should download.

    It’s important to note that file uploads happen after all other syncs in that bulk request are handled, which means that the note will always exist before the file even starts uploading.

    Encrypted file contents are stored on S3.

    Incoming syncs (API -> client)

    When the client sees an incoming “file” sync come through, much like with outgoing file syncs, it pulls the record aside and adds it to a file download queue instead of processing it normally. The download queue grabs the file via the note attachment API call and, once downloaded, saves it into the local files database table.

    After this is all done, if the note that the file is attached to is in memory (decrypted and in the user’s profile) it is notified of the new file contents and will re-render itself. In the case of an image attachment, a preview is generated and displayed via a Blob URL.

    What’s not in offline mode?

    All actions work in offline mode, except for a few that require server approval:

    • login (requires checking your auth against the API’s auth database)
    • joining (creating an account)
    • creating a persona (requires a connection to see if the email is already taken)
    • changing your password
    • deleting your account

    What’s next?

    It’s worth mentioning that after v0.6 launches (which will include an Android app), there will be a “Sync” interface in the app that shows you what’s waiting to be synced out, as well as file uploads/downloads that are active/pending.

    For now, you’ll just have to trust that things are working ok in the background while I find the time to build such an interface =].

  • 201507.26

    Squeezebox setup without the remote/controller

    My dad recently gave me a Squeezebox for a present after he’d upgraded his home audio system. I was grateful but ultimately stumped on how to set it up. I read a bunch online about setting it up with the remote it comes with. Oh wait. Mine doesn’t have a remote.

    Let the fun begin.

    Connecting to the Squeezebox

    This is harder than it sounds. Initially, I tried wiring it into my router and seeing if it could see it. It could not. This was a WRT54G with Tomato firmware. Maybe the setups just weren’t compatible or something ridiculous like that.

    So I tried another way I found after poking around a lot: the Squeezebox has a build in wireless SSID that you can connect to in ad-hoc mode (after holding the only button for > 6 seconds and it enters reset/config mode).

    However, doing this is finicky and had me tearing my hair out. Ultimately, I got my Windows machine connected to it. When it connects, it gives your machine an IP in the range. If it gives you a 169.xxx.xxx.xxx address, it’s game over. Try restarting your machine. Try resetting the Squeezebox. Try a rain dance while wearing a tribal loincloth. You just might get that IP.

    I recently bought a new router (Buffalo, w/ DD-WRT) and plugged the reset-mode Squeezebox into it (via LAN) and was able to connect to it instantly, so try that first and only go the ad-hoc wireless route if you absolutely have to.

    Talking to the Squeezebox

    The Net-UDAP software is amazing and wonderful. I don’t know what it does under the hood, but it lets you talk to your Squeezebox should you finally get connected to it in some capacity.

    Unless you’re on Windows. Yes, I know, it supposedly works on Windows but just didn’t find the Squeezebox with running discover.

    My only solution was to spin up a linux VM with a bridged network adapter and run Net-UDAP there instead. Worked flawlessly. Hopefully you have a linux box laying around, or maybe it will just work for you in Windows. Try the Windows perl binary instead of cygwin’s perl.

    Anyway, once you’ve got everything connected, you run the Net-UDAP like so:

    cd /path/to/net-udap
    ./scripts/udap_shell.pl -a

    Note that the is the address for the machine you’re running the shell on, not the Squeezebox itself.

    You should get a prompt:


    Now run the discover command:

    UDAP> discover
    info: *** Broadcasting adv_discovery message to MAC address 00:00:00:00:00:00 on
    info:   adv_discovery response received from 69:69:69:69:69:69
    info: *** Broadcasting get_ip message to MAC address 69:69:69:69:69:69 on
    info:   get_ip response received from 69:69:69:69:69:69
    info: *** Broadcasting get_data message to MAC address 69:69:69:69:69:69 on
    info:   get_data response received from 69:69:69:69:69:69

    Hopefully you get output like that. If you get empty output, see “Connecting to the Squeezebox” =[.


    Once that finicky little bastard is discovered, run list:

    UDAP> list  
     #    MAC Address    Type       Status
    == ================= ========== ===============
     1 69:69:69:69:69:69 squeezebox init

    You can see it has an ID of 1 so we do:

    UDAP> conf 1

    Your prompt will now change, and you’re in config mode:

    UDAP [1] (squeezebox 696969)>

    Now you can connect it to your network via wireless by setting values into its config. This will differ network to network, but here are the commands I run to get things working on a network with WPA2-PSK TKIP+AES:

    set hostname=jammy interface=0 lan_gateway= lan_ip_mode=1 primary_dns=
    set wireless_SSID='my network SSID' wireless_wpa_mode=2 wireless_wpa_cipher=3 wireless_keylen=0 wireless_mode=0 wireless_region_id=4 wireless_wpa_on=1 wireless_wpa_psk='WPA passwordddd' wireless_channel=9
    set server_address=

    For a list of the fields and what they mean, type fields:

    UDAP [1] (squeezebox 696969)> fields
                 bridging: Use device as a wireless bridge (not sure about this)
                 hostname: Device hostname (is this set automatically?)
                interface: 0 - wireless, 1 - wired (is set to 128 after factory reset)
              lan_gateway: IP address of default network gateway, (e.g.
              lan_ip_mode: 0 - Use static IP details, 1 - use DHCP to discover IP details
      lan_network_address: IP address of device, (e.g.
          lan_subnet_mask: Subnet mask of local network, (e.g.
              primary_dns: IP address of primary DNS server
            secondary_dns: IP address of secondary DNS server
           server_address: IP address of currently active server (either Squeezenetwork or local server
    squeezecenter_address: IP address of local Squeezecenter server
       squeezecenter_name: Name of local Squeezecenter server (???)
            wireless_SSID: Wireless network name
         wireless_channel: Wireless channel (used by AdHoc mode???)
          wireless_keylen: Length of wireless key, (0 - 64-bit, 1 - 128-bit)
            wireless_mode: 0 - Infrastructure, 1 - Ad Hoc
       wireless_region_id: 4 - US, 6 - CA, 7 - AU, 13 - FR, 14 - EU, 16 - JP, 21 - TW, 23 - CH
       wireless_wep_key_0: WEP Key 0 - enter in hex
       wireless_wep_key_1: WEP Key 1 - enter in hex
       wireless_wep_key_2: WEP Key 2 - enter in hex
       wireless_wep_key_3: WEP Key 3 - enter in hex
          wireless_wep_on: 0 - WEP Off, 1 - WEP On
      wireless_wpa_cipher: 1 - TKIP, 2 - AES, 3 - TKIP & AES
        wireless_wpa_mode: 1 - WPA, 2 - WPA2
          wireless_wpa_on: 0 - WPA Off, 1 - WPA On
         wireless_wpa_psk: WPA Public Shared Key

    To see the values already set, run list when in config mode.

    Great, so you’ve set up all your network values and are confident that you’ve done it all right the first time. Good for you. Now you can run save_data:

    UDAP [1] (squeezebox 696969)> save_data
    info: *** Broadcasting set_data message to MAC address 69:69:69:69:69:69 on
    ucp_method set_data callback not implemented yet at /path/to/udap/../src/Net-UDAP/lib/Net/UDAP.pm line 292.
    Raw msg:
              00 01 02 03 04 05 06 07 - 08 09 0A 0B 0C 0D 0E 0F  0123456789ABCDEF
    00000000  00 02 00 00 00 00 00 00 - 00 01 00 04 20 16 5A 05  ............ .Z.
    00000010  00 01 C0 01 00 00 01 00 - 01 00 06 00 1A           .............
    info:   set_data response received from 69:69:69:69:69:69

    Make sure save_data returns a response similar to this. If it doesn’t, run it again. In fact, run it again anyway. Run it again…and again…and again.

    Great, now run reset to finalize everything:

    UDAP [1] (squeezebox 696969)> reset
    info: *** Broadcasting reset message to MAC address 69:69:69:69:69:69 on
    ucp_method reset callback not implemented yet at /path/to/udap/../src/Net-UDAP/lib/Net/UDAP.pm line 292.
    Raw msg:
              00 01 02 03 04 05 06 07 - 08 09 0A 0B 0C 0D 0E 0F  0123456789ABCDEF
    00000000  00 02 00 00 00 00 00 00 - 00 01 00 04 20 16 5A 05  ............ .Z.
    00000010  00 01 C0 01 00 00 01 00 - 01 00 04                 ...........
    info:   reset response received from 69:69:69:69:69:69

    All done. Now don’t ever change your network setup ever again or you’ll have to deal with this shit again. Or just get a fucking remote…

  • 201507.23

    Harry's razors review

    This is a review of Harry’s razors. I haven’t been payed by them at all or been sent any promotional materials. The words/opinions expressed here are my own.

    I hate shaving, but even more I hate having facial hair. I find it uncomfortable. I’ve used a good amount of shaving products throughout my life, and have settled on the standard cartridge razor, which gives (in my opinion) the best shave-time to shave-closeness ratio and offers a near-perfectly smooth face and neck while only taking about 5-8 minutes to complete.

    Now, I’m a bit different from other shavers (I think) in that, like my clothing, I keep my razors around far longer than most people. I will use the same razor head for up to four months (basically until it’s so dull it just won’t work anymore). I usually shave about 3-4 times a week.

    “Razor companies HATE him!!”

    Up until about 6 months ago, I’d been using mainly Gillette razors. I’d get a big pack of refills at Costco every now and then and work through them over a year or so.

    One thing that pissed me off endlessly about Gillette is that by the time I had gone through my set of razor heads, the handle would be obsolete and I’d have to buy a whole new kit (which they charge a lot extra for). So about the third time this happened I decided there had to be a better way than continuously throwing money at Gillette. By the way, their higher-end razors are great, but their practices of having different handles every week is infuriating.

    I had previously seen ads for Harry’s razors so decided to give them a shot. The company seems small enough that redesigning their handles/connectors every few weeks would bankrupt them, but initial reviews on the razors themselves were good. I picked up the Truman handle with a set of blades.

    Enough babbling, here’s a pros/cons list:

    The good

    • The handle is solid, and has a nice weight to it (as opposed to plasticy and bendy).
    • The razor heads snap in nicely, without any play.
    • The razors have an open back.

      open back I can’t say enough how great this is. All razors I have ever used hide the back of the razor with a bunch of plastic.

      With a covered back, most of the hair you shave off over the course of the blade’s life ends up staying inside the razor head. You can beat it against the sink or blast it with water all you want, there’s always going to be a bunch of old, moldy hair stuck inside your razor.

      With Harry’s, the back is open and a quick rinse under the faucet or showerhead gets rid of all hair on the blade. I cannot stress how easily it is both to unclog and to clean the blades.

    • The blades are easy to unclog. Because of the open back, you can easy rinse the blades to get rid of any hair. This makes them ideal for shaving areas with lots of hair, and while Harry’s is marketed towards men I see no reason why these razors wouldn’t be able to work for women as well (and for a lot cheaper than women’s razor heads).
    • The blades last a long time. My maximum is about four months on one blade. This is made easier by how well the blades clean up after use (once again, thanks to the open back). This means for me that a 4-pack of blades should last about a year (sorry, Harry).

      They do get noticeably duller after about 4-5 uses, but they continue funcitoning admirably for many, many uses. Once again, I shave maybe 3-4 times a week. So conservatively (3 shaves/week over 12 weeks), that’s about 35-40 shaves per razor head.

    The bad

    • The handle is slippery. I routinely drop the handle while shaving. Having ugly rubber grips would detract from the look, but make shaving a lot easier.
    • The razor heads are somewhat bulky. I find it incredibly hard to reach places of my neck/face that the Gillette razors would glide over no problem. I think if they found a way to remove the thickness of the plastic housing the blades themselves, and possibly make the blades stick out of the housing by a few more micrometers, this would make shaving a lot easier.
    • The razors have a strange pulling feeling when shaving, somewhat like pulling a rubber eraser across your face. This is not painful or irritating, just somewhat odd feeling.
    • The piece of plastic that gives the blade “spring” when pushed against your face wears down over time, making it feel spongy (and eventually requiring you to hold the razor flat against your face with a finger/thumb on the hand holding the handle). Not a huge deal, and probably not an issue for most people since razor re-use isn’t a 3-4 month affair.


    The pros outweigh the cons easily.

    Definitely would recommend this brand. So far, they haven’t changed the handle or blade connectors at all. The blades work admirably. They are a bit bulky, but easy to clean and unclog. The whole setup also looks really nice.

    As mentioned, while Harry’s is marketed towards men, this setup could easily work great for women (or anyone) who wants to shave arms/legs as well because of the easy unclogging.

  • 201507.21

    Hackernews: a typical day

  • 201507.18

    Switching to Jekyll

    I’ve decided to get rid of Wordpress that was on blog.killtheradio.net as well as the PHP site at killtheradio.net and combine both into a Jekyll blog on the http://killtheradio.net/ domain.

    Moving to Jekyll from Wordpress took a few days, but I got all my posts moved, edited to fix formatting errors, and switching all discussions to use Disqus (and of course imported the old comments).

    This site now works on mobile devices as well.

    There are a few reasons for all this, but mainly I’ve been intrigued by the idea of static site generators for a while now and wanted to try it out. Also, as time went on, I grew to desipise Wordpress, including all the idiotic security vulnerabilities I suffered through week after week. It’s a slapped-together platform, and the plugins for it are even worse.

    There’s a certain thrill to authoring and publishing new content using only the command line.

  • 201501.26

    Node.js and Cygwin: Unknown system errno 203

    A recent Cygwin upgrade left me ripping my hair out, because none of my npm or grunt commands would work. They spat out the error

    Unknown system errno 203

    Helpful, right?

    The fix

    In your Cygwin ~/.bash_profile file (create it and chmod 755 .bash_profile if it doesn't exist):

    export TMP=/tmp
    export TEMP=$TMP

    This did the trick for me. Special thanks to this github comment.

  • 201409.18

    Sudoers syntax error for specific commands

    This will be short but sweet. When deploying some new servers today, I ran into a problem where no matter what, sudo bitched about syntax errors in my sudoers file. I tried a bunch of different options/whitespace tweaks/etc and nothing worked.

    deploy ALL= NOPASSWD: monit restart my-app

    Looks fine right? Nope.

    Use absolute paths

    This fixed it:

    deploy ALL= NOPASSWD: /usr/bin/monit restart my-app

    Everyone in the world's advice is to "just use visudo" but I couldn't find any info on what was actually causing the syntax error. Hopefully this helps a few lost souls.

  • 201407.20

    Composer.js v1.0 released

    The Composer.js MVC framework has just released version 1.0! Note that this is a near drop-in replacement for Composer v0.1.x.

    There are some exciting changes in this release:

    • Composer no longer requires Mootools... jQuery can be used as a DOM backend instead. In fact, it really only needs the selector libraries from Moo/jQuery (Slick/Sizzle) and can use those directly. This means you can now use Composer in jQuery applications.
    • Controllers now have awareness of more common patterns than before. For instance, controllers can now keep track of sub-controllers as well as automatically manage bindings to other objects. This frees you up to focus on building your app instead of hand-writing boilerplate cleanup code (or worse, having rogue objects and events making your app buggy).
    • The ever-popular RelationalModel and FilterCollection are now included by default, fully documented, and considered stable.
    • New class structures in the framework expose useful objects, such as Composer.Class which gives you a class structure to build on, or Composer.Event which can be used as a standalone event bus in your app.
    • There's now a full test suite so people who want to hack away on Composer (including us Lyon Bros) can do so without worrying about breaking things.
    • We updated the doc site to be much better organized!

    Breaking changes

    Try as we might, we couldn't let some things stay the same and keep a clear conscience. Mainly, the problems we found were in the Router object. It no longer handles hashbang (#!) fallback...it relies completely on History.js to handle this instead. It also fixes a handful of places where non-idiomatic code was used (see below).

    • Composer.Router: the on_failure option has been removed. Instead of

      var router = new Composer.Router(routes, {on_failure: fail_fn});

      you do

      var router = new Composer.Router(routes);
      router.bind('fail', fail_fn);
    • Composer.Router: The register_callback function has been removed. In order to achieve the same functionality, use router.bind('route', myfunction);.
    • Composer.Router: The "preroute" event now passes {path: path} as its argument instead of path. This allows for easier URL rewriting, but may break some apps depending on the old method.
    • Composer.Router: History.js is now a hard requirement.

    Sorry for any inconvenience this causes. However, since the rest of the framework is backwards compatible, you should be able to just use the old Composer.Router object with the new framework without any problems if you don't wish to convert your app.

    Have fun!

    Check out the new Composer.js, and please open an issue if you run into any problems. Thanks!

    - The Lyon Bros.

  • 201407.16

    Nanomsg as the messaging layer for turtl-core

    I recently embarked on a project to rebuild the main functionality of Turtl in common lisp. This requires embedding lisp (using ECL) into node-webkit (or soon, Firefox, as node-webkit is probably getting dumped).

    To allow lisp and javascript to communicate, I made a simple messaging layer in C that both sides could easily hook into. While this worked, I stumbled on nanomsg and figured it couldn't hurt to give it a shot.

    So I wrote up some quick bindings for nanomsg in lisp and wired everything up. So far, it works really well. I can't tell if it's faster than my previous messaging layer, but one really nice thing about it is that it uses file descriptors which can be easily monitored by an event loop (such as cl-async), making polling and strange thread < --> thread event loop locking schemes a thing of the past (although cl-async handles all this fairly well).

    This simplified a lot of the Turtl code, and although right now it's only using the nanomsg "pair" layout type, it could be easily expanded in the future to allows different pieces of the app to communicate. In other words, it's a lot more future-proof than the old messaging system and probably a lot more resilient (dedicated messaging library authored by 0MQ mastermind beats hand-rolled, hard-coded simple messaging built by non-C expert).

  • 201406.19

    Windows GUI apps: Bad file descriptor. (or how to convert a GUI app into a console app for easy debugging)

    Lately I've been neck-deep in embedding. Currently, I'm building a portable (hopefully) version of Turtl's core features in ECL.

    Problem is, when embedding turtl-core into Node-webkit or Firefox, any output that ECL writes to STDOUT triggers:

    C operation (write) signaled an error. C library explanation: Bad file descriptor.

    Well it turns out Windows doesn't let you write to STDOUT unless a console is available, and even if using msys, it doesn't create a console for GUI apps. So here's a tool (in lisp, of course) that will let you convert an executable between GUI and console.

    Seems to work great. Special thanks to death.

  • 201402.02

    Access your Firefox extension/add-on variables from the browser console

    It can be nice to access your FF extension's variables/functions from the browser console (ctrl+shift+j) if you need some insight into its state.

    It took me a while to figure this out, so I'm sharing it. Somewhere in your extension, do:

    var chromewin = win_util.getMostRecentBrowserWindow();
    chromewin.my_extension_state = ...;

    Now in the browser console, you can access whatever variables you set in the global variable my_extension_state. In my case, I used it to assign a function that lets me evaluate code in the addon's background page. This lets me gain insight into the background page's variables and state straight from the browser console.

    Note! This is a security hole. Only enable this when debugging your extension/addon. Disable it when you release it.

  • 201401.22

    Debugging javascript in the default Android browser

    Note that this may or may not work on your device. If you're running into an app that works in a real browser but on in your Android's stock browser, do this:

    1. Navigate to your app in the browser.
    2. In the same tab go to about:debug
    3. Reload (it may reload for you).
    4. Profit.

    This will show you errors that even window.onerror doesn't catch, which should help you narrow down your problem(s).

    Source: This stackoverflow answer.

  • 201310.03

    Debugging Firefox addons

    As you all know, I'm building a Turtl, a browser extension for client-side encrypted note/file storage.

    Well once in a while the I need to debug the release version. There are docs scattered around detailing how to do this, but as usual with this type of thing you really need to do some digging.

    By default, Firefox's Browser Console only logs error events. You can change this to log any and all console.log() calls from your addon (and any other addon) by doing this:

    1. Go to about:config
    2. Search for the key extensions.sdk.console.logLevel
    3. If it exists, set it to "info", otherwise add a new string with the key extensions.sdk.console.logLevel and the value "info"

    Boom, all your addon's log calls now show up in the browser console.

  • 201310.01

    Curl CLI not sending full file data when using --data-binary

    I use curl to test out my HTTP libraries all the time. Recently, I ran into an issue where when uploading a file (25mb) from curl in the command line to my common lisp app server, only about half the data showed up (12.5mb). I was doing this:

    curl -H 'Authorization: ...' -H 'Transfer-Encoding: chunked' --data-binary @/media/large_vid.mov

    Naturally, I assumed the issue was with my libraries. It could be the cl-async library dropping packets, it could be the HTTP parser having issues, and it could be the app server itself. I mean, it has to be one of those. Curl has been around for ages, and there's no way it would just drop data. So I spent days tearing my hair out.

    Finally, I ran curl with the --trace and looked at the data. It provides a hex dump of everything it sends. It's not formatted perfectly, but with vim's block select and a few handy macros, I was able to get the length of the data being sent: 12.5mb. That's right, curl was defying me. There was no error in my code at all.

    I did a search online for curl not sending the full file data when using --data-binary. Nothing. So I looked over my options and found -T which looks surprisingly similar to --data-binary with the @ modifier. I tried:

    curl -H 'Authorization: ...' -H 'Transfer-Encoding: chunked' -T /media/large_vid.mov

    All 25mb came through (every byte accounted for).


    If you're uploading files, use -T /path/to/file instead of --data-binary @/path/to/file. Note that -d/-D were also "broken."

  • 201309.22

    Turtl: an encrypted Evernote alternative

    Hi FORKS. Tuesday I announced my new app, Turtl for Chrome (and soon Firefox). Turtl is a private Evernote alternative. It uses AES-256bit encryption to obscure your notes/bookmarks before leaving the browser. What this means is that even if your data is intercepted on the way to the server or if the server itself is compromised, your data remains private.

    Even with all of Turtl's privacy, it's still easy to share boards with friends and colleagues: idea boards, todo lists, youtube playlists, etc. With Turtl, only you and the people you share with can see your data. Not even the guys running the servers can see it...it's just gibberish without the key that you hold.

    One more thing: Turtl's server and clients are open-source under the GPLv3 license, meaning anyone can review the code or use it for themselves. This means that Turtl can never be secretly compromised by the prying hands of hackers or government gag orders. The world sees everything we do.

    So check out Turtl and start keeping track of your life's data. If you want to keep up to date, follow Turtl on Twitter.

  • 201301.14

    Keyboard/mouse not working in Xorg on FreeBSD

    I recently installed FreeBSD 9.1-RELEASE on a VirtualBox VM to do some cl-async testing. I wanted to get Xorg running so I could edit code at a more "comfortable" resolution. I was able to get Xorg running fairly easily just by installing Xfce from /usr/ports.

    However, upon starting Xorg, my keyboard mouse would not work. I tried many things: following the steps in the handbook, enabling/disabling hald, reconfiguring Xorg, etc. No luck. My Xorg.0.log was telling me that it couldn't load the kdb/mouse drivers. After snooping around some forums, I found the solution:

    • Install the x11-drivers/xf86-input-keyboard port
    • Install the x11-drivers/xf86-input-mouse port

    After doing this, all was right with the world. Just to clarify, I am using dbus/hald and more or less using the default configuration that Xorg -configure gave me.

  • 201211.26

    cl-async: Non-blocking, asynchronous programming for Common Lisp

    A while ago, I released cl-async, a library for non-blocking programming in Common Lisp. I've been updating it a lot over the past month or two to add features and fix bugs, and it's really coming along.

    My goal for this project is to create a general-purpose library for asynchronous programming in lisp. I think I have achieved this. With the finishing of the futures implementation, not only is the library stable, but there is now a platform to build drivers on top of. This will be my next focal point over the next few months.

    There are a few reasons I decided to build something new. Here's an overview of the non-blocking libraries I could find:

    • IOLib - An IO library for lisp that has a built-in event loop, only works on *nix.
    • Hinge - General purpose, non-blocking library. Only works on *nix, requires libev and ZeroMQ.
    • Conserv - A nice layer on top of IOLib (so once again, only runs on *nix). Includes TCP client/server and HTTP server implementations. Very nice.
    • teepeedee2 - A non-blocking, performant HTTP server written on top of IOLib.

    I created cl-async because of all the available libraries, they are either non-portable, not general enough, have too many dependencies, or a combination of all three. I wanted a library that worked on Linux and Windows. I wanted a portable base to start from, and I also wanted tools to help make drivers.

    Keeping all this in mind, I created bindings for libevent2 and built cl-async on top of them. There were many good reasons for choosing libevent2 over other libraries, such as libev and libuv (the backend for Node.js). Libuv would have been my first choice because it supports IOCP in Windows (libevent does not), however wrapping it in CFFI was like getting a screaming toddler to see the logic behind your decision to put them to bed. It could have maybe happened if I'd written a compatibility layer in C, but I wanted to have a maximum of 1 (one) dependency. Libevent2 won. It's fast, portable, easy to wrap in CFFI, and on top of that, has a lot of really nice features like an HTTP client/server, TCP buffering, DNS, etc etc etc. The list goes on. That means less programming for me.

    Like I mentioned, my next goal is to build drivers. I've already built a few, but I don't consider them stable enough to release yet. Drivers are the elephant in the room. Anybody can implement non-blocking IO for lisp, but the real challenge is converting everything that talks over TCP/HTTP to be async. If lisp supported coroutines, this would be trivial, but alas, we're stuck with futures and the lovely syntax they afford.

    I'm going to start with drivers I use every day: beanstalk, redis, cl-mongo, drakma, zs3, and cl-smtp. These are the packages we use at work in our queue processing system (now threaded, soon to be evented + threaded). Once a few of these are done, I'll update the cl-async drivers page with best practices for building drivers (aka wrapping async into futures). Then I will take over the world.

    Another goal I have is to build a real HTTP server on top of the bare http-server implementation provided by cl-async. This will include nice syntax around routing (allowing REST interfaces), static file serving, etc.

    Cl-async is still a work in progress, but it's starting to become stabilized (both in lack of bugs and the API itself), so check out the docs or the github project and give it a shot. All you need is a lisp and libevent =].

  • 201207.07

    cl-mongo and multithreading

    We're building a queuing system for Musio written in common lisp. To be accurate, we already built a queuing system in common lisp, and I recently needed to add a worker to it that communicates with MongoDB via cl-mongo. Each worker spawns four worker threads, each thread grabbing jobs from beanstalkd via cl-beanstalk. During my testing, each worker was updating a Mongo collection with some values scraped from our site. However, after a few seconds of processing jobs, the worker threads begin to spit out USOCKET errors and eventually Clozure CL enters it's debugger of death (ie, lisp's version of a segfault). SBCL didn't fare too much better, either.

    The way cl-mongo's connections work is that it has a global hash table that holds connections: cl-mongo::*mongo-registry*. When the threads are all running and communicating with MongoDB, they are using the same hash table without any inherent locking or synchronization. There are a few options to fix this. You can implement a connection pool that supports access from multiple threads (complicated), you can give each thread its own connection and force the each thread to use its connection when communicating, or you can take advantage of special variables in lisp (the easiest, simplest, and most elegant IMO). Let's check out the last option.

    Although it's not in the CL spec, just about all implementations allow you to have global thread-local variables by using (defparameter) or (defvar), both of which create special variables (read: dynamic variables, as opposed to lexical). Luckily, cl-mongo uses defvar to create *mongo-registry*. This means in our worker, we can re-bind this variable above the top level loop using (let) and all subsequent calls to MongoDB will use our new thread-local version of *mongo-registry* instead of the global one that all the threads we're bumping into each other using:

    ;; Main worker loop, using global *mongo-registry* (broken)
    (defun start-worker ()
        (let ((job (get-job)))
          (let ((results (process-job job)))
            ;; this uses the global registry. not good if running multiple threads.
            (with-mongo-connection (:db "musio")
              (db.save "scraped" results))))))

    New version:

    ;; Replace *mongo-registry* above worker loop, creating a local version of the
    ;; registry for this thread.
    (defun start-worker ()
      ;; setting to any value via let will re-create the variable as a local thread
      ;; variable. nil will do just fine.
      (let ((cl-mongo::*mongo-registry* nil))
          (let ((job (get-job)))
            (let ((results (process-job job)))
              ;; with-mongo-connection now uses the local registry, which stops the
              ;; threads from touching each other.
              (with-mongo-connection (:db "musio")
                (db.save "scraped" results)))))))

    BOOM everything works great after this change, and it was only a one line change. It may not be as efficient as connection pooling, but that's a lot more prone to strange errors and synchronization issues than just segregating the connections from each other and calling it a day. One issue: *mongo-registry* is not exported by cl-mongo, which is why we access it via cl-mongo::*mongo-registry* (notice the double colon instead of single). This means in future versions, the variable name may change, breaking our above code. So, don't update cl-mongo without testing. Not hard.

    Hopefully this helps a few people out, let me know if you have better solutions to this issue!

  • 201204.15

    Email is not broken: It's a framework, not an application

    I've been seeing a lot of posts on the webz lately about how we can fix email. I have to say, I think it's a bit short-sighted.

    People are saying it has outgrown it's original usage, or it contains bad error messages, or it's not smart about the messages received.

    These are very smart people, with real observations. The problem is, their observations are misplaced.

    What email is

    Email is a distributed, asynchronous messaging protocol. It does this well. It does this very well. So well, I'm getting a boner thinking about it. You send a message and it either goes where it's supposed to, or you get an error message back. That's it, that's email. It's simple. It works.

    There's no company controlling all messages and imposing their will on the ecosystem as a whole. There's no single point of failure. It's beautifully distributed and functions near-perfectly.

    The problem

    So why does it suck so much? It doesn't. It's awesome. The problem is the way people view it. Most of the perceived suckiness comes from its simplicity. It doesn't manage your TODOs. It doesn't have built-in calendaring. It doesn't give you oral pleasure (personally I think this should be built into the spec though). So why don't we build all these great things into it if they don't exist? We could add TODOs and calendaring and dick-sucking to email!!

    Because that's a terrible idea. People are viewing email as an application; one that has limited features and needs to be extended so it supports more than just stupid messages.

    This is wrong.

    We need to view email as a framework, not an application. It is used for sending messages. That's it. It does this reliably and predictably.

    Replacing email with "smarter" features will inevitably leave people out. I understand the desire to have email just be one huge TODO list. But sometimes I just want to send a fucking message, not "make a TODO." Boom, I just "broke" the new email.

    Email works because it does nothing but messaging.

    How do we fix it then?

    We fix it by building smart clients. Let's take a look at some of our email-smart friends.

    Outlook has built-in calendaring. BUT WAIT!!!!! Calendaring isn't part of email!!1 No, it's not.

    Gmail has labels. You can categorize your messages by using tags essentially. Also, based on usage patterns, Gmail can give weight to certain messages. That's not part of email either!! No, my friend, it's not.

    Xobni also has built incredible contact-management and intelligence features on top of email. How do they know it's time to take your daily shit before you do? Defecation scheduling is NOT part of the email spec!!

    How have these companies made so much fucking money off of adding features to email that are not part of email?

    It's all in the client

    They do it by building smart clients! As I said, you can send any message your heart desires using email. You can send JSON messages with a TODO payload and attach a plaintext fallback. If both clients understand it, then BAM! Instant TODO list protocol. There, you just fixed email. Easy, no? Why, with the right client, you could fly a fucking space shuttle with email. That's right, dude, a fucking space shuttle.

    If your client can create a message and send it, and the receiving client can decode it, you can build any protocol you want on top of email.

    That's it. Use your imaginations. I'll say it one more time:

    There's nothing to fix

    Repeat after me: "There's nothing to fix!" If you have a problem with email, fork a client or build your own! Nobody's stopping you from "fixing" email. Many people have made a lot of cash by "fixing" email.

    We don't have to sit in fluorescent-lit, university buildings deliberating for hours on end about how to change the spec to fit everyone's new needs. We don't need 100 stupid startups "disrupting" the "broken" email system with their new protocols, that will inevitably end up being  a proprietary, non-distributed, "ad hoc, informally-specified, bug-ridden, slow implementation of half of" the current email system.

    Please don't try to fix email, you're just going to fuck it up!! Trust me, you can't do any better. Instead, let's build all of our awesome new features on top of an already beautifully-working system by making smarter clients.


Newer  |