• 0 Posts
  • 31 Comments
Joined 1 year ago
cake
Cake day: June 25th, 2023

help-circle
  • For one thing: don’t bother with fancy log destinations. Just log to stderr and let your daemon manager take care of directing that where it needs to go. (systemd made life a lot easier in the Linux world).

    Structured logging is overrated since it means you can’t just do the above.

    Per-module (filterable) logging are quite useful, but must be automatic (use __FILE__ or __name__ whatever your language supports) or you will never actually do it. All semi-reasonable languages support some form of either macros-which-capture-the-current-module-and-location or peek-at-the-caller-module-name-and-location.


    One subtle part of logging: never conditionally defer a computation that can fail. Many logging APIs ultimately support something like:

    if (log_level >= INFO) // or <= depending on how levels are numbered
        do_log(INFO, message, arguments...)
    

    This is potentially dangerous - if logging of that level is disabled, the code is never tested, and trying to enable logging later might introduce an error when evaluating the arguments or formatting them into the message. Also, if logging of that level is disabled, side-effects might not happen.

    To avoid this, do one of:

    • never use the if-style deferring, internally or externally. Instead, squelch the I/O only. This can have a significant performance cost (especially at the DEBUG level), which is why the API is made in the first place.
    • ensure that your type system can statically verify that runtime errors are impossible in the conditional block. This requires that you are using a sane language and logging library.
    • run your testsuite at every log level, ensure 100% coverage of log code, and hope that the inevitable logic bug doesn’t have an unexpected dynamic failure.



  • I’ve only ever seen two parts of git that could arguably be called unintuitive, and they both got fixes:

    • git reset seems to do 2 unrelated things for some people. Nowadays git restore exists.
    • the inconsistent difference between a..b and a...b commit ranges in various commands. This is admittedly obscure enough that I would have to look up the manual half the time anyway.
    • I suppose we could call the fact that man git foo didn’t used to work unintuitive I guess.

    The tooling to integrate git submodule into normal tree operations could be improved though. But nowadays there’s git subtree for all the people who want to do it wrong but easily.


    The only reason people complain so much about git is that it’s the only VCS that’s actually widely used anymore. All the others have worse problems, but there’s nobody left to complain about them.


  • Unfortunately both of those are used in common English or computer words. The only letter pairs not used are: bq, bx, cf, cj, dx, fq, fx, fz, hx, jb, jc, jf, jg, jq, jv, jx, jz, kq, kz, mx, px, qc, qd, qg, qh, qj, qk, ql, qm, qn, qp, qq, qr, qt, qv, qx, qy, qz, sx, tx, vb, vc, vf, vj, vm, vq, vw, vx, wq, wx, xj, zx.

    Personally I have mappings based on <CR>, and press it twice to get a real newline.


  • You’ve clearly thought about the problem, so the solutions should be relatively obvious. Some less obvious ones:

    • It is impossible to make TCP reliable no matter how hard you try, because anybody can inject an RST flag at any time and cut off your connections (this isn’t theoretical, it’s actually quite common for long-lived gaming connections). That leaves UDP, for which there are several reliability layers, but most of them are not battle-tested - remember, TCP is most notable for congestion-control! HTTP3 is probably the only viable choice at scale, but beware that many implementations are very bad (e.g. not even supporting recvmmsg/sendmmsg which are critical for performance unlike with TCP; note the extra m)
    • If you don’t encrypt all your packets, you will have random middleware mess with their data. Think at least a little about key rotation.
    • To avoid application-centric DoS, make sure the client always does “more” than the server; this extends to e.g. packet sizes.
    • Prefer to ultimately define things in data, not code (e.g. network packet layouts). Don’t be afraid to write several bespoke code-generators; many real-world serialization formats in particular have unacceptable tradeoffs. Make sure the core code doesn’t care about the details (e.g. make every packet physically variable-length even if logically it is always fixed-length; you can also normalize zero-padding at this level for future compatibility. I advise against delta-compression at this level because that’s extra processing you don’t need).
    • Make sure the client only has to connect to a single server. If you have multiple servers internally, have a thin bouncer/proxy that forwards packets appropriately. This also has benefits for the inevitable DDoS attacks.
    • Latency is a bitch and has far-ranging effects, though this is highly dependent on not just genre but also UI. For example “hold down a key to move continuously through the world” is problematic whereas “click to move to a location” is not.
    • Beware quadratic complexity, e.g. if every player must send a location update to every player.
    • Think not only about the database, but how to back up the database and how to roll back in case of catastrophe or exploit. An append-only flat file has a lot going for it; only periodic repacking is needed and you can keep the old version for a while with a guarantee that it’ll replay to identical state to the initial version of the new file. Of course, the best state is no state at all. You will need to consider the notion of “transaction” at many levels, including scripting (you must give me 20 bear asses for me to give), trading between players, etc.
    • You will have abuse in chat. You will also have cybersex. It’s possible to deal with this in a privacy-preserving way by merely signing chat, not logging it, so the player can present evidence only if they wish, but there are a lot of concerns about e.g. replays, selective message subsets, etc.
    • There will be bots, especially if the official client isn’t good enough.
    • It’s $CURRENTYEAR; write code for IPv6 exclusively. There are sockopts for transparently handling legacy IPv4 clients.
    • Client IP address is private information. It is also the only way to deal with certain kinds of abuse. Sometimes, you just have to block all of Poland.
    • Note that routing in parts of the world is really bad. Sometimes setting up your own dedicated connection chain between datacenters can improve performance by orders of magnitude, rather than letting clients use whatever their ISP says. If nesting proxies be sure to correctly validate IPs.
    • Life is simpler if internal stuff listens on a separate port than external stuff, but still verify your peer. IP whitelisting is useless except for localhost (which, mind, is all of 127.0.0.0/8 for IPv4 - about the only time IPv4 is actually useful rather than a mere mirage).


  • Speed is far from the only thing that matters in terminal emulators though. Correctness is critical.

    The only terminals in which I have any confidence of correctness are xterm and pangoterm. And I suppose technically the BEL-for-ST extension is incorrect even there, but we have to live with that and a workaround is available.

    A lot of terminal emulators end up hard-coding a handful of common sequences, and fail to correctly ignore sequences they don’t implement. And worse, many go on to implement sequences that cannot be correctly handled.

    One simple example that usually fails: \e!!F. More nasty, however, are the ones that ignore intermediaries and execute some unrelated command instead.

    I can’t be bothered to pick apart specific terminals anymore. Most don’t even know what an IR is.




  • Even logging can sometimes be enough to hide the heisgenbug.

    Logging to a file descriptor can sometimes be avoided by logging to memory (which for crash-safety includes the possibility of an mmap’ed file, since the kernel will just take care of them as long as the whole system doesn’t go down). But logging from every thread to a single section of memory can also be problematic (even without mutexes, atomics can be expensive and certainly have side-effects) - sometimes you need a separate per-thread log, and combine in the log-reader tool.





  • I haven’t managed to break into the JS-adjacent ecosystem, but tooling around Typescript is definitely a major part of the problem:

    • following a basic tutorial somehow ended up spending multiple seconds just to transpile and run “Hello, World!”.
    • there are at least 3 different ways of specifying the files and settings you want to use, and some of them will cause others to be ignored entirely, even though it looks like they should be used.
    • embracing duck typing means many common type errors simply cannot be caught. Also that means dynamic type checks are impossible, even though JS itself supports them (admittedly with oddities, e.g. with string vs String).
    • there are at least 3 incompatible ways to define and use a “module”, and it’s not clear what’s actually useful or intended to be used, or what the outputs are supposed to be for different environments.

    At this point I’m seriously considering writing my own sanelanguage-to-JS transpiler or using some other one (maybe Haxe? but I’m not sure its object model allows full performance tweaking), because I’ve written literally dozens of other languages without this kind of pain.

    WASM has its own problems (we shouldn’t be quick to call asm.js obsolete … also, C’s object model is not what people think it is) but that’s another story.


    At this point, I’d be happy with some basic code reuse. Have a “generalized fibonacci” module taking 3 inputs, and call it 3 ways: from a web browser on the client side, as a web browser request to server (which is running nodejs), or as a nodejs command-line program. Transpiling one of the callers should not force the others to be transpiled, but if multiple of the callers need to be transpiled at once, it should not typecheck the library internals multiple times. I should also be able to choose whether to produce a “dynamic” library (which can be recompiled later without recompiling the dependencies) or a “static” one (only output a single merged file), and whether to minify.

    I’m not sure the TS ecosystem is competent enough to deal with this.