HNNewShowAskJobs
Built with Tanstack Start
A job queue in two lines of JavaScript(jameshfisher.com)
57 points by chmaynard 6 days ago | 39 comments
  • lerp-io2 days ago

    The chain variable continuously grows because each call to enqueue() creates a new promise that references the previous one. This creates an ever-growing chain of promises that never gets garbage collected.

    • paulddraper2 days ago |parent

      Though it seems like that, no.

        a = b.then(c);
      
      When `b` is resolved, it is garbage collected, even if a reference to `a` is retained.

      (If the runtime maintains async stack traces, that could be an issue...but that is a common issue, and the stack depth is limited.)

    • bapak2 days ago |parent

      That's not how promises work, there's no chain. It's no different from

          await job1()
          await job2()
          await job3()
          await job4()
      
      Except dynamic
    • ath922 days ago |parent

      Couldn’t the browser garbage collect the promises (and their callbacks) that have been rejected or resolved? I.e. effectively “shift” the promises at the start of the queue that have been dealt with and will never be relevant again?

      At least this stackoverflow answer suggests the handlers are GC’ed: https://stackoverflow.com/questions/79448475/are-promise-han...

    • ameliaquining2 days ago |parent

      At least in V8, this is not the case. To prove it, start a Chromium-based browser with the command-line flag --js-flags=--expose-gc, then go to https://runjs.app/play/#aWYgKCF3aW5kb3cuZ2MpIHsKICB0aHJvdyBu...

    • jameshart2 days ago |parent

      Why does the promise returned by a promise’s then() method need to reference that promise?

      The original promise needs to reference the chained promise, not the other way round.

      As jobs complete I would expect the top of the chain to be eligible to be garbage collected.

    • bigiain2 days ago |parent

      > This creates an ever-growing chain of promises that never gets garbage collected.

      Modern frontend development performance best practise is to allow for garbage collection only when the browser crashes.

    • Inviza day ago |parent

      I wonder if people who are claiming its not the case ever tried to use promises in hardcore ways like this, chaining 10s of thousands of calls. You often hit some bullshit, i.e. Promise.race is a big cause of issues.

      Even if you are doing `then().then()`, something _else_ in the code base could retain the promise somehow and then your whole chain isn't GCable!

      For example https://github.com/rxaviers/async-pool library that implements concurrency for async iterators (and uses promise.race) subtly creates GC pressure which you dont see until you make a whole lot of calls and suddenly its slow

    • 2 days ago |parent
      [deleted]
    • moralestapiaa day ago |parent

      Nope.

    • pinoy4202 days ago |parent

      [dead]

  • 90s_dev2 days ago

    I came across this trick a few months ago when I was dealing with what seemed to be a race condition from a chrome-only API, and I just felt so clever! I don't remember though whether the race condition actually was one or not though, and I eventually had to rip that entire class out of my codebase. But it's such a good feeling to know I came up with a solution that clever all by myself.

  • Inviz2 days ago

    Chaining promises like this is not a good idea for high throughput cases (creates gc pressure), but perfectly valid for regular cases.

    • paulddraper2 days ago |parent

      Huh?

      It creates 1 object allocation per enqueue.

      • Inviz2 days ago |parent

        yeah but it creates closure as well, which typically references some objects that isn't easy to GC. So if you have long-living promise with some data captured in closure, it will not be cleared. So doing then().then().then() may not release objects referenced in callbacks that resolved time ago.

        • paulddraper2 days ago |parent

          There are zero closures here.

  • ameliaquining2 days ago

    To make it slightly more idiomatic, you can use chain.finally(job) instead of chain.then(job, job). (All browsers have supported this API for over seven years.)

  • theThree2 days ago

    Something like this?

    async function runTasks(tasks: Job[]) { for (let task of tasks) { try { await task() } catch (e) { } } }

    • foxygen2 days ago |parent

      This only works if you have the full list of tasks beforehand.

  • cluckindan2 days ago

    If you wanted to e.g. log something on failures, you’d need to do something like this:

        chain
          .then(job)
          .catch((err) => {
            console.error(err);
            return job();
          });
    
    Otherwise failures would need to be logged by the job itself before rejecting the promise.
  • egorfine2 days ago

    > If you need a job queue in JS

      type Job = () => Promise<unknown>;
    
    This is not JS.
    • ameliaquining2 days ago |parent

      It's trivial to remove the type annotations.

  • vivzkestrel2 days ago

    A production grade application would need a much more robust mechanism like BullMQ

    • ameliaquining2 days ago |parent

      There are use cases for a non-persistent task queue, like doing things one at a time on the frontend. (JavaScript is single-threaded but will interleave multiple separate async call chains that each span multiple event-loop iterations, and there could be situations where you might not want that.)

  • neals2 days ago

    Anybody willing to walk me through this code? I don't get it.

    • foxygen2 days ago |parent

      chain is a global variable. It starts with an empty promise. First call to enqueue changes the (global)value of chain to emptyPromise.then(firstJob), second call to enqueue changes it to emptyPromise.then(firstJob).then(secondJob) and so on.

    • fwlr2 days ago |parent

      JavaScript promises are objects with a resolver function and an internal asynchronous computation. At some point in the future, the asynchronous computation will complete, and at that point the promise will call its resolver function with the return value of the computation.

      `prom.then(fn)` creates a new promise. The new promise’s resolver function is the `fn` inside `then(fn)`, and the new promise’s asynchronous computation is the original promise’s resolver function.

  • gabrielsroka2 days ago

    That's TS, not JS.

    • jameshart2 days ago |parent

      One line of TS, then two lines of JS (with a type annotation in one of them)

      • goloroden2 days ago |parent

        It’s not even 2 lines:

        1. Type definition 2. Chain definition 3. Enqueue definition

        • jakelazaroff2 days ago |parent

          Line 1 is fully erased when transpiling, leaving two lines of JavaScript.

          • Brian_K_White2 days ago |parent

            But you still had to write it and it's required to get the result, so I don't see any rational for not counting it.

            Transpilation does not change the fact that the input is these 3 lines of code in 2 different languages.

            • jakelazaroff18 hours ago |parent

              Right, and only two of those lines are JavaScript. The type definition is not required to get the result.

    • 90s_dev2 days ago |parent

      For now.

      https://github.com/tc39/proposal-type-annotations

    • metadat2 days ago |parent

      I noticed the same thing when I saw the code. Is it honest for TFA to title it as "2-lines of Javascript" in this case?

    • egorfine2 days ago |parent

      I came here to make the same comment.

      I wonder why are you downvoted for being as factual and neutral as it is technically possible.

      • ameliaquining2 days ago |parent

        I think because a reader who doesn't want to use TypeScript can trivially remove the type annotations, so they're not actually a barrier to using this code in JavaScript. Some people might also be suspicious that the reason to bring this up is to imply hostility to JavaScript tooling that requires a build step, since it's become popular to hate on this.

        • egorfine2 days ago |parent

          So as a community of developers we're totally okay calling TypeScript "JavaScript" because ... mentioning that hard cold fact is a sign of hostility towards TS? Ok. Got it.

    • dcre2 days ago |parent

      Oh, come on.