• 0 Posts
  • 548 Comments
Joined 3 years ago
cake
Cake day: June 10th, 2023

help-circle





  • Let’s do some estimates:

    • An 8x H100 machine costs about $20 / hr to rent.
    • With a 70B model with 4K context, a H100 node can do about 300 requests in parallel.
    • A single response takes around 30 seconds to generate.
    • An average user sends about 300 messages / month.

    The throughput of a node is

    300 concurrent * (3600 / 30) = 36 000 messages / hour.

    The cost per message, then, is $20 / 36 000 = $.00055…

    With 300 messages per month, the compute cost for the AI vendor is 300*$20/36000 = $0.16 / month per user. By contrast, a subscription costs $20.

    So given these assumptions, it’s other things (like R&D, safety research, training runs, free accounts, etc) that represent the bulk of the cost and those could be scaled down to turn a profit. What will they do? Give how hyped AI is currently and the competitive landscape, I don’t think they’ll increase prices that much. We have products like DeepSeek on the horizon which are much cheaper, so it’s more likely that they squeeze money out of it by becoming more efficient.













  • Adobe has no scruples.

    I used to work for a full-disk encryption vendor in the 2000s, and one customer had an issue where the machine would BSOD sometimes if both our product and Adobe Acrobat were installed. It seemed a mystery or just a red herring - what on Earth did Acrobat do that could trigger a kernel-mode crash?

    Turned out that every hour or so, Acrobat would be reading and writing back the master boot record (containing the OS bootstrap code and partition table) on the primary hard drive. The bug was ours (to unlock the hard drive keys at boot we had to put different data there and redirect I/O after Windows started, and this redirection code would crash once in a blue moon), but Adobe has no business mucking about with this extremely sensitive data.