Lettuce eat lettuce

Always eat your greens!

  • 11 Posts
  • 672 Comments
Joined 3 years ago
cake
Cake day: July 12th, 2023

help-circle

  • I personally think that general consumers will never use LLMs in any significant number. I think that LLMs will exist in two distinct spaces, FOSS for devs and other technical people who want to run there own infra locally - and B2B for everything else.

    The few big AI companies that manage to last will be selling access to their models for much higher prices. Probably similar to current proprietary commercial software like VMWare, SolidWorks, VEEAM, Splunk, etc. Companies will pay hundreds, possibly thousands of dollars per seat depending on the niche offering and amount of usage.

    Suppose that a company developed an LLM that is trained & tuned specifically to do legal work, and suppose it produced work that was around 95% the quality of a typical paralegal. If that company charged $6,000 a year per license to work on their platform, that’s expensive, but if you’re a small firm with say, a dozen full time lawyers, then for the yearly price of a single average paralegal, you could have each lawyer using that software to do most of the work that the paralegal would have done. I can see those kinds of applications happening more and more.

    This assumes though that LLMs will continue to improve at a significant rate for a long time into the future, (5-10 more years) which isn’t at all obvious, and there is some evidence that it’s already starting to hit a ceiling.

    There are other ways it might work, like if there is a method of compression that is discovered that reduces the necessary RAM and Compute needs by 2-3 orders of magnitude. So models that are considered very large today (100-300 billion params at full quality) might be able to run effectively on a single 32GB GPU that costs a few thousand dollars.

    So the cost to run these models is reduced immensely, and a single small data center could run enormous models with 1,000,000+ context windows for tens of thousands of users at once.

    But that cuts both ways, which is something that any AI company is going to have to deal with. Once small free models get good enough to do the vast majority of a task, a user is going to start weighing the cost/benefits, and the prospect of just buying a box and throwing one of these models in for a few grand will be very appealing.

    I think there may be a good market out there for “AI boxes”, compact computers designed to run a tuned LLM, set up with a little special sauce so the interface is user-friendly, etc. Companies could sell these with support contracts to legal firms, indie Dev studios, startups, small government agencies, etc.

    Idk, it’s so up in the air right now, and everything is constantly changing so fast. It’s impossible to predict where things will be in 6 months, let alone 6 years from now.


  • I’ve been loving Incus containers for this very use case. Unlike Docker, Incus containers are by default persistent, and are full system containers, not just applications. So when you launch an Incus Debian 13 container for instance, you get a full Debian 13 installation, but at a fraction of the size of even a small traditional VM.

    It’s a great happy medium between Ultra-minimal Docker containers designed for single applications, and old-school heavy VMs.



  • Gaming PC - Nobara (Fedora base with lots of gaming-specifc kernel optimizations baked in.)

    Personal laptop - Linux Mint

    Business laptop - Linux Mint Debian Edition

    Junk/Test laptops - Void

    Home lab main hypervisor - XCP-ng (Highly customized Fedora under the hood.)

    NAS - TrueNAS (Debian under the hood.)

    Virtual servers - Mostly Debian, but a few Alma Linux VMs to get that RHEL experience. Ubuntu Server for my self-hosted gaming servers.

    Steam Deck - SteamOS (Valve’s immutable spin of Arch.)


    1. Typically, but not always. Some trans women are biologically intersex. (This also depends on how you define “biologically male” which is not totally straightforward.)
    2. It matters in some contexts, not in others. Their physician should know, because various hormone treatments cause different effects in people’s bodies, and certain health conditions effect biologically male or female people differently too. That’s nobody else’s business but the patient and their trusted medical providers. As far as their dignity, opportunities, and general acceptance, it doesn’t matter. Trans folks deserve the exact same rights, opportunities, and acceptance as anybody else.
    3. Usually people who bring this up aren’t acting in good faith, so I don’t engage with them. On the rare occasion where somebody is genuinely curious and wants to learn, I answer them in the same way as I am doing right now.
    4. Because the word “woman” denotes multiple concepts, like the word “parent”. If a child is adopted at birth and is raised by a couple, the child and their community will refer to those people as the child’s parents. This is not a false statement, because the word “parent” doesn’t only mean the direct biological progenitors of a person. Parent also is a social role, hence the verb form “to parent somebody.” This is also why we have the terms, “biological parent” and “adoptive parent” to add additional information when it’s necessary.

    Trans women are women in the sense that they are filling their society’s sociological role that surrounds the expected concept of a woman. That will be different depending on many factors, and will have many different aspects including their pronouns, fashion and clothing, voice, makeup, hair, activities, and so forth.

    Just like any other woman, they will chose which social roles they desire to fit into, and which ones they don’t, and all of that is completely acceptable.


  • “My friend says the story is stupid and no one would want to read it.”

    That’s not real constructive feedback. If your friend has actual critiques of your concept, that’s one thing, but just saying something seems stupid is meaningless and carries zero weight.

    Don’t let people live their life and your life too. If you’re passionate about an idea, try it and see if it works. Worse case scenario, it fails, and you learn from it and get lots of practice for your next idea. Which still might be bad, but it will almost certainly be less bad, and same with the next, and the next, and before you know it, you have hundreds of hours of practice and experience and you’re creating real cool stuff.

    Also, sometimes ideas are good, but you currently lack the skill to execute them well. That just means you need to increase your skill level. An idea that fails badly when you first start out, might turn out fantastic 5 - 10 years down the road.

    Film directors/writers sometimes talk about this, where they had an idea or a script for a movie that they wanted to make, but they didn’t have the budget and necessary experience to do it justice early in their career.

    TL;DR Your friend’s “feedback” is worthless, if you’re really passionate about this idea, go for it. Worst case scenario, you gain a bunch of experience trying to make it.




  • I’ve been using Graphene for several years and I love it. I could never go back now, Google android feels so incredibly bloated and invasive by comparison.

    Double check your backups just to be safe, and then go for it. It’s not hard to revert if you hate it. There is a big of a learning curve, mainly just using the alternative app stores like Accresent, F-Droid, etc.

    But once you spend a bit of time getting your apps installed and your system set up the way you like, you’ll love it.







  • Thanks for the response. I’m doing great now. Got a new job as a sysadmin making about 35% more than my old job, and I get to work on Linux a bunch, and my team is really solid.

    Still sucks that I lost all that work, but I was able to get some of the old hardware back for free, so my old servers can live again in my home lab.





  • Sorry for your loss :( Same thing happened to me about a year ago.

    I was the sole IT admin for a small company. Used Debian with KDE on a snappy little Thinkpad. No issues managing all the infra with it, even though most of it was MS trash. I used Reminnia for RDP into the Windows servers, and the Browser for all O365/Entra administration. A Windows 11 VM for the rare times I needed to test Windows-only apps or configs.

    Worked like a dream, but then we got bought out by a huge competitor. Their IT team took everything over. I had to decommission my on-prem Linux servers, Ansible automations, Open Project tracking and FOSS ticketing system. Finally, I had to give up my Sweet little Linux Thinkpad and use their standard-issue HP Windows 11 garbage laptop. They were slow, clunky, buggy, and ugly, it was awful.

    I quit a few months later after securing the job I have now. It pays about 35% more, has twice as much PTO, and about 50% of my workload is Linux stuff. It’s so much better.

    My advice, if it’s truly non negotiable, install WSL first thing. It’s not nearly as good as having actual Linux, because it’s running inside of Microslop’s horrid OS, but it’s better than nothing. Try to be an advocate for FOSS at the company, see if you can convince leadership to let you implement Linux-based solutions wherever they might fit, make yourself the de facto expert on them so you at least get to work on Linux and FOSS infra.

    Aside from that, start job hunting. Try to find a job that will let you be more Linuxy.