A software developer and Linux nerd, living in Germany. I’m usually a chill dude but my online persona doesn’t always reflect my true personality. Take what I say with a grain of salt, I usually try to be nice and give good advice, though.

I’m into Free Software, selfhosting, microcontrollers and electronics, freedom, privacy and the usual stuff. And a few select other random things, too.

  • 2 Posts
  • 29 Comments
Joined 6 months ago
cake
Cake day: June 25th, 2024

help-circle
  • I think this warrants an extra post. And the beginners thread is a year old and I guess not a lot of people watch comments there.

    I use KoboldCpp and like to recommend that to people who are new to the hobby or don’t own a proper gaming rig. It’s relatively easy to install and you can try it now, without any GPU, and see if you like it. I’d say it’s usable on CPU up to about 13B (with quantized models). Of course it’ll be orders of magnitude slower than a GPU.

    I’d say every bit of VRAM counts. So you might as well buy as much as you can afford. And you’ll be able to run more intelligent models. Use one of the VRAM calculators to see what fits in 16GB or 24GB. And if you need that model and context size.


  • A VM is practically as secure as a dedicated machine. I mean in theory it isn’t. But in practice, that’s how everybody does it, including the big tech companies. And there’s rarely any dangerous vulnerabilities.

    It all depends on how you set it up. If the machines lack proper firewalling and can reach internal services. If you didn’t set some permissions right. Or there is a vulnerability in the software or the way it’s installed… I think that’d be the main concerns. And it doesn’t matter too much which exact virtualization or containerization setup you choose, as long as it provides isolation, and also isolation from the networks it isn’t supposed to access.








  • Idk if kids are a target group of Nobara Linux, but I heard that’s good for gaming.

    Rest sound good. Don’t forget to give them lots of productivity tools… OBS, Kdenlive, LMMS so they can practice shooting videos, create music… They’d need LibreOffice, maybe an IDE to learn coding, or something to create a website. As a kid I had a lot of fun tinkering with the Free Software tools, next to gaming.

    Test the website blocking, if you restrict for example porn. Nowadays the browsers all try to do some privacy enhanced DNS over HTTP and that might circumvent the default DNS. I’m not up to date with the current solutions to restrict usage… Maybe someone can chip in, I’m pretty sure that’s available.


  • The big one is fine. I think it’s already grown up. And does most of the chores around. And the small one: I tend to bring it with me. I carry it in my backpack, especially to some nerd activities. I don’t think there is any rivalry. They’re individuals and I think my machines all feel my love and affection. No, I don’t miss them in day to day life. We’ll meet each afternoon anyways and I think some leeway is healthy. But that changes if I go on vacation for a week. I sometimes really miss them when I’m in a different city for several days… Idk. Seems one-sided anyways, because they then just sleep all day like a cat and don’t really mind my absence.


  • They’re awesome. There are foldable ones which you can carry around. Or big ones that you don’t even need to charge, because they’re directly hooked to the power grid. You don’t need to hold them, because they sit on the table. Have about 105 separate keys (ever wondered what your 8 other fingers are for?), a mega large screen… Mine even has 2 screens, I can open like 5 apps at the same time! They have headphone jacks, sd-slots, multiple(!) charging ports which you can use for other things than charging… almost endless storage… And there are very smooth operating systems available, which don’t even lock you into Google or Apple’s ecosystem… I really like them!

    Best thing is, they won’t even drop out of cellphone coverage when you go grocery shopping. You’ll just leave them at home and they keep on downloading. Once you return, you might have all bad sitcoms of the 90s saved on them.










  • I skimmed the link you provided. Yes, that seems to include solid advice. Good for beginners, nothing new to me, since I (somewhat) followed the AI hobby enthusiast community since LLaMA1. But I have to look up what writing all caps does, I suppose that severely messes with the tokenizer?! But I’ve seen the big companies do this, too, in some of the leaked prompts.

    And I guess with the “early” models from 2023 and before, it was much more important to get the prompts exactly right. Not confuse it etc. That got way better as models improved substancially, and now these models (at least) get what I want from them almost every time. But I think we picked the low hanging fruits and we can’t expect the models itself to improve as fast as they did in the past. So it’s down to prompting strategies and other methods to improve the performance of chatbots.