@prologic@twtxt.net The headline is interesting and sent me down a rabbit hole understanding what the paper (https://aclanthology.org/2024.acl-long.279/) actually says.

The result is interesting, but the Neuroscience News headline greatly overstates it. If I’ve understood right, they are arguing (with strong evidence) that the simple technique of making neural nets bigger and bigger isn’t quite as magically effective as people say — if you use it on its own. In particular, they evaluate LLMs without two common enhancements, in-context learning and instruction tuning. Both of those involve using a small number of examples of the particular task to improve the model’s performance, and they turn them off because they are not part of what is called “emergence”: “an ability to solve a task which is absent in smaller models, but present in LLMs”.

They show that these restricted LLMs only outperform smaller models (i.e demonstrate emergence) on certain tasks, and then (end of Section 4.1) discuss the nature of those few tasks that showed emergence.

I’d love to hear more from someone more familiar with this stuff. (I’ve done research that touches on ML, but neural nets and especially LLMs aren’t my area at all.) In particular, how compelling is this finding that zero-shot learning (i.e. without in-context learning or instruction tuning) remains hard as model size grows.

@prologic@twtxt.net +1 for FrankenPHP. And built into caddy is also swell.

@movq@www.uninformativ.de Variable names used with -eq in [[ ]] are automatically expanded even without $ as explained in the “ARITHMETIC EVALUATION” section of the bash man page. Interesting. Trying this on OpenBSD’s ksh, it seems “set -u” doesn’t affect that substitution.

@movq@www.uninformativ.de It’s hot here as well. Luckily should only last a couple of days. Bunkering down in our home and keeping all the doors and windows closed. No airco. Fans give some relieve.

@prologic@twtxt.net 35°C outside. 🫤 I’m just gonna sit here and wait for November. 😂

@movq@www.uninformativ.de If it still existed I bet the first thing he’d do is convert it to Golang 👌🤣

@prologic@twtxt.net I don’t know what you mean when you call them stochastic parrots, or how you define understanding. It’s certainly true that current language models show an obvious lack of understanding in many situations, but I find the trend impressive. I would love to see someone achieve similar results with much less power or training data.

@prologic@twtxt.net I thought “stochastic parrot” meant a complete lack of understanding.

@movq@www.uninformativ.de The success of large neural nets. People love to criticize today’s LLMs and image models, but if you compare them to what we had before, the progress is astonishing.

@prologic@twtxt.net @movq@www.uninformativ.de Nothing can beat AI powered rice cooker! 😜

@falsifian@www.falsifian.org I do like to think of that recommendation as sarcastic stab at twt-errr’s rules 😉 so, IMO the more a twt exceeds the recommended limit the merrier! 😂

@prx@si3t.ch ROFL. 🤣 Come on! That’s evil. At least give them cup of hot chocolate, make it a win-win.

@prologic@twtxt.net Thanks. It’s from a non-Euclidean geometry project: https://www.falsifian.org/blog/2022/01/17/s3d/

@prologic@twtxt.net Thanks for the invitation. What time of day?

@falsifian@www.falsifian.org by the way, on the last Saturday of every month, we generally hold a online video call/social meet up, where we just get together and talk about stuff if, you’re interested in joining us this month.

@prologic@twtxt.net Fair enough! I just added some metadata.

Thanks @prologic@twtxt.net! I like the way Yarn.social is making all of twtxt stronger, not just Yarn.social pods.

@falsifian@www.falsifian.org You need an Avatar 😅

@movq@www.uninformativ.de Thanks!

Hello twtxt! I’m James (or @falsifian@www.falsifian.org). I live in Toronto. Recent interests include space complexity, simple software, and science fiction.

@movq@www.uninformativ.de HAHA 😂 How come I’ve never seen that one before!? Weeeeeeeeeeeeehehehe 😂

@lyse@lyse.isobeef.org Milk crates aren’t that practical to be honest. Especially when you have a nosy cat around. 😂 but it was worth a try.

@prologic@twtxt.net There is a Typo in the link! is it intentional? 🤔

@prologic@twtxt.net HAHA! Couldn’t say it better. I started abandoning main stream social media as soon as is it stopped feeling like connecting and sharing with other human beings and became an urge for feeding an algorithm and hoping for it’s blessing to get a glimpse of human interaction It deems worthy of having.

@aelaraji@aelaraji.com Because we don’t have milk crates here in Germany. :-D At least I never came across them for milk, just the cardboard boxes for the milk tetra paks. But they don’t hold the weight of a monitor.

@movq@www.uninformativ.de LOL²!

@aelaraji@aelaraji.com Exactly! 😂

It’s not what I meant (I was referring to the motor of the desk making a whirring sound 😂), but now I’m reminded of this: https://www.youtube.com/watch?v=9sKppwrLBY8

@movq@www.uninformativ.de A good slogan for an ad: “Wwwwwwwwwwwwwwwwwwww! a hell of a bang for your buck!” 😂

@xuu@txt.sour.is foo who? 😆

@prologic@twtxt.net Remember when we used to lose access to e-mail, IM and forum accounts after 30 days of inactivity? 😂 … Then storage became cheaper and companies figured out that any tiny bit of someone’s data is worth something to someone(thing) else. 🥲

@movq@www.uninformativ.de my bad man. I left off a return in the formatter func. I have a PR to fix waiting on @prologic@twtxt.net

@lyse@lyse.isobeef.org wow on my browser it shows up as all stars! •••••••

Let me suggest to use a more secure password, @bender@twtxt.net. One, that does not contain “password”. Like hunter2!!

@prologic@twtxt.net ‘Clownflare’ 🤣🤣🤣 Love it.

But yes the idea of a cheap VPS as a tunnel and keeping home network all local is a good one I reckon.

@bender@twtxt.net / @mckinley@twtxt.net could you both please change your password immediately? I will also work on some other security hardening that I have a hunch about, but will not publicize for now.

@shreyan@twtxt.net Haha my criteria is being inactive for over two years 🤣

@aelaraji@aelaraji.com Ahh it might very well be a Clownflare thing as @lyse@lyse.isobeef.org eluded to 🤣 One of these days I’m going to get off Clownflare myself, when I do I’ll share it with you. My idea is to basically have a cheap VPS like @eldersnake@we.loveprivacy.club has and use Wireguard to tunnel out. The VPS becomes the Reverse Proxy that faces the internet. My home network then has in inbound whatsoever.

It ended in a crash but I dreamt it thank God @lyse@lyse.isobeef.org

@prologic@twtxt.net I’m using CF Tunnel on a raspberry pi, can’t do direct at the moment.

@grumpy@grumpygordie.great-site.net Umm… If you haven’t already, you can give this Jenny + Mutt setup a try, you’ll probably (I mean definitely) love it. Nice to meet you by the way! 🫡

@prologic@twtxt.net Probably has something to do with how the nickname is set up, It starts with a capital letter and has a white space. I couldn’t fetch their feed until I fixed that in my ‘follow’ file. But I dunno, maybe it’s just me…

@prologic@twtxt.net @lyse@lyse.isobeef.org I checked my logs and all I see are 304 responses and a couple of delayed requests here and there due to rate limiting, but not that many. I’ll disable it (the rate limiting) for a couple of days, let me know if you still get the ‘forbidden access’ thing 🫣 I may have effed up my configuration trying to deal with some weird stuff.

@prologic@twtxt.net I am! 😅 I’ll check my logs and see if there’s something I can do about that!

@lyse@lyse.isobeef.org Ahh so it’s not just me! 😅

I’m working on getting my twtxt.txt file up to https://yarn.social standards so that it will be more than yelling in the wind.

@prologic@twtxt.net Good to know. I must admit I’ve never actually used a Docker instance, probably as I just assumed the overhead might be a bit much for my usual very modest servers.

@bender@twtxt.net Is it so maxed out you couldn’t fit a pretty small program like Headscale on it? Headscale by itself and only personal home type use as far as amount of peers go, it really isn’t noticeable I don’t think resource-wise. The Docker version I guess could be a different story.

@bender@twtxt.net Mine is about the same, though I have 20GB left 😅 In terms of resources, Headscale is using next to nothing though.

@eldersnake@we.loveprivacy.club how big is that VPS, if you can tell? My 1 vCPU, 2GB, 50GB is maxed out. 😬

@prologic@twtxt.net Yes I suppose that is true. There is an article on Tailscale’s site that explains it all quite a bit: https://tailscale.com/blog/how-nat-traversal-works

To me, with CGNAT, it’s a small miracle that a direct connection can be made between peers (as opposed to going through a relay constantly) but it does indeed work. I guess to host it at home you would need to have it WAN accessible, and if you’ve already gone to the trouble of port forwarding etc… well 😅
Not that I could personally do that, but for those with static IPs etc.

@bender@twtxt.net on my hosted VPS, as I’m on Starlink which is CGNAT, I need some sort of external intermediary.