robots.txt file. only noticed it because the OpenAI bot was hitting me with a lot of nonsensical requests. here is the list from last month:
@bmallred@staystrong.run Surprisingly, my
User-agent: *
Disallow: /
seems to work. Or maybe those bastards change their user agent and claim to be someone nice. In any case, I just added a bunch of
location = /robots.txt {
add_header Content-Type text/plain;
return 200 "User-agent: *\nDisallow: /\n";
}
in my nginx config. No need for any bot to visit, crawl and index most of my sites.
@movq@www.uninformativ.de Photographic memory, eh?
@movq@www.uninformativ.de I also thought that I have a new Linux friend the other day. But it was just a fake KDE look from Redmond. :-(
Totally agree @jost@jost.sdfeu.org @prologic@twtxt.net
I have uploaded a new version of #twtxtel 🥳. It’s now possible to view profiles, either your own or others. #twtxt #emacs @prologic@twtxt.net @xuu@txt.sour.is
reviewing logs this morning and found i have been spammed hard by bots not respecting the robots.txt file. only noticed it because the OpenAI bot was hitting me with a lot of nonsensical requests. here is the list from last month:
- (810) bingbot
- (641) Googlebot
- (624) http://www.google.com/bot.html
- (545) DotBot
- (290) GPTBot
- (106) SemrushBot
- (84) AhrefsBot
- (62) MJ12bot
- (60) BLEXBot
- (55) wpbot
- (37) Amazonbot
- (28) YandexBot
- (22) ClaudeBot
- (19) AwarioBot
- (14) https://domainsbot.com/pandalytics
- (9) https://serpstatbot.com
- (6) t3versionsBot
- (6) archive.org_bot
- (6) Applebot
- (5) http://search.msn.com/msnbot.htm
- (4) http://www.googlebot.com/bot.html
- (4) Googlebot-Mobile
- (4) DuckDuckGo-Favicons-Bot
- (3) https://turnitin.com/robot/crawlerinfo.html
- (3) YandexNews
- (3) ImagesiftBot
- (2) Qwantify-prod
- (1) http://www.google.com/adsbot.html
- (1) http://gais.cs.ccu.edu.tw/robot.php
- (1) YaK
- (1) WBSearchBot
- (1) DataForSeoBot
i have placed some middleware to reject these for now but it is not a full proof solution.
@movq@www.uninformativ.de Oh man this made my morning!
Pinellas County - 5 mile run: 5.05 miles, 00:09:12 average pace, 00:46:32 duration
slept like shit. fought back the urge to shit for four fifths of the run. bleh.
#running
@lyse@lyse.isobeef.org That is how it feels for sure
Atom
⌘ Read more
@jost@jost.sdfeu.org Happens to everybody. 😂 Well, except for a few people. I have this colleague at work who remembers everything. It’s scary. 😅
Recovery run: 3.11 miles, 00:10:31 average pace, 00:32:39 duration
KDE Plasma 6.3 brings drawing tablet improvements
Speaking of KDE, Plasma 6.3 has been released. It brings with it a ton of improvements aimed at digital artists, such as much improved management and configuration of drawing tablets. You can now map an area of the tablet’s surface to a part of the screen, change the functions of stylus buttons, customise the pressure curve and range of a stylus, and much more. The entire settings panel for drawing tablets has also been redesigned t … ⌘ Read more
Moving KDE’s styling into the future
One of the major issues with KDE’s styling system is the fact that over the year, it has accumulated four ways of styling applications – which makes themeing and changing aspects of the default theme far more cumbersome than it should be. In fact, with the current version of KDE, it’s effectively impossible to consistently theme the entire KDE desktop, as several parts of it, like Kirigami applications, only inherit parts of the theme you’re applying. It’s a … ⌘ Read more
@jost@jost.sdfeu.org Yeah, this AI crap is a big reason not to blog.
@andros@twtxt.andros.dev The article is a good reminder of the true blogging mindset. But let’s try to think beyond. 2 ideas: (1) writing “forces clarity, structures your thoughts, sharpens your perspective”. But it also generates thoughts in the sense of Heinrich von Kleist (1805). (2) You’re writing for “the future you, one right person, one day” but you are also writing for the AI. The idea of AI as an audience.
Pinellas County - 3 mile run: 3.17 miles, 00:09:12 average pace, 00:29:07 duration
legs are soooo tight. felt light though.
#running
Oasis: a small, statically-linked Linux system
You might think the world of Linux distributions is a rather boring, settled affair, but there’s actually a ton of interesting experimentation going on in the Linux world. From things like NixOS with its unique packaging framework, to the various immutable distributions out there like the Fedora Atomic editions, there’s enough uniqueness to go around to find a lid for every pot. Oasis Linux surely falls into this category. One of its main … ⌘ Read more
Redox’ relibc becomes a stable ABI
The Redox project has posted its usual monthly update, and this time, we’ve got a major milestone creeping within reach. Thanks to Anhad Singh for his amazing work on Dynamic Linking! In this southern-hemisphere-Redox-Summer-of-Code project, Anhad has implemented dynamic linking as the default build method for many recipes, and all new porting can use dynamic linking with relatively little effort. This is a huge step forward for Redox, because relibc can now beco … ⌘ Read more
Rediscovering Plan 9 from Bell Labs
During a weekend of tidying up – you know, the kind of chore where you’re knee-deep in old boxes before you realize it. Digging through the dusty cables and old, outdated user manuals, I found something that I had long forgotten: an old Plan 9 distribution. Judging by the faded ink and slight warping of the disk sleeve, it had to be from around 1994 or 1995. I couldn’t help but wonder: why had I kept this? Back then, I was curious about Plan 9. It was a forwar … ⌘ Read more
NASA tests new nuclear rocket fuel + 2 more stories
NASA and General Atomics test nuclear fuel; Ukraine employs more drones for war effort; engineered bacteria shows promise in recycling nylon waste. ⌘ Read more
FreeBSD and hi-fi audio setup: bit-perfect, equalizer, real-time
A complete guide to configuring FreeBSD as an audiophile audio server: setting up system and audio subsystem parameters, real-time operation, bit-perfect signal processing, and the best methods for enabling and parameterising the system graphic equalizer (equalizer) and high-quality audio equalization with FFmpeg filters. Linux users will also find useful information, especially in the context of configuri … ⌘ Read more
Three years of ephemeral NixOS: my experience resetting root on every boot
We had a bit of a bug caused by changes we made to make quotes look better, but we’ve fixed it now, so we’re back on track (you may need to do a force-reload in your browser). Sorry for the disruption – and if you want to stay up-to-date on such issues next time it (inevitably) happens, you should follow the OSNews Fedi account (or just bookmark it without following it, if you’re not … ⌘ Read more
Excellent article where you reflect on why it is important to write in your blog, even knowing that nobody will read it.
https://andysblog.uk/why-blog-if-nobody-reads-it/
At least this article does.
Incoming Asteroid
⌘ Read more
@movq@www.uninformativ.de So they say. :-D
I’m not a huge fan of docker. Sorry for the poor screen grab quality, but this is the funniest analogy for “no docker” vs “docker” I’ve come across: https://lyse.isobeef.org/tmp/no-docker-vs-docker-analogy.png :-D
Holly Hill - Long run: 12.06 miles, 00:09:43 average pace, 01:57:15 duration
did not sleep last night. the bed was not too comfortable and i was burning up for some reason. the run went well. it was a decent temperature and i kept the pace pretty moderate (mainly around a 9:30). hit two bridges going back-and-forth between daytona. i did walk a bit around mile ten to recollect myself but a good run nonetheless.
my daughter got first all-around in her gymnastic meet and did really well on all events, too!
#running
@prologic@twtxt.net we need to remove: https://git.mills.io/yarnsocial/go-lextwt/src/branch/main/ast.go#L776-L784
apparently i can’t make the edit via gitea.. i am guessing its hitting one of your firewall rules.
@xuu@txt.sour.is da fuq?!
@andros@twtxt.andros.dev Awesome! I’ve seen the demo earlier on mastodon, things are getting better and better with each update 👌 Good luck!
Pinellas County - 6 mile run: 6.05 miles, 00:08:49 average pace, 00:53:18 duration
pretty good run. been tough logging this last week or so with all the work, but its going well. there was a lot of people waiting to get in to walsingham park today… probably the 5km event i saw posted a couple of weeks ago. i need to be better about planning these things.
#running
I’m realizing that my performance bottleneck is @prologic@twtxt.net ! It is actually calculating the hash to make the replicas, and specifically users with very long feeds 😂 . I’m seriously thinking about enabling replies via configuration.
I’ve seen people refer to themselves as “Quakers” and I thought, “yeah, I liked those games, too!” But apparently, they mean something else. 😬 https://en.wikipedia.org/wiki/Quakers
Look at the size of this coffee!!! 😱 
Cassette: a POSIX application framework featuring a retro-futurist GUI toolkit
Cassette is a GUI application framework written in C11, with a UI inspired by the cassette-futurism aesthetic. Built for modern POSIX systems, it’s made out of three libraries: CGUI, CCFG and COBJ. Cassette is free and open-source software, licensed under the LGPL-3.0. ↫ Cassette GitHub page Upon first reading this description, you might wonder what a “cassette-futurism aesthe … ⌘ Read more
Well, that’s another bug: The search https://twtxt.net/search?q=%22LOOOOL%2C+great+programming+tutorial+music%22 yields the wrong hash. It should have been poyndha instead.
@prologic@twtxt.net hmm this isnt right..
@thecanine@twtxt.net Lol… I just don’t change my default profile pictures. (Well, only when my teammates ask me to.)
@doesnm@doesnm.p.psf.lt Haha, that’s great! :-D
They fixed it. :-D https://www.youtube.com/watch?v=A8b7HFUXPqk
2°C warming limit deemed impossible + 2 more stories
A climate expert declares the 2-degree warming goal unattainable; Israel readies for potential Gaza relocation under Trump’s contentious proposal; an innovative vaccine shows promise in preventing kidney cancer recurrence after surgery. ⌘ Read more
@<url>. Submitting this writes @<domain url> instead of @<nick url> in the feed.
hmm interesting work here.. ill give it a look.. @lyse@lyse.isobeef.org do you know if it is even storing the url into the AST object? afair the code to parse tags url should be the same as the mention url.
One of the things I love about Plan 9 is how the interfaces make this possible: add snapshots to any block storage, at the kernel level, in less than 700 lines of code. http://9p.io/sources/contrib/blstuart/snap/devsnap.c
Suspension Bridge
⌘ Read more
Recovery run: 3.11 miles, 00:10:53 average pace, 00:33:48 duration
@thecanine@twtxt.net Some precious cloud space. Probably the Atlassian one.
How does one end up with an avatar of that weird size to begin with? :-D