The forum robots

[Hourai]

I'm talking about these fuckers.

Are they internet scrapers, or something? If not, what are they?

 

[ciro64]

Web crawlers, I guess. Haven't touched these in the configuration yet, tbh.

We need those if we want to be discovered in search engines. Although this also probably come with the downside of having content there used for AI training (although disabling discoverability in search engines doesn't really stop this)

How do ya'll feel about this forum popping up on Google and other places, btw?

 

[Ranmyaku Ichi]

I looked this up, and it looks like it's a web crawler bot that collects info for Bing's search engine.

 

[Yousef]

I'm talking about these fuckers.
View attachment 2314
Are they internet scrapers, or something? If not, what are they?

Weird. These never popped up when I’m moderating RGT. I wonder what they really are? I think there’s a way to pick them up with a spam filter but it’s not always reliable.

 

[GardeniaSky]

I like to remain hidden. However, I know we're trying to grow.

 

[Hourai]

How do ya'll feel about this forum popping up on Google and other places, btw?

I'd be surprised if it showed up outside of extremely specific search results, given SEO and the lack of something that specific that draws people to it (like CDR did, and RGT probably does).

I think that works, though. I like the size of this place, but I do also think it would benefit from more people.

 

[Ranmyaku Ichi]

We'll be the 87th result on Bing any day now!

 

[Yousef]

How do ya'll feel about this forum popping up on Google and other places, btw?

I’d be concerned for anyone unfortunate enough to stumble upon my writing rambles.

 

[SylverReZ]

Web crawlers, I guess. Haven't touched these in the configuration yet, tbh.

We need those if we want to be discovered in search engines. Although this also probably come with the downside of having content there used for AI training (although disabling discoverability in search engines doesn't really stop this)

How do ya'll feel about this forum popping up on Google and other places, btw?

Cloudflare has robots settings built-in so that you can block AI scrapers/crawlers and such, I highly recommend you do configure it if you can, so you wouldn't end up with massive server slowdowns. https://www.theregister.com/2025/08/21/ai_crawler_traffic/

 

[Yousef]

Cloudflare has robots settings built-in so that you can block AI scrapers/crawlers and such, I highly recommend you do configure it if you can, so you wouldn't end up with massive server slowdowns. https://www.theregister.com/2025/08/21/ai_crawler_traffic/

Cool stuff, will probably share with the lads if this ever becomes a huge deal in our forum.

 

[RiskyAsh]

Weird. These never popped up when I’m moderating RGT. I wonder what they really are? I think there’s a way to pick them up with a spam filter but it’s not always reliable.

(Yousef hon they absolutely pop up on RGT as well, it's in the Current Visitors section of the Members tab.)

 

[Nebulon_Galaxy]

I like to remain hidden. However, I know we're trying to grow.

weed?

 

[MasterEn]

I'm talking about these fuckers.
View attachment 2314
Are they internet scrapers, or something? If not, what are they?

most likely. there are a variety of robots out there, collecting various kinds of data. i saw on rgt that there were about 4,000 of them browsing around. a lot of them being google, amazon, bing and others.
i usually see that one or two check this place out every once in a while.

 

[Hourai]

We'll be the 87th result on Bing any day now!

I joined a website because I saw it on the 32nd or so page of search results, way back when. Anything's possible!

 

[GardeniaSky]

weed?

IM CALLING THE COPS

 

[Yousef]

(Yousef hon they absolutely pop up on RGT as well, it's in the Current Visitors section of the Members tab.)

View attachment 2342

What the hell I never seen this before.
Well, I’ll be a bastard and embarrass you too by reminding you it took you months to find the mod panel.
Edit: BLIMEY crapbiscuits, i just tried it and that scared the shit out of me. So is this the robot invasion everyone been talking about?
Edit 2: so they’re visible to everyone I guess. Push Square forums (where I spent the last 15 years) don’t really run on XenForo so this is my first time seeing these guys. I dunno if that’s a blessing or a curse.

 

[RiskyAsh]

What the hell I never seen this before.
Well, I’ll be a bastard and embarrass you too by reminding you it took you months to find the mod panel.
Edit: BLIMEY crapbiscuits, i just tried it and that scared the shit out of me. So is this the robot invasion everyone been talking about?
Edit 2: so they’re visible to everyone I guess. Push Square forums (where I spent the last 15 years) don’t really run on XenForo so this is my first time seeing these guys. I dunno if that’s a blessing or a curse.

How dare you, Spike literally just hadn't enabled it for me and Waffles, he didn't know it was a thing until you came on!

 

[Yousef]

How dare you, Spike literally just hadn't enabled it for me and Waffles, he didn't know it was a thing until you came on!

Sorry, could you say that any louder? I can’t hear you with all these EXCUSES.

Alrighty, that’s enough. I can’t keep this bit. You caught me at a bad evening, I was honestly venting off this one bastard I faced in Lich cus Chess.com has really bad players, but how the hell was I supposed to see that fork coming? And that bishops discovered attack…BLIMEY! If it helps you feel any better, you may go ahead and subject to one bad comic. I’ll read it no matter what it is. (Actually, I was just gonna suggest violence, but you’d probably try to murder me. I just have a bastard’s aura).