Hi,

I have an idea.

In addition to cancelling articles with perl-nocem, I'd like to repost
them to a private, unfeeded group, so they still could be accessed by
local users who would want to dig there. Of course some headers (like
Message-ID or Newsgroups) would have to be rewritten, but it seems
possible.

It should be possible to modify perl-nocem program to do this, but I don't
know Perl well enough to do it...

Has anyone done this before and has a ready-made solution?

Adam W. wrote:

> In addition to cancelling articles with perl-nocem, I'd like to repost
> them to a private, unfeeded group, so they still could be accessed by
> local users who would want to dig there. Of course some headers (like
> Message-ID or Newsgroups) would have to be rewritten, but it seems
> possible.

Rewriting Message-ID is always a great idea.

Thomas Hochstein <thh@thh.name> wrote:
>Adam W. wrote:

>>In addition to cancelling articles with perl-nocem, I'd like to repost
>>them to a private, unfeeded group, so they still could be accessed by
>>local users who would want to dig there. Of course some headers (like
>>Message-ID or Newsgroups) would have to be rewritten, but it seems
>>possible.

>Rewriting Message-ID is always a great idea.

I'd say it's absolutely required to avoid screwing things up hugely, or
the original article should be encapsulated as an attachment, and THAT
should be posted instead to Adam W.'s local group with spam evidence in
it. Is Adam W. actually planning on eyeballing these articles for false
positives caught by SpamAssasin?

On Sat, 14 Oct 2023 14:47:15 +0200
Thomas Hochstein <thh@thh.name> wrote:

> Adam W. wrote:
>
> > In addition to cancelling articles with perl-nocem, I'd like to
> > repost them to a private, unfeeded group, so they still could be
> > accessed by local users who would want to dig there. Of course some
> > headers (like Message-ID or Newsgroups) would have to be rewritten,
> > but it seems possible.
>
> Rewriting Message-ID is always a great idea.

One may add a prefix or suffix to the existing Message-ID when doing
this, which preserves the original Message-ID.

If the Message-ID is: digital-salami-slicer@spam.host,

Change to: spam-0000000000001-digital-salami-slicer@spam.host.

One could rename the original Message-ID header to Original-Message-ID
or something like that.

--
3883@sugar.bug | sybershock.com | alt.sources.crypto

Adam H. Kerman <ahk@chinet.com> wrote:

> Is Adam W. actually planning on eyeballing these articles for false
> positives caught by SpamAssasin?

Yes, if it's not too much work. I'd also like to give users the
opportunity to view these posts. But I abandoned the idea for now.

I noticed one false positive on one group I read (I came across the reply
to this msgid, but the post was not found on my server, and I found it in
NoCeM logs):

Message-ID: <bcc98944-4e92-45b2-aef2-38998ef41dfan@googlegroups.com>

This was posted by a known troll, but it wasn't part of this spam attack.

Ray, is it possible to check what triggered this hit? It was found in this
notice:

Message-ID: <20231017131504$9237@news.eternal-september.org>

But, what's interesting, it was also in i2pn2 notice (I don't process
these... yet):

Message-ID: <bot-spam-c4719d9951807e31@i2pn2.org>

Adam W. wrote:

> Adam H. Kerman <ahk@chinet.com> wrote:

>> Is Adam W. actually planning on eyeballing these articles for false
>> positives caught by SpamAssasin?

> Yes, if it's not too much work. I'd also like to give users the
> opportunity to view these posts. But I abandoned the idea for now.

> I noticed one false positive on one group I read (I came across the reply
> to this msgid, but the post was not found on my server, and I found it in
> NoCeM logs):

> Message-ID: <bcc98944-4e92-45b2-aef2-38998ef41dfan@googlegroups.com>

> This was posted by a known troll, but it wasn't part of this spam attack.

> Ray, is it possible to check what triggered this hit? It was found in this
> notice:

> Message-ID: <20231017131504$9237@news.eternal-september.org>

> But, what's interesting, it was also in i2pn2 notice (I don't process
> these... yet):

> Message-ID: <bot-spam-c4719d9951807e31@i2pn2.org>

As for i2pn2 notice, it's automated to add specific users to a rule (to be
filtered) based on previous spamassassin scores. This article appears to have
made it into that list previously, so was automatically filtered.

i2pn2 doesn't really differentiate between "this spam attack" and stuff to filter,
so a user can get caught in the filter after several (what appear to be junk
to spamassassin) posts.

--
Retro Guy

Thus spake gof-cut-this-news@cut-this-chmurka.net.invalid (Adam W.)

> Message-ID: <bcc98944-4e92-45b2-aef2-38998ef41dfan@googlegroups.com>
> This was posted by a known troll, but it wasn't part of this spam attack.
> Ray, is it possible to check what triggered this hit? It was found in this
> notice:
> Message-ID: <20231017131504$9237@news.eternal-september.org>
> But, what's interesting, it was also in i2pn2 notice (I don't process
> these... yet):

No specific rule was triggered, just the assessment of the Bayes filter:

X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
news.eternal-september.org
X-Spam-Flag: YES
X-Spam-Level: **********
X-Spam-Status: Yes, score=10.5 required=10.0 tests=BAYES_99,CONTENT_QP
autolearn=no autolearn_force=no version=3.4.6
X-Spam-Report:
* 10 BAYES_99 BODY: Bayes spam probability is 99 to 100%
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* [score: 0.9913]
X-Spam-Languages:
Message-ID: <bcc98944-4e92-45b2-aef2-38998ef41dfan@googlegroups.com>

Maybe I need to feed it more ham.

FWIW: The overall percentage of false positives here is below 0.1%

--
Пу́тін — хуйло́
http://www.eternal-september.org

Le 17/10/2023 à 17:42, gof-cut-this-news@cut-this-chmurka.net.invalid
(Adam W.) a écrit :

> Adam H. Kerman <ahk@chinet.com> wrote:

>> Is Adam W. actually planning on eyeballing these articles for false
>> positives caught by SpamAssasin?
> Yes, if it's not too much work. I'd also like to give users the
> opportunity to view these posts. But I abandoned the idea for now.

Alphanet did this, it worked fine, there was a configuration page and you
could check or uncheck what you wanted to see. Unfortunately it's lost.

Eric M wrote:

> Le 17/10/2023 à 17:42, gof-cut-this-news@cut-this-chmurka.net.invalid
> (Adam W.) a écrit :

>> Adam H. Kerman <ahk@chinet.com> wrote:

>>> Is Adam W. actually planning on eyeballing these articles for false
>>> positives caught by SpamAssasin?

>> Yes, if it's not too much work. I'd also like to give users the
>> opportunity to view these posts. But I abandoned the idea for now.

> Alphanet did this, it worked fine, there was a configuration page and you
> could check or uncheck what you wanted to see. Unfortunately it's lost.

What I do (at i2pn2.org) is have my spamassassin script dump all filtered
messages to a mbox file. Then I can review that file in mutt daily for
false positives (which I very rarely find).

It still is tens of thousands of messages per day, but almost all of them
are obvious spam just from the subject line. Scrolling through them in
mutt is relatively fast.

If I do find a false positive, I can review the spamassassin headers to
see what may have triggered it, and modify from there.

--
Retro Guy

In article <ugma0r$6af$1$arnold@news.chmurka.net>,
Adam W. <gof-cut-this-news@cut-this-chmurka.net.invalid> wrote:
>Adam H. Kerman <ahk@chinet.com> wrote:
>
>> Is Adam W. actually planning on eyeballing these articles for false
>> positives caught by SpamAssasin?
>
>Yes, if it's not too much work. I'd also like to give users the
>opportunity to view these posts. But I abandoned the idea for now.
>
>I noticed one false positive on one group I read (I came across the reply
>to this msgid, but the post was not found on my server, and I found it in
>NoCeM logs):
>
>Message-ID: <bcc98944-4e92-45b2-aef2-38998ef41dfan@googlegroups.com>
>
>This was posted by a known troll, but it wasn't part of this spam attack.
>
>Ray, is it possible to check what triggered this hit? It was found in this
>notice:
>
>Message-ID: <20231017131504$9237@news.eternal-september.org>
>
>But, what's interesting, it was also in i2pn2 notice (I don't process
>these... yet):
>
>Message-ID: <bot-spam-c4719d9951807e31@i2pn2.org>

Blockking Google seems to be the only way to go!
--
Member - Liberal International This is doctor@nk.ca Ici doctor@nk.ca
Yahweh, King & country!Never Satan President Republic!Beware AntiChrist rising!
Look at Psalms 14 and 53 on Atheism ; unsubscribe from Google Groups to be seen
Confusion gives protections to evil while ignoring the innocent. -unknown Beware https://mindspring.com

The Doctor wrote:

> In article <ugma0r$6af$1$arnold@news.chmurka.net>,
> Adam W. <gof-cut-this-news@cut-this-chmurka.net.invalid> wrote:
>>Adam H. Kerman <ahk@chinet.com> wrote:
>>
>>> Is Adam W. actually planning on eyeballing these articles for false
>>> positives caught by SpamAssasin?
>>
>>Yes, if it's not too much work. I'd also like to give users the
>>opportunity to view these posts. But I abandoned the idea for now.
>>
>>I noticed one false positive on one group I read (I came across the reply
>>to this msgid, but the post was not found on my server, and I found it in
>>NoCeM logs):
>>
>>Message-ID: <bcc98944-4e92-45b2-aef2-38998ef41dfan@googlegroups.com>
>>
>>This was posted by a known troll, but it wasn't part of this spam attack.
>>
>>Ray, is it possible to check what triggered this hit? It was found in this
>>notice:
>>
>>Message-ID: <20231017131504$9237@news.eternal-september.org>
>>
>>But, what's interesting, it was also in i2pn2 notice (I don't process
>>these... yet):
>>
>>Message-ID: <bot-spam-c4719d9951807e31@i2pn2.org>

> Blockking Google seems to be the only way to go!

Do you think that one (sort of) false positive is worse than all the
false positives you'd get blocking all GG posts at the server level?

If you want to do that in a client, do whatever you want, it's your client,
but a server has somewhat of a responsibility to at least try to serve
legitimate messages.

--
Retro Guy

retro.guy@rocksolidbbs.com (Retro Guy) writes:
> Do you think that one (sort of) false positive is worse than all the
> false positives you'd get blocking all GG posts at the server level?
>
> If you want to do that in a client, do whatever you want, it's your client,
> but a server has somewhat of a responsibility to at least try to serve
> legitimate messages.

The server's only responsibility is to serve the messages its users want
to see. I'm the only user of my server, and I'm not interested in seeing
anything originating in Google Groups, so I just use cleanfeed to drop
all that traffic. It's really cleaned things up.

The beauty of Usenet is that if you really want those messages from
Google Groups, you can add a peer that carries them -- my decision to
shitcan all that garbage doesn't stop anyone else from retrieving it if
they so desire, they just won't retrieve it from my server.

john

Retro Guy <retro.guy@rocksolidbbs.com> wrote:

>> Blockking Google seems to be the only way to go!
>
> Do you think that one (sort of) false positive is worse than all the
> false positives you'd get blocking all GG posts at the server level?
>
> If you want to do that in a client, do whatever you want, it's your client,
> but a server has somewhat of a responsibility to at least try to serve
> legitimate messages.

Yes, precisely that's why I don't block Google. Legitimate posters still
for some reasons use it and I want to have a complete feed (minus obvious
spam), so my users don't wonder why they can't see some posts.

Retro Guy <retro.guy@rocksolidbbs.com> wrote:
>The Doctor wrote:
>>Adam W. <gof-cut-this-news@cut-this-chmurka.net.invalid> wrote:
>>>Adam H. Kerman <ahk@chinet.com> wrote:

>>>>Is Adam W. actually planning on eyeballing these articles for false
>>>>positives caught by SpamAssasin?

>>>Yes, if it's not too much work. I'd also like to give users the
>>>opportunity to view these posts. But I abandoned the idea for now.

>>>I noticed one false positive on one group I read (I came across the reply
>>>to this msgid, but the post was not found on my server, and I found it in
>>>NoCeM logs):

>>>Message-ID: <bcc98944-4e92-45b2-aef2-38998ef41dfan@googlegroups.com>

>>>This was posted by a known troll, but it wasn't part of this spam attack.

>>>Ray, is it possible to check what triggered this hit? It was found in this
>>>notice:

>>>Message-ID: <20231017131504$9237@news.eternal-september.org>

>>>But, what's interesting, it was also in i2pn2 notice (I don't process
>>>these... yet):

>>>Message-ID: <bot-spam-c4719d9951807e31@i2pn2.org>

>>Blockking Google seems to be the only way to go!

>Do you think that one (sort of) false positive is worse than all the
>false positives you'd get blocking all GG posts at the server level?

>If you want to do that in a client, do whatever you want, it's your client,
>but a server has somewhat of a responsibility to at least try to serve
>legitimate messages.

Please don't make me agree with Yads. That's cruel.

The server Yads runs serves one user: himself.

Even if Yads served dozens or hundreds of users, he can peer with
whomever he chooses to and can junk all articles from a particular
server. His server, his rules. As long as Yads isn't taking actions that
harm the network, no one other than Yads gets to object to how he wishes
to present Usenet to his own users.

On Tue, 17 Oct 2023 22:21:50 +0000, retro.guy@rocksolidbbs.com (Retro Guy) wrote:
>The Doctor wrote:
>>
snip
>
>> Blockking Google seems to be the only way to go!
>
>Do you think that one (sort of) false positive is worse than all the
>false positives you'd get blocking all GG posts at the server level?
>If you want to do that in a client, do whatever you want, it's your client,
>but a server has somewhat of a responsibility to at least try to serve
>legitimate messages.

the universal invasion by the shameless googlegroups admins has absolutely
eliminated any possibility of "legitimate messages" originating from their
diabolical weapon of mass destruction . . . udp would only be just deserts

In article <ugn2gn$vjk$1$arnold@news.chmurka.net>,
Adam W. <gof-cut-this-news@cut-this-chmurka.net.invalid> wrote:
>Retro Guy <retro.guy@rocksolidbbs.com> wrote:
>
>>> Blockking Google seems to be the only way to go!
>>
>> Do you think that one (sort of) false positive is worse than all the
>> false positives you'd get blocking all GG posts at the server level?
>>
>> If you want to do that in a client, do whatever you want, it's your client,
>> but a server has somewhat of a responsibility to at least try to serve
>> legitimate messages.
>
>Yes, precisely that's why I don't block Google. Legitimate posters still
>for some reasons use it and I want to have a complete feed (minus obvious
>spam), so my users don't wonder why they can't see some posts.

GG : high noise low signal.
--
Member - Liberal International This is doctor@nk.ca Ici doctor@nk.ca
Yahweh, King & country!Never Satan President Republic!Beware AntiChrist rising!
Look at Psalms 14 and 53 on Atheism ; unsubscribe from Google Groups to be seen
Confusion gives protections to evil while ignoring the innocent. -unknown Beware https://mindspring.com

In article <ugn4l1$37ntc$1@dont-email.me>,
Adam H. Kerman <ahk@chinet.com> wrote:
>Retro Guy <retro.guy@rocksolidbbs.com> wrote:
>>The Doctor wrote:
>>>Adam W. <gof-cut-this-news@cut-this-chmurka.net.invalid> wrote:
>>>>Adam H. Kerman <ahk@chinet.com> wrote:
>
>>>>>Is Adam W. actually planning on eyeballing these articles for false
>>>>>positives caught by SpamAssasin?
>
>>>>Yes, if it's not too much work. I'd also like to give users the
>>>>opportunity to view these posts. But I abandoned the idea for now.
>
>>>>I noticed one false positive on one group I read (I came across the reply
>>>>to this msgid, but the post was not found on my server, and I found it in
>>>>NoCeM logs):
>
>>>>Message-ID: <bcc98944-4e92-45b2-aef2-38998ef41dfan@googlegroups.com>
>
>>>>This was posted by a known troll, but it wasn't part of this spam attack.
>
>>>>Ray, is it possible to check what triggered this hit? It was found in this
>>>>notice:
>
>>>>Message-ID: <20231017131504$9237@news.eternal-september.org>
>
>>>>But, what's interesting, it was also in i2pn2 notice (I don't process
>>>>these... yet):
>
>>>>Message-ID: <bot-spam-c4719d9951807e31@i2pn2.org>
>
>>>Blockking Google seems to be the only way to go!
>
>>Do you think that one (sort of) false positive is worse than all the
>>false positives you'd get blocking all GG posts at the server level?
>
>>If you want to do that in a client, do whatever you want, it's your client,
>>but a server has somewhat of a responsibility to at least try to serve
>>legitimate messages.
>
>Please don't make me agree with Yads. That's cruel.
>
>The server Yads runs serves one user: himself.
>
>Even if Yads served dozens or hundreds of users, he can peer with
>whomever he chooses to and can junk all articles from a particular
>server. His server, his rules. As long as Yads isn't taking actions that
>harm the network, no one other than Yads gets to object to how he wishes
>to present Usenet to his own users.

Of course I did get a hoorah from a client on this action.
--
Member - Liberal International This is doctor@nk.ca Ici doctor@nk.ca
Yahweh, King & country!Never Satan President Republic!Beware AntiChrist rising!
Look at Psalms 14 and 53 on Atheism ; unsubscribe from Google Groups to be seen
Confusion gives protections to evil while ignoring the innocent. -unknown Beware https://mindspring.com

-------- Original Message --------
From: Ray Banana <rayban@raybanana.net>
Date: mar, ott 17 2023 04:33:48PM GMT+00:00
Subject: NoCeM and reposting cancelled articles to another group
>
> Maybe I need to feed it more ham.
>
> FWIW: The overall percentage of false positives here is below 0.1%
>

Hi Ray, RetroGuy,

please check my "junk" group on my server, we have a loot of flood and
spam. Maybe you can set better your filters.

Sincerely

--
Ivo Gandolfo

Ivo Gandolfo wrote:

> -------- Original Message --------
> From: Ray Banana <rayban@raybanana.net>
> Date: mar, ott 17 2023 04:33:48PM GMT+00:00
> Subject: NoCeM and reposting cancelled articles to another group
>>
>> Maybe I need to feed it more ham.
>>
>> FWIW: The overall percentage of false positives here is below 0.1%
>>

> Hi Ray, RetroGuy,

> please check my "junk" group on my server, we have a loot of flood and
> spam. Maybe you can set better your filters.

Hi Ivo, I just took a look. Since these are all in de. and I cannot read
German, I'm not sure what is spam about these messages and entire threads.

I translated a few messages and still don't see what is wrong. Can you
make any suggestions or would someone like to take on de. by sending
specific articles and reasons that they are spam or flood?

Also, did you (Ivo) receive my email yesterday?

--
Retro Guy

Retro Guy <retro.guy@rocksolidbbs.com> wrote:

> As for i2pn2 notice, it's automated to add specific users to a rule (to be
> filtered) based on previous spamassassin scores. This article appears to have
> made it into that list previously, so was automatically filtered.
>
> i2pn2 doesn't really differentiate between "this spam attack" and stuff to filter,
> so a user can get caught in the filter after several (what appear to be junk
> to spamassassin) posts.

So, to clarify: once a user gets caught, all his future posts will be
filtered?

Ray Banana <rayban@raybanana.net> wrote:

> No specific rule was triggered, just the assessment of the Bayes filter:
[...]
> * 10 BAYES_99 BODY: Bayes spam probability is 99 to 100%
[...]
> Maybe I need to feed it more ham.

Makes sense... the post contained only a short text and a link, so it
might have looked like spam.

Adam W. wrote:

> Retro Guy <retro.guy@rocksolidbbs.com> wrote:

>> As for i2pn2 notice, it's automated to add specific users to a rule (to be
>> filtered) based on previous spamassassin scores. This article appears to have
>> made it into that list previously, so was automatically filtered.
>>
>> i2pn2 doesn't really differentiate between "this spam attack" and stuff to filter,
>> so a user can get caught in the filter after several (what appear to be junk
>> to spamassassin) posts.

> So, to clarify: once a user gets caught, all his future posts will be
> filtered?

No, not specifically. I use a higher score to permanently list a user as filtered.

For example, if 5.0 is the trigger for spam, you need to actually hit 10.0 to get
permanently filtered. That usually works fine, but it is possible for someone to
trigger this by maybe replying to a very spammy message (and leaving the content in).

I can easily remove someone from this permanent block, which I did for the user
above. I just need to be aware of it, so I'm glad it was mentioned here.

--
Retro Guy

D <J@m> wrote:

> the universal invasion by the shameless googlegroups admins has absolutely
> eliminated any possibility of "legitimate messages" originating from their
> diabolical weapon of mass destruction . . . udp would only be just deserts

This is wishful thinking.

I agree that it would be best if Usenet was separated from Google Groups,
but the current reality is that normal, valid users still post from
Google, at least to pl.* groups.

Maybe they couldn't find a server to use, maybe they didn't want to
register on mine (but I don't want to monopolize Usenet in pl.*, so it's
better if users use different services), maybe they prefer the web
interface over a dedicated reader. I don't know what their reasons are.

I make daily statistics of pl.* and alt.pl.* activity, counting posts from
different servers and grouping them by From (to count unique users, not
individual posts) and the picture is clear: most users post using Google
Groups and my server.

Statistics are here (look for "Serwery"):
http://news.chmurka.net/stats.php

They're also available in JSON format:

http://news.chmurka.net/stats.php?all_json

Of course it also counts spam, but even after removing the spamming Froms,
the picture is similar.

My priority is to provide a complete feed to my users, not to fight Google
(at the expense of my users). They probably wouldn't want to use my server
if I launched an anti-Google campaign at their expense (and I couldn't
blame them for it). I'll happily remove obvious spam and floods, but I
don't want to remove valid posters in the process.

On Wed, 18 Oct 2023 14:44:07 -0000 (UTC), gof-cut-this-news@cut-this-chmurka.net.invalid (Adam W.) wrote:
>D <J@m> wrote:
>> the universal invasion by the shameless googlegroups admins has absolutely
>> eliminated any possibility of "legitimate messages" originating from their
>> diabolical weapon of mass destruction . . . udp would only be just deserts
>
>This is wishful thinking.
>I agree that it would be best if Usenet was separated from Google Groups,
>but the current reality is that normal, valid users still post from
>Google, at least to pl.* groups.
>Maybe they couldn't find a server to use, maybe they didn't want to
>register on mine (but I don't want to monopolize Usenet in pl.*, so it's
>better if users use different services), maybe they prefer the web
>interface over a dedicated reader. I don't know what their reasons are.
>I make daily statistics of pl.* and alt.pl.* activity, counting posts from
>different servers and grouping them by From (to count unique users, not
>individual posts) and the picture is clear: most users post using Google
>Groups and my server.
>Statistics are here (look for "Serwery"):
>http://news.chmurka.net/stats.php
>They're also available in JSON format:
>http://news.chmurka.net/stats.php?all_json
>Of course it also counts spam, but even after removing the spamming Froms,
>the picture is similar.
>My priority is to provide a complete feed to my users, not to fight Google
>(at the expense of my users). They probably wouldn't want to use my server
>if I launched an anti-Google campaign at their expense (and I couldn't
>blame them for it). I'll happily remove obvious spam and floods, but I
>don't want to remove valid posters in the process.

when in rome . . .

Retro Guy <retro.guy@rocksolidbbs.com> wrote:

> No, not specifically. I use a higher score to permanently list a user as filtered.
>
> For example, if 5.0 is the trigger for spam, you need to actually hit 10.0 to get
> permanently filtered. That usually works fine, but it is possible for someone to
> trigger this by maybe replying to a very spammy message (and leaving the content in).
>
> I can easily remove someone from this permanent block, which I did for the user
> above. I just need to be aware of it, so I'm glad it was mentioned here.

Now it's clear, thanks :)