Anonymous wrote:

> open:
>
> -fix date for message import
> -fix proper handling of attachments from imported messages
>
> closed:
> -fix catalog link in the footer, catalog is off: i thought i had to change
> the templates, but it is just a switch in the config -fix threading: fixed
> some issues in the import script

> -mime encode subject if needed: thx to
> Retro Guys line

This mod will require php-mbstring. Just mentioning here so you know to list
that if you ever need to list dependencies.

Retro Guy

I've written a script that will pull an article, and convert it to the output I think you need for vichan. I can add this to the nntp server, so you can send a command like:

GROUP rocksolid.shared.test
VICHAN 90

and the article will be sent in a serialized format. This is necessary to transfer the data over tcpip:

$ ./vichan_send.php rocksolid.shared.test 90
O:11:"messageType":2:{s:6:"header";O:10:"headerType":26:{s:6:"number";N;s:2:"id";s:24:"<ts.338.39ms37@anon.com>";s:4:"from";s:15:"poster@anon.com";s:4:"name";s:9:"Anonymous";s:7:"subject";s:3:"yes";s:10:"newsgroups";s:21:"rocksolid.shared.test";s:8:"followup";N;s:4:"date";i:1590584048;s:11:"date_thread";N;s:12:"organization";s:16:"i2pn2 (i2pn.org)";s:10:"xnoarchive";N;s:10:"references";a:1:{i:0;s:59:"<635faa159efe9999f6f2b469d70207e6$1@z5bqfv5v75kxy7pj.onion>";}s:13:"bestreference";N;s:25:"content_transfer_encoding";s:4:"8bit";s:12:"mime_version";N;s:12:"content_type";a:1:{i:0;s:10:"text/plain";}s:20:"content_type_charset";a:1:{i:0;s:5:"utf-8";}s:17:"content_type_name";a:1:{i:0;s:7:"unnamed";}s:21:"content_type_boundary";N;s:19:"content_type_format";a:1:{i:0;s:5:"fixed";}s:7:"answers";N;s:8:"isAnswer";b:0;s:8:"username";s:6:"poster";s:10:"user_agent";N;s:7:"isReply";N;s:10:"threadsize";N;}s:4:"body";a:1:{i:0;s:18:"all still working
";}}

Then, you can '$class_i_want = $unserialize($response);' and you should have your class containing the message info. Is this of any use?

Retro Guy

--
Posted on: Rocksolid Light
www.novabbs.com

I've applied to my test nntp server. You should be able to test here:

cm6lpdymfvvndq3wxpu676m7wd2cjimhy7axdlebdejbv7uk3hcq.b32.i2p

200 Rocksolid Light NNTP Server ready (no posting)
group rocksolid.shared.test
211 170 1 170 rocksolid.shared.test
vichan 90
O:11:"messageType":2:{s:6:"header";O:10:"headerType":26:{s:6:"number";N;s:2:"id";s:24:"<ts.338.39ms37@anon.com>";s:4:"from";s:15:"poster@anon.com";s:4:"name";s:9:"Anonymous";s:7:"subject";s:3:"yes";s:10:"newsgroups";s:21:"rocksolid.shared.test";s:8:"followup";N;s:4:"date";i:1590584048;s:11:"date_thread";N;s:12:"organization";s:16:"i2pn2 (i2pn.org)";s:10:"xnoarchive";N;s:10:"references";a:1:{i:0;s:59:"<635faa159efe9999f6f2b469d70207e6$1@z5bqfv5v75kxy7pj.onion>";}s:13:"bestreference";N;s:25:"content_transfer_encoding";s:4:"8bit";s:12:"mime_version";N;s:12:"content_type";a:1:{i:0;s:10:"text/plain";}s:20:"content_type_charset";a:1:{i:0;s:5:"utf-8";}s:17:"content_type_name";a:1:{i:0;s:7:"unnamed";}s:21:"content_type_boundary";N;s:19:"content_type_format";a:1:{i:0;s:5:"fixed";}s:7:"answers";N;s:8:"isAnswer";b:0;s:8:"username";s:6:"poster";s:10:"user_agent";N;s:7:"isReply";N;s:10:"threadsize";N;}s:4:"body";a:1:{i:0;s:18:"all still working
";}}
quit
205 closing connection - goodbye!

You will need to take that output and run it through 'unserialize($data);'

Right now I don't check to see if the article exists, just testing if this will be useful or helpful for you. If it is, I'll clean up the command and add error checking, etc.

Retro Guy

--
Posted on: Rocksolid Light
www.novabbs.com

>I've written a script that will pull an article, and convert it to the output I think you need for vichan. I can add this to the nntp server, so you can send a command like:

this is more or less what i began to write as well :-).
i believe though it is best to not change the severs answers as such, but to make any necessary conversion for vichan on the clients side after the message has been obtained using the regular commands. this way you could use any news server to pull messages from, not just one that is adapted for the purpose.

Posted on def4

>to make any necessary conversion for vichan on the clients side

and this is all the more true because for vichan to build threads all answers need to have the op as the first reference, which is not respected by def3 and def4 when composing the messages. so some internal logic needs to implemented anyway on the clients side for this to work.

Posted on def4

anon wrote:

>>I've written a script that will pull an article, and convert it to the output I think you need for vichan. I can add this to the nntp server, so you can send a command like:

> this is more or less what i began to write as well :-).
> i believe though it is best to not change the severs answers as such, but to make any necessary conversion for vichan on the clients side after the message has been obtained using the regular commands. this way you could use any news server to pull messages from, not just one that is adapted for the purpose.

Sounds good. I can send you the script if you want it. I'm not currently at a computer where I have access to it but I can send it tomorrow. Just let me know.

Also, I do plan to add features to the rslight nntp server that handles things like file and private message transfer (for admins), and auto-update stuff (if authenticated by pgp), so a vichan command is no problem. But the idea that vichan can use any nntp server is a good idea :)

Retro Guy

--
Posted on: Rocksolid Light
news.novabbs.com

anon wrote:

>>to make any necessary conversion for vichan on the clients side

> and this is all the more true because for vichan to build threads all answers need to have the op as the first reference, which is not respected by def3 and def4 when composing the messages. so some internal logic needs to implemented anyway on the clients side for this to work.

I believe rslight lists the first reference (that it's aware of) in the References: header. I didn't write that code, but I think it works that way.

Retro Guy

--
Posted on: Rocksolid Light
news.novabbs.com

>I believe rslight lists the first reference (that it's aware of) in the References: header. I didn't write that code, but I think it works that way.

yes, rslight does this, but not fudforum and bbs. so all messages need to checked and modified (if necessary), before they can be posted to vichan.
of course you could also change this on fudforum and bbs, but it is more work, and you would have to redo this each time you adopt a new software. better to do this on the side of the software that needs it, i think.

Posted on def4

>open:

>-fix date for message import
>-fix proper handling of attachments from imported messages

the 2nd one turns out to be harder than it looked. only to parse out the textparts from a multipart message still fails too often, partly due to the many different ways clients use to compose the message body i guess. damn. that sure was easy with message_read from newsportal...
found some libs on github for email parsing (which should be basically the same thing), but they are either not functional or with a lot of deps again. damn...
guess i will have to implement this by myself, before i can even think about detecting and decoding attachments.

any thoughts ?

trw

Posted on def4

anon wrote:

>>open:

>>-fix date for message import
>>-fix proper handling of attachments from imported messages

> the 2nd one turns out to be harder than it looked. only to parse out the textparts from a multipart message still fails too often, partly due to the many different ways clients use to compose the message body i guess. damn. that sure was easy with message_read from newsportal...
> found some libs on github for email parsing (which should be basically the same thing), but they are either not functional or with a lot of deps again. damn...
> guess i will have to implement this by myself, before i can even think about detecting and decoding attachments.

> any thoughts ?

I'll send you my script when I get home later.
It's basically message_read separate from newsportal

--
Posted on: Rocksolid Light
news.novabbs.com

Retro Guy wrote:

> anon wrote:

>>>open:

>>>-fix date for message import
>>>-fix proper handling of attachments from imported messages

>> the 2nd one turns out to be harder than it looked. only to parse out the textparts from a multipart message still fails too often, partly due to the many different ways clients use to compose the message body i guess. damn. that sure was easy with message_read from newsportal...
>> found some libs on github for email parsing (which should be basically the same thing), but they are either not functional or with a lot of deps again. damn...
>> guess i will have to implement this by myself, before i can even think about detecting and decoding attachments.

>> any thoughts ?

> I'll send you my script when I get home later.
> It's basically message_read separate from newsportal

It's attached here. It's all the code necessary from rslight to produce the output, but it is not linked to the site, so it can be run independently.

If you want to use it, it needs a little cleaning up as the spool location is hard coded, and probably a couple of other things.

Retro Guy

>It's basically message_read separate from newsportal

you mean i could just include this without all the rest of newsportal (like a standalone function or class) ? that would be wonderful.

cheers

trw

Posted on def4

anon wrote:

>>It's basically message_read separate from newsportal

> you mean i could just include this without all the rest of newsportal (like a standalone function or class) ? that would be wonderful.

That "should" be how it works. I ran it numerous times on my test site, and also ran it quite a few times through the nntp server, and I have not noticed any issues.

Retro Guy

--
Posted on: Rocksolid Light
www.novabbs.com

Retro Guy wrote:

> anon wrote:

>>>It's basically message_read separate from newsportal

>> you mean i could just include this without all the rest of newsportal (like a standalone function or class) ? that would be wonderful.

> That "should" be how it works. I ran it numerous times on my test site, and also ran it quite a few times through the nntp server, and I have not noticed any issues.

I should say I ran it as a standalone script:
$msg = shell_exec("/etc/rslight/scripts/vichan.php ".$command[1]." ".$group);

It's basically 'vichan.php <article_number> <group>' Right now it echos a serialzed output, that can easily be changed back to the class with unserialize(...);

Again, if you do this you'll need to not hardcode the directories, etc.

Retro Guy

--
Posted on: Rocksolid Light
www.novabbs.com

>It's basically 'vichan.php <article_number> <group>' Right now it echos a serialzed output, that can easily be changed back to the class with unserialize(...);

the main thing for me is if i can get the text part of the message and the files nicely separated (bonus points for file detection and decoding of course).
what is the representation of the data when there is an attachment ?
and where can i find the script :-) ?

cheers

trw

Posted on def4

anon wrote:

>>It's basically 'vichan.php <article_number> <group>' Right now it echos a serialzed output, that can easily be changed back to the class with unserialize(...);

> the main thing for me is if i can get the text part of the message and the files nicely separated (bonus points for file detection and decoding of course).
> what is the representation of the data when there is an attachment ?
> and where can i find the script :-) ?

I attached the script to an earlier post, but I just put it on the rslight download site 'vichan.php.txt'.

The output is exactly what message_read would output, just it's serialized so it's one big string. 'unserialze()' will change it back to a class. You should be able to modify it to output whatever format you want.

If I understand correctly, vichan was using the output of message_read, and you should be able to get that same output with this script. You may want another way to introduce the message to the script, but processing should be the same as in rslight.

Retro Guy

--
Posted on: Rocksolid Light
www.novabbs.com

>cm6lpdymfvvndq3wxpu676m7wd2cjimhy7axdlebdejbv7uk3hcq.b32.i2p

>200 Rocksolid Light NNTP Server ready (no posting)
>group rocksolid.shared.test
>211 170 1 170 rocksolid.shared.test
>vichan 90

this gives me syntax error. but i appreciate this:

>help
>100 Sorry, can't help

lol

Posted on def4

anon wrote:

>>cm6lpdymfvvndq3wxpu676m7wd2cjimhy7axdlebdejbv7uk3hcq.b32.i2p

>>200 Rocksolid Light NNTP Server ready (no posting)
>>group rocksolid.shared.test
>>211 170 1 170 rocksolid.shared.test
>>vichan 90

> this gives me syntax error. but i appreciate this:

>>help
>>100 Sorry, can't help

> lol

Oh, I removed the VICHAN command from the nntp server after you mentioned you didn't want to use different commands. I can put it back if you like.

--
Posted on: Rocksolid Light
www.novabbs.com

Retro Guy wrote:

> anon wrote:

>>>cm6lpdymfvvndq3wxpu676m7wd2cjimhy7axdlebdejbv7uk3hcq.b32.i2p

>>>200 Rocksolid Light NNTP Server ready (no posting)
>>>group rocksolid.shared.test
>>>211 170 1 170 rocksolid.shared.test
>>>vichan 90

>> this gives me syntax error. but i appreciate this:

>>>help
>>>100 Sorry, can't help

>> lol

> Oh, I removed the VICHAN command from the nntp server after you mentioned you didn't want to use different commands. I can put it back if you like.

Just added it back if you would like to check it out:

200 Rocksolid Light NNTP Server ready (no posting)
group rocksolid.shared.test
211 176 1 176 rocksolid.shared.test
vichan 90
O:11:"messageType":2:{s:6:"header";O:10:"headerType":26:{s:6:"number";N;s:2:"id";s:24:"<ts.338.39ms37@anon.com>";s:4:"from";s:15:"poster@anon.com";s:4:"name";s:9:"Anonymous";s:7:"subject";s:3:"yes";s:10:"newsgroups";s:21:"rocksolid.shared.test";s:8:"followup";N;s:4:"date";i:1590584048;s:11:"date_thread";N;s:12:"organization";s:16:"i2pn2 (i2pn.org)";s:10:"xnoarchive";N;s:10:"references";a:1:{i:0;s:59:"<635faa159efe9999f6f2b469d70207e6$1@z5bqfv5v75kxy7pj.onion>";}s:13:"bestreference";N;s:25:"content_transfer_encoding";s:4:"8bit";s:12:"mime_version";N;s:12:"content_type";a:1:{i:0;s:10:"text/plain";}s:20:"content_type_charset";a:1:{i:0;s:5:"utf-8";}s:17:"content_type_name";a:1:{i:0;s:7:"unnamed";}s:21:"content_type_boundary";N;s:19:"content_type_format";a:1:{i:0;s:5:"fixed";}s:7:"answers";N;s:8:"isAnswer";b:0;s:8:"username";s:6:"poster";s:10:"user_agent";N;s:7:"isReply";N;s:10:"threadsize";N;}s:4:"body";a:1:{i:0;s:18:"all still working
";}}

Retro Guy

--
Posted on: Rocksolid Light
www.novabbs.com

Retro Guy wrote:

> Retro Guy wrote:

>> anon wrote:

>>>>cm6lpdymfvvndq3wxpu676m7wd2cjimhy7axdlebdejbv7uk3hcq.b32.i2p

>>>>200 Rocksolid Light NNTP Server ready (no posting)
>>>>group rocksolid.shared.test
>>>>211 170 1 170 rocksolid.shared.test
>>>>vichan 90

>>> this gives me syntax error. but i appreciate this:

>>>>help
>>>>100 Sorry, can't help

>>> lol

>> Oh, I removed the VICHAN command from the nntp server after you mentioned you didn't want to use different commands. I can put it back if you like.

> Just added it back if you would like to check it out:

> 200 Rocksolid Light NNTP Server ready (no posting)
> group rocksolid.shared.test
> 211 176 1 176 rocksolid.shared.test
> vichan 90
> O:11:"messageType":2:{s:6:"header";O:10:"headerType":26:{s:6:"number";N;s:2:"id";s:24:"<ts.338.39ms37@anon.com>";s:4:"from";s:15:"poster@anon.com";s:4:"name";s:9:"Anonymous";s:7:"subject";s:3:"yes";s:10:"newsgroups";s:21:"rocksolid.shared.test";s:8:"followup";N;s:4:"date";i:1590584048;s:11:"date_thread";N;s:12:"organization";s:16:"i2pn2 (i2pn.org)";s:10:"xnoarchive";N;s:10:"references";a:1:{i:0;s:59:"<635faa159efe9999f6f2b469d70207e6$1@z5bqfv5v75kxy7pj.onion>";}s:13:"bestreference";N;s:25:"content_transfer_encoding";s:4:"8bit";s:12:"mime_version";N;s:12:"content_type";a:1:{i:0;s:10:"text/plain";}s:20:"content_type_charset";a:1:{i:0;s:5:"utf-8";}s:17:"content_type_name";a:1:{i:0;s:7:"unnamed";}s:21:"content_type_boundary";N;s:19:"content_type_format";a:1:{i:0;s:5:"fixed";}s:7:"answers";N;s:8:"isAnswer";b:0;s:8:"username";s:6:"poster";s:10:"user_agent";N;s:7:"isReply";N;s:10:"threadsize";N;}s:4:"body";a:1:{i:0;s:18:"all still working
> ";}}

> Retro Guy

This calls the vichan.php script I put up for download, and sends out the result in reply to the ViCHAN command. This command is not in the nntp.php I have on the download site. If you want it I'll send this copy also. It will require vichan.php to be in the scripts/ directory.

It's there now as vichan_nntp.php,txt. You'll need to rename it to 'nntp.php'.

Retro Guy

Retro Guy

--
Posted on: Rocksolid Light
www.novabbs.com

actually, the messages are mime encoded so vichan should be able to parse them if they are sent correctly. that would be the best way imo (to let vichan do the parsing).
if i could just read the code that was used to upload.

Posted on def4

i will try that soon. meanwhile, here is what a request to vichans post.php looks like:

POST /post.php HTTP/1.1
Host: xxxxxxxxxxxxxxxx.onion
User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:68.0) Gecko/20100101 Firefox/68.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: http://xxxxxxxxxxxxxxxx.onion/ts/index.html
Content-Type: multipart/form-data; boundary=---------------------------234374660515071969761262837861
Content-Length: 16935
Connection: keep-alive
Upgrade-Insecure-Requests: 1

-----------------------------234374660515071969761262837861
Content-Disposition: form-data; name="nke73c⛻lgoujxyd8⛆4asvr6wf92ih0t"

♦M+<Q4]#(X7J☳FLzh*q,⛻-x8'⚜☯AHac☀IS
-----------------------------234374660515071969761262837861
Content-Disposition: form-data; name="q"

M3Ce⚋tv+&hA `!>1j;q
-----------------------------234374660515071969761262837861
Content-Disposition: form-data; name="board"

ts
-----------------------------234374660515071969761262837861
Content-Disposition: form-data; name="t82o♁kq5970cs3♠dpujmbwvz"

66873
-----------------------------234374660515071969761262837861
Content-Disposition: form-data; name="page"

1
-----------------------------234374660515071969761262837861
Content-Disposition: form-data; name="lastname"

-----------------------------234374660515071969761262837861
Content-Disposition: form-data; name="subject"

testing some
-----------------------------234374660515071969761262837861
Content-Disposition: form-data; name="post"

Post
-----------------------------234374660515071969761262837861
Content-Disposition: form-data; name="url"

♊TK@!mr2=QM4J:).0I65ny,⛳oWa
-----------------------------234374660515071969761262837861
Content-Disposition: form-data; name="body"

here is some text
-----------------------------234374660515071969761262837861
Content-Disposition: form-data; name="text"

2nP.e0mZ_C{,)E?Q>d!1♭b@V|J7<
-----------------------------234374660515071969761262837861
Content-Disposition: form-data; name="file"; filename="woman-e1590571316950-380x214.jpg"
Content-Type: image/jpeg

<FF><D8><FF><E0>^@^PJFIF^@^A^A^A^@H^@H^@^@<FF><DB>^@C^@^H^F^F^G^F^E^H^G^G^G
^L^T^M^L^K^K^L^Y^R^S^O^T^]^Z^_^^^]^Z^\^\ $.' ",#^\^\
| |cut out the rest here
|

-----------------------------234374660515071969761262837861
Content-Disposition: form-data; name="password"

-----------------------------234374660515071969761262837861
Content-Disposition: form-data; name="hash"

9f9927bd6ca11c6b4379c17b9273cab58fadefb3
-----------------------------234374660515071969761262837861--

note all the fields that are disregarded later, just to catch spambots. :-)

if i can rewrite that with php and send it as raw http i think we might be able to use the native post.php of vichan (with some minor adaptions).
but for shortterm, i will try your script soon.

cheers

trw

Posted on def4

i have downloaded, renamed the script and put the other script in /scripts, but
vichan 1
just gives me a blank line.
--
Posted on def2
z5bqfv5v75kxy7pj.onion

anonymous wrote:

> i have downloaded, renamed the script and put the other script in /scripts, but
> vichan 1
> just gives me a blank line.

I'm not at my main computer right now, so I can't look at it. The script, vichan.php, expects to be in /etc/rslight/scripts, and expects articles in /var/spool/rslight/articles

If you run ./vichan.php 90 rocksolid.shared.test, it should output the result.

If not, it's something simple, but I'm not where I can offer more info atm. It works fine on my end (I run it on my test system).

Feel free to try the VICHAN command again on the test server nntp server.

Retro Guy

--
Posted on: Rocksolid Light
news.novabbs.com

>Feel free to try the VICHAN command again on the test server nntp server.

I could reach your server now, and tried it. It works well for articles with text only, for articles with attachments it returns nothing.

See this, for example:

vichan <ts.368.2r2bix@anon.com>

cheers

trw

Posted on def4