Rocksolid Light

Welcome to Rocksolid Light

mail  files  register  newsreader  groups  login

Message-ID:  

As a computer, I find your faith in technology amusing.


devel / comp.compression / Re: ZX0 variant only okay...

SubjectAuthor
* Re: ZX0 variant only okay...Harry Potter
`* Re: ZX0 variant only okay...Harry Potter
 `* Re: ZX0 variant only okay...Harry Potter
  `* Re: ZX0 variant only okay...Harry Potter
   `- Re: ZX0 variant only okay...Harry Potter

1
Re: ZX0 variant only okay...

<049eb618-a577-401f-a95c-7257f4f4fdc2n@googlegroups.com>

  copy mid

https://news.novabbs.org/devel/article-flat.php?id=621&group=comp.compression#621

  copy link   Newsgroups: comp.compression
X-Received: by 2002:a05:6214:9a9:b0:626:15f7:623a with SMTP id du9-20020a05621409a900b0062615f7623amr1736796qvb.0.1687186648759;
Mon, 19 Jun 2023 07:57:28 -0700 (PDT)
X-Received: by 2002:a05:622a:1891:b0:3fd:e156:77bb with SMTP id
v17-20020a05622a189100b003fde15677bbmr2813527qtc.11.1687186648495; Mon, 19
Jun 2023 07:57:28 -0700 (PDT)
Path: i2pn2.org!i2pn.org!usenet.blueworldhosting.com!diablo1.usenet.blueworldhosting.com!peer01.iad!feed-me.highwinds-media.com!news.highwinds-media.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.compression
Date: Mon, 19 Jun 2023 07:57:28 -0700 (PDT)
In-Reply-To: <8069fb9d-60f0-46b4-aa8b-7dcb91002cc1n@googlegroups.com>
Injection-Info: google-groups.googlegroups.com; posting-host=74.108.229.93; posting-account=xRocggoAAACFej4w6sQauoZjUP9yroE5
NNTP-Posting-Host: 74.108.229.93
References: <1760a481-2322-4a88-8785-47ca74ad7371n@googlegroups.com>
<2b8fb9cd-407c-4d4c-a22d-ecfe78c39bf4n@googlegroups.com> <b4f16d7e-314c-481b-86e4-a6637ce7d88dn@googlegroups.com>
<a69cdbbf-4461-4d9d-adb4-fd405654c602n@googlegroups.com> <8069fb9d-60f0-46b4-aa8b-7dcb91002cc1n@googlegroups.com>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <049eb618-a577-401f-a95c-7257f4f4fdc2n@googlegroups.com>
Subject: Re: ZX0 variant only okay...
From: rose.joseph12@yahoo.com (Harry Potter)
Injection-Date: Mon, 19 Jun 2023 14:57:28 +0000
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Received-Bytes: 2126
 by: Harry Potter - Mon, 19 Jun 2023 14:57 UTC

I'm doing much better with this variant now and am almost ready to start optimizing and debugging. But first, I want to buy a few more points with the compression ratio. I have ways to optimize lz77: somebody online told me about hash tables; while processing this information, I thought to use an 8k buffer of bits, where each bit determines whether an associated word was already encountered; somebody else helped me optimize the main loop, and I was testing the current word in the lz77 sliding dictionary before checking for a match. Are there any other ways I can optimize lz77? I am working for 8-bit computers as well as more modern computers, so memory may sometimes be at a premium.

Re: ZX0 variant only okay...

<2b8fb9cd-407c-4d4c-a22d-ecfe78c39bf4n@googlegroups.com>

  copy mid

https://news.novabbs.org/devel/article-flat.php?id=1349&group=comp.compression#1349

  copy link   Newsgroups: comp.compression
X-Received: by 2002:ac8:5e53:0:b0:3e3:8bbd:b367 with SMTP id i19-20020ac85e53000000b003e38bbdb367mr4209499qtx.7.1682874116592;
Sun, 30 Apr 2023 10:01:56 -0700 (PDT)
X-Received: by 2002:a05:622a:1885:b0:3ef:1c64:a9ff with SMTP id
v5-20020a05622a188500b003ef1c64a9ffmr4217495qtc.10.1682874116466; Sun, 30 Apr
2023 10:01:56 -0700 (PDT)
Path: rocksolid2!i2pn.org!usenet.blueworldhosting.com!diablo1.usenet.blueworldhosting.com!peer02.iad!feed-me.highwinds-media.com!news.highwinds-media.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.compression
Date: Sun, 30 Apr 2023 10:01:56 -0700 (PDT)
In-Reply-To: <1760a481-2322-4a88-8785-47ca74ad7371n@googlegroups.com>
Injection-Info: google-groups.googlegroups.com; posting-host=2603:7000:153f:bfe0:2db6:7a20:241d:cc6f;
posting-account=xRocggoAAACFej4w6sQauoZjUP9yroE5
NNTP-Posting-Host: 2603:7000:153f:bfe0:2db6:7a20:241d:cc6f
References: <1760a481-2322-4a88-8785-47ca74ad7371n@googlegroups.com>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <2b8fb9cd-407c-4d4c-a22d-ecfe78c39bf4n@googlegroups.com>
Subject: Re: ZX0 variant only okay...
From: rose.joseph12@yahoo.com (Harry Potter)
Injection-Date: Sun, 30 Apr 2023 17:01:56 +0000
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Received-Bytes: 2035
 by: Harry Potter - Sun, 30 Apr 2023 17:01 UTC

Okay. I added some optimizations to Adaptive Huffman codes that sometimes updates the Huffman code tree when writing an LZ77 or Last16 block or a reference to one of the last two literals (a derivative of my Placement Offset Basic technique--ask for more information). I also have some optimizations to LZ77 that will decrease the size of an LZ77 block if doing so produces better results. Right now, this technique does 2.6% better than Exomizer. My goal is 3% or better by the end of the day. I could apply the rllz technique mentioned in a previous post here. I plan to try it out at a later date but will need a way to identify which bytes are being filled by the rllz blocks. For now, I'm looking for other ways to do better. If you have any suggestions, I'd be happy.

Re: ZX0 variant only okay...

<b4f16d7e-314c-481b-86e4-a6637ce7d88dn@googlegroups.com>

  copy mid

https://news.novabbs.org/devel/article-flat.php?id=1350&group=comp.compression#1350

  copy link   Newsgroups: comp.compression
X-Received: by 2002:a05:620a:4f:b0:74e:324:d6eb with SMTP id t15-20020a05620a004f00b0074e0324d6ebmr1930775qkt.7.1682875262477;
Sun, 30 Apr 2023 10:21:02 -0700 (PDT)
X-Received: by 2002:a05:622a:1006:b0:3f0:a9a0:45e3 with SMTP id
d6-20020a05622a100600b003f0a9a045e3mr4201393qte.12.1682875262317; Sun, 30 Apr
2023 10:21:02 -0700 (PDT)
Path: rocksolid2!i2pn.org!weretis.net!feeder6.news.weretis.net!1.us.feeder.erje.net!feeder.erje.net!usenet.blueworldhosting.com!diablo1.usenet.blueworldhosting.com!peer02.iad!feed-me.highwinds-media.com!news.highwinds-media.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.compression
Date: Sun, 30 Apr 2023 10:21:01 -0700 (PDT)
In-Reply-To: <2b8fb9cd-407c-4d4c-a22d-ecfe78c39bf4n@googlegroups.com>
Injection-Info: google-groups.googlegroups.com; posting-host=2603:7000:153f:bfe0:2db6:7a20:241d:cc6f;
posting-account=xRocggoAAACFej4w6sQauoZjUP9yroE5
NNTP-Posting-Host: 2603:7000:153f:bfe0:2db6:7a20:241d:cc6f
References: <1760a481-2322-4a88-8785-47ca74ad7371n@googlegroups.com> <2b8fb9cd-407c-4d4c-a22d-ecfe78c39bf4n@googlegroups.com>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <b4f16d7e-314c-481b-86e4-a6637ce7d88dn@googlegroups.com>
Subject: Re: ZX0 variant only okay...
From: rose.joseph12@yahoo.com (Harry Potter)
Injection-Date: Sun, 30 Apr 2023 17:21:02 +0000
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Received-Bytes: 1583
 by: Harry Potter - Sun, 30 Apr 2023 17:21 UTC

I'm also not compressing using Adaptive Huffman codes if doing so will result in larger data, am sometimes updating the Huffman tree if repeating one of the last two literals in literal compression and making certain literal compression techniques optional if not being used in a block of data.

Re: ZX0 variant only okay...

<a69cdbbf-4461-4d9d-adb4-fd405654c602n@googlegroups.com>

  copy mid

https://news.novabbs.org/devel/article-flat.php?id=1351&group=comp.compression#1351

  copy link   Newsgroups: comp.compression
X-Received: by 2002:a05:620a:19a7:b0:74f:b51b:3de4 with SMTP id bm39-20020a05620a19a700b0074fb51b3de4mr189579qkb.5.1683284745518;
Fri, 05 May 2023 04:05:45 -0700 (PDT)
X-Received: by 2002:a05:620a:1a90:b0:74d:f7d0:6a57 with SMTP id
bl16-20020a05620a1a9000b0074df7d06a57mr236168qkb.12.1683284745365; Fri, 05
May 2023 04:05:45 -0700 (PDT)
Path: rocksolid2!i2pn.org!news.1d4.us!usenet.blueworldhosting.com!diablo1.usenet.blueworldhosting.com!peer02.iad!feed-me.highwinds-media.com!news.highwinds-media.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.compression
Date: Fri, 5 May 2023 04:05:44 -0700 (PDT)
In-Reply-To: <b4f16d7e-314c-481b-86e4-a6637ce7d88dn@googlegroups.com>
Injection-Info: google-groups.googlegroups.com; posting-host=74.108.229.93; posting-account=xRocggoAAACFej4w6sQauoZjUP9yroE5
NNTP-Posting-Host: 74.108.229.93
References: <1760a481-2322-4a88-8785-47ca74ad7371n@googlegroups.com>
<2b8fb9cd-407c-4d4c-a22d-ecfe78c39bf4n@googlegroups.com> <b4f16d7e-314c-481b-86e4-a6637ce7d88dn@googlegroups.com>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <a69cdbbf-4461-4d9d-adb4-fd405654c602n@googlegroups.com>
Subject: Re: ZX0 variant only okay...
From: rose.joseph12@yahoo.com (Harry Potter)
Injection-Date: Fri, 05 May 2023 11:05:45 +0000
Content-Type: text/plain; charset="UTF-8"
X-Received-Bytes: 1428
 by: Harry Potter - Fri, 5 May 2023 11:05 UTC

I also have a version that uses MTF instead of Adaptive Huffman. Right now, that version is doing way too well to be true. In case it is true, what can I add to MTF to make it better?

Re: ZX0 variant only okay...

<8069fb9d-60f0-46b4-aa8b-7dcb91002cc1n@googlegroups.com>

  copy mid

https://news.novabbs.org/devel/article-flat.php?id=1352&group=comp.compression#1352

  copy link   Newsgroups: comp.compression
X-Received: by 2002:a05:622a:447:b0:3ef:3395:de6a with SMTP id o7-20020a05622a044700b003ef3395de6amr401849qtx.6.1683286958495;
Fri, 05 May 2023 04:42:38 -0700 (PDT)
X-Received: by 2002:a05:620a:3715:b0:74e:2e3:8e24 with SMTP id
de21-20020a05620a371500b0074e02e38e24mr240588qkb.6.1683286958366; Fri, 05 May
2023 04:42:38 -0700 (PDT)
Path: rocksolid2!i2pn.org!usenet.blueworldhosting.com!diablo1.usenet.blueworldhosting.com!peer02.iad!feed-me.highwinds-media.com!news.highwinds-media.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.compression
Date: Fri, 5 May 2023 04:42:38 -0700 (PDT)
In-Reply-To: <a69cdbbf-4461-4d9d-adb4-fd405654c602n@googlegroups.com>
Injection-Info: google-groups.googlegroups.com; posting-host=74.108.229.93; posting-account=xRocggoAAACFej4w6sQauoZjUP9yroE5
NNTP-Posting-Host: 74.108.229.93
References: <1760a481-2322-4a88-8785-47ca74ad7371n@googlegroups.com>
<2b8fb9cd-407c-4d4c-a22d-ecfe78c39bf4n@googlegroups.com> <b4f16d7e-314c-481b-86e4-a6637ce7d88dn@googlegroups.com>
<a69cdbbf-4461-4d9d-adb4-fd405654c602n@googlegroups.com>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <8069fb9d-60f0-46b4-aa8b-7dcb91002cc1n@googlegroups.com>
Subject: Re: ZX0 variant only okay...
From: rose.joseph12@yahoo.com (Harry Potter)
Injection-Date: Fri, 05 May 2023 11:42:38 +0000
Content-Type: text/plain; charset="UTF-8"
X-Received-Bytes: 1458
 by: Harry Potter - Fri, 5 May 2023 11:42 UTC

I was wrong, and I'm sorry. :'( I was exiting a loop at the first byte in a block compressed. I am still doing better but nearly as much better than I thought.

1
server_pubkey.txt

rocksolid light 0.9.81
clearnet tor