Rocksolid Light

Welcome to Rocksolid Light

mail  files  register  newsreader  groups  login

Message-ID:  

19 May, 2024: Line wrapping has been changed to be more consistent with Usenet standards.
 If you find that it is broken please let me know here rocksolid.nodes.help

devel / comp.lang.python / Question about garbage collection

SubjectAuthor
* Question about garbage collectionFrank Millman
`* Re: Question about garbage collectionPaul Rubin
 `- Re: Question about garbage collection (Posting On Python-List Prohibited)Lawrence D'Oliveiro

1
Question about garbage collection

<mailman.62.1705327022.15798.python-list@python.org>

  copy mid

https://news.novabbs.org/devel/article-flat.php?id=25111&group=comp.lang.python#25111

  copy link   Newsgroups: comp.lang.python
Path: i2pn2.org!i2pn.org!weretis.net!feeder8.news.weretis.net!fu-berlin.de!uni-berlin.de!not-for-mail
From: frank@chagford.com (Frank Millman)
Newsgroups: comp.lang.python
Subject: Question about garbage collection
Date: Mon, 15 Jan 2024 15:51:26 +0200
Lines: 69
Message-ID: <mailman.62.1705327022.15798.python-list@python.org>
References: <35f5aba8-047e-477f-b1d5-6892d77aa9d8@chagford.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Trace: news.uni-berlin.de mYg6CU4c4BBGM7ArjEfqzwNu5j1YWXLzfcwvcqjHlX7A==
Cancel-Lock: sha1:AHCvjFqBmoYoCykGrSxze/gwlss= sha256:tyjjMLvlM1z9dbV3JbP2UvT/Jo9HCn0lGixig0pq8Q4=
Return-Path: <frank@chagford.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=pass
reason="2048-bit key; unprotected key"
header.d=chagford.com header.i=@chagford.com header.b=G5fXUVz3;
dkim-adsp=pass; dkim-atps=neutral
X-Spam-Status: OK 0.006
X-Spam-Evidence: '*H*': 0.99; '*S*': 0.00; 'comments': 0.03; 'def':
0.04; 'parent': 0.07; 'child': 0.09; 'included,': 0.09;
'objects,': 0.09; 'import': 0.15; 'that.': 0.15;
'from:addr:chagford.com': 0.16; 'from:addr:frank': 0.16;
'from:name:frank millman': 0.16; 'long-running': 0.16; 'message-
id:@chagford.com': 0.16; 'millman': 0.16; 'received:196.35': 0.16;
'received:196.35.198': 0.16; 'received:197.90': 0.16;
'received:197.90.32': 0.16; 'received:197.90.32.26': 0.16;
'received:synaq.com': 0.16; 'self.name': 0.16; 'server,': 0.16;
'something?': 0.16; 'python': 0.16; 'subject:Question': 0.19;
'to:addr:python-list': 0.20; 'skip:_ 10': 0.22; 'lines': 0.23;
'object': 0.26; 'header:User-Agent:1': 0.30; 'program': 0.31;
'objects': 0.32; 'skip:= 50': 0.32; 'but': 0.32; 'appreciated.':
0.34; 'missing': 0.37; 'class': 0.37; 'this.': 0.37; 'read': 0.38;
'enough': 0.39; 'break': 0.39; 'want': 0.40; 'should': 0.40;
'reference': 0.60; 'skip:\xc2 10': 0.62; 'identify': 0.64;
'experience': 0.64; 'closing': 0.69; 'received:196': 0.69;
'stores': 0.69; 'below': 0.69; 'del': 0.70; '8bit%:100': 0.76;
'garbage': 0.84; 'itself.': 0.84; 'stuff,': 0.84; 'alive.': 0.93;
'worry': 0.95
ARC-Seal: i=1; cv=none; a=rsa-sha256; d=synaq.com; s=securemail; t=1705327019;
b=1RSKXQbT/hiSM0LwKSSh15k7dxuojqB4jN/4YDfw2VoFuWzDl5n0WCOJzj1O18+OSFRQ3+XDyN
s+zrRB7XfNvAtc5ZzqcxtnlobsLNjMBxvKgaRqRwur+T6If8Dwdk5hXKa3fvREOA2CJh/D8Pe+
jYr7uAvd8dzSFFiHHRvVlReS3x1gMd3aO0PlcDUI39p4rh46/xu1RlNPCM7E2YOMYBnt5pUBJB
k8E0B5cTT66zJGvBw6yKY4Sv7OnHX+B2pZ6yYH0F6mDXfpFUF1MHwKo5Rqj/uFvPQ0YUAXwjRx
h7RphY0Ycuk08UErduSWNWXNc3eCr8MAuxHJJItAVaxhhw==;
ARC-Authentication-Results: i=1; synaq.com;
iprev=fail smtp.remote-ip=197.90.32.26;
auth=pass (PLAIN) smtp.auth=frank@chagford.com;
arc=none
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed; d=synaq.com; s=securemail;
t=1705327019;
bh=fAjonSIwnT3JLZThOw2zVstNTlp6OYbC0wYnDH78jvk=;
h=Content-Transfer-Encoding:Content-Type:Subject:From:To:MIME-Version:Date:
Message-ID:DKIM-Signature;
b=0n3wdxaGWUHj/sQ6Rd48zOKzq++9/Yb0+svcHruDm102lqM8obOoJhsymcS2veUe11wqRib3zP
6HmS18FVabzt9dcpszfpxjhxe91bu/UD1y9DzjALlVq9DdklggicJOQQ+fWxk3bKTBHvpdZ22c
6Y3eF0tO1lOm3ligA0UngVcfMYiR0L6WZlnkxzAmJUw+z1Vd+bawfBAlqvAKFNH+pyRWDO43ul
QRyVeQ0gVjpVkO/YyKDUxqZROLQyUX1IWeAhAdDHCom48ky+rGDTqLlruNeEElxInAdpNacn1g
/FD5+dy0oXbsHmMIl8DKN2fALF6R/y/m4okdMv8i9BuOgA==;
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
d=chagford.com; s=securemail; h=From:To:Date:Message-ID:in-reply-to;
bh=eo7yd1Sb+z4BtOTj1pDCaAXg5pAWpcpeCWsbO2q/quY=; b=G5fXUVz3qBrzgYZSRYi/Sn1Vdv
gLiKcYxrO7ner2ohYxVPSFUS/Qr+MDLiCZLJYVdqynPFt94tol/dyR0FINhSNGOZXVph+67gIdaWt
ZskEDFvmK31N6FfBbkX0z1rjoBRhCdArP7/+R9LPYvG9cql1wPf1tsi28ZfXfISqdxBuP2A7IUew0
1obEMBFr0agH0jr6Ug4+jC9FX37E7f9iBoSVPG9tYIDYBZDk+CJ0fY9kb8gNnslWLFhcurhWfG5bt
wdw3ZTv4Gwsj3T2knUpRnxv9eR8Ly86EdbMfD2X+BGjRNL6OXUVkqtfpaUvajQzrMMcQ7ihSCj5Ue
lr+r190g==;
Authentication-Results: synaq.com; iprev=fail smtp.remote-ip=197.90.32.26;
auth=pass (PLAIN) smtp.auth=frank@chagford.com;
arc=none
User-Agent: Mozilla Thunderbird
Content-Language: en-US
X-Red-Router: yes
X-SYNAQ-Pinpoint-Information: Please contact SYNAQ for more information
X-SYNAQ-Pinpoint-ID: 1rPNNS-0006iH-SQ
X-SYNAQ-Pinpoint: No virus infections found
X-Pinpoint-From: frank@chagford.com
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
<python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
<mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
<mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <35f5aba8-047e-477f-b1d5-6892d77aa9d8@chagford.com>
 by: Frank Millman - Mon, 15 Jan 2024 13:51 UTC

Hi all

I have read that one should not have to worry about garbage collection
in modern versions of Python - it 'just works'.

I don't want to rely on that. My app is a long-running server, with
multiple clients logging on, doing stuff, and logging off. They can
create many objects, some of them long-lasting. I want to be sure that
all objects created are gc'd when the session ends.

I do have several circular references. My experience is that if I do not
take some action to break the references when closing the session, the
objects remain alive. Below is a very simple program to illustrate this.

Am I missing something? All comments appreciated.

Frank Millman

==================================================

import gc

class delwatcher:
    # This stores enough information to identify the object being watched.
    # It does not store a reference to the object itself.
    def __init__(self, obj):
        self.id = (obj.type, obj.name, id(obj))
        print('***', *self.id, 'created ***')
    def __del__(self):
        print('***', *self.id, 'deleted ***')

class Parent:
    def __init__(self, name):
        self.type = 'parent'
        self.name = name
        self.children = []
        self._del = delwatcher(self)

class Child:
    def __init__(self, parent, name):
        self.type = 'child'
        self.parent = parent
        self.name = name
        parent.children.append(self)
        self._del = delwatcher(self)

p1 = Parent('P1')
p2 = Parent('P2')

c1_1 = Child(p1, 'C1_1')
c1_2 = Child(p1, 'C1_2')
c2_1 = Child(p2, 'C2_1')
c2_2 = Child(p2, 'C2_2')

input('waiting ...')

# if next 2 lines are included, parent and child can be gc'd
# for ch in p1.children:
#     ch.parent = None

# if next line is included, child can be gc'd, but not parent
# p1.children = None

del c1_1
del p1
gc.collect()

input('wait some more ...')

Re: Question about garbage collection

<87v87uvxej.fsf@nightsong.com>

  copy mid

https://news.novabbs.org/devel/article-flat.php?id=25124&group=comp.lang.python#25124

  copy link   Newsgroups: comp.lang.python
Path: i2pn2.org!i2pn.org!eternal-september.org!feeder3.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: no.email@nospam.invalid (Paul Rubin)
Newsgroups: comp.lang.python
Subject: Re: Question about garbage collection
Date: Mon, 15 Jan 2024 18:12:52 -0800
Organization: A noiseless patient Spider
Lines: 16
Message-ID: <87v87uvxej.fsf@nightsong.com>
References: <35f5aba8-047e-477f-b1d5-6892d77aa9d8@chagford.com>
<mailman.62.1705327022.15798.python-list@python.org>
MIME-Version: 1.0
Content-Type: text/plain
Injection-Info: dont-email.me; posting-host="d9d7d5da9281a8765f94bf1754d7c367";
logging-data="1222293"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1/kWZFcTyjggEe3kjdklYET"
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux)
Cancel-Lock: sha1:K9ZZ4ScWZY2yfROBEdRg3XFerwo=
sha1:o8jxRs9XIUKjZPvf99t3MtkuxQ4=
 by: Paul Rubin - Tue, 16 Jan 2024 02:12 UTC

Frank Millman <frank@chagford.com> writes:
> I don't want to rely on that. My app is a long-running server, with
> multiple clients logging on, doing stuff, and logging off. They can
> create many objects, some of them long-lasting. I want to be sure that
> all objects created are gc'd when the session ends.

It's very hard to be sure there are no memory leaks in a system with
gc. Wanting to do that is why Rust was invented. The best you can do
is monitor for leaks, by looking at the gc stats now and then to make
sure that the footprint isn't increasing.

There is an Erlang saying that no uniprocessor system can be reliable,
since the power cord is a single point of failure. That is, if you are
trying to run a high availability system, you need a way to restart the
server now and then without dropping connections. There are ways to do
that, but it drifts from gc per se.

Re: Question about garbage collection (Posting On Python-List Prohibited)

<uo55mm$1amss$1@dont-email.me>

  copy mid

https://news.novabbs.org/devel/article-flat.php?id=25129&group=comp.lang.python#25129

  copy link   Newsgroups: comp.lang.python
Path: i2pn2.org!i2pn.org!eternal-september.org!feeder3.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: ldo@nz.invalid (Lawrence D'Oliveiro)
Newsgroups: comp.lang.python
Subject: Re: Question about garbage collection (Posting On Python-List
Prohibited)
Date: Tue, 16 Jan 2024 05:54:30 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 12
Message-ID: <uo55mm$1amss$1@dont-email.me>
References: <35f5aba8-047e-477f-b1d5-6892d77aa9d8@chagford.com>
<mailman.62.1705327022.15798.python-list@python.org>
<87v87uvxej.fsf@nightsong.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Injection-Date: Tue, 16 Jan 2024 05:54:30 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="ca75c4ad8a1ef3af6a8bb895abab775c";
logging-data="1399708"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1/Z19HkxaBe/M81tW+ceUFO"
User-Agent: Pan/0.155 (Kherson; fc5a80b8)
Cancel-Lock: sha1:8eKsK+/ESz+OlIJE+fHfK+qFRSc=
 by: Lawrence D'Oliv - Tue, 16 Jan 2024 05:54 UTC

On Mon, 15 Jan 2024 18:12:52 -0800, Paul Rubin wrote:

> It's very hard to be sure there are no memory leaks in a system with gc.

Luckily, Python doesn’t do pure GC. I think the term is “ORC”, for
“reference counting with cycles”.

There are techniques you can use. Like when installing a callback into an
object which points back to the object itself, you can stick a weak
reference somewhere to break the cycle of strong references. If it’s a
user-supplied callback, then the trick is to hide the weak reference in a
part of the chain that the user will never see.

devel / comp.lang.python / Question about garbage collection

1
server_pubkey.txt

rocksolid light 0.9.81
clearnet tor