Skip to content

Commit

Permalink
Changes:
Browse files Browse the repository at this point in the history
- docs and tests for cages
- add use_wrapper_for_read
- rename parameter self_register to skip_self_register and invert
- add use_wrapper
  • Loading branch information
devkral committed Nov 26, 2024
1 parent a5981eb commit 4647956
Show file tree
Hide file tree
Showing 6 changed files with 163 additions and 26 deletions.
92 changes: 90 additions & 2 deletions docs/cages.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
# Cages

Cages are a way to manage global states in an non-global maner.
Cages are transparent proxies to context variables. It forwards all function calls to the wrapped object so it behaves
normal in most cases like the original despite all operations are executed on the contextvar clone of the original.
The original is copied via copy or optionally via deepcopy.

This allows monkey patching other libraries which are not async safe. Even from other libraries,

## Usage

There are two methods
There are two ways

1. registering via self registering (recommended)
2. registering manually
Expand All @@ -14,3 +17,88 @@ The first way is recommended because it can be detected if it would nest another
In this case it would just skip the initialization and the old Cage is kept.

Advantage of this: multiple libraries can patch other libraries without fearing to overwrite another cage.

``` python
from monkay import Cage

foo = []
foo2: list

# we can move an existing variable in a cage
Cage(globals(), name="foo", update_fn=lambda overwrite, new_original: new_original+overwrite)
# we can inject
Cage(globals(), [], name="foo2")
# we can manually assign
original: list = []
cage_for_original = Cage(globals(), name="original", skip_self_register=True)

foo.add("a")
foo.add("b")
assert foo == ["a", "b"]

with foo.monkay_with_override(["b", "c"]):
assert foo == ["b", "c"]
assert foo == ["a", "b"]

with foo.monkay_with_original() as original:
assert original == []
original.append("updated")

# thanks to the update function
assert foo == ["updated", "a", "b"]
```

### With thread Lock

Cages are async safe designed. They can even protect updates of the original via locks.
With `use_wrapper_for_reads=True` inconsistent states for non-async safe copyable structures are prevented.

``` python
from threading import Lock
from monkay import Cage

foo: list
Cage(globals(), [], name="foo", original_wrapper=Lock(), update_fn=lambda overwrite, new_original: new_original+overwrite)

# now threadsafe
with foo.monkay_with_original() as original:
assert original == []
original.append("updated")
```

### Preloads

Of course cages support also preloads. See [Tutorial](./tutorial.md) for examples and the syntax.


### Using deep copy

By default when no contextvariable was initialized the original is copied via `copy.copy()` into the contextvariable.
By providing `deep_copy=True` `copy.deepcopy()` is used.


## Advanced

### New context variable name

By default a context-variable is created and injected to globals with the pattern:

`"_{name}_ctx"` where name is the provided name.

You can define a different by providing the parameter

`context_var_name="different_pattern"` optionally with the name placeholder.


### Skip wrapper

Sometimes you want to skip the wrapper for this call

`cage.monkay_with_original(use_wrapper=False)`

And if you want to persist the contextvar without wrapper despite having `use_wrapper_for_reads=True` set, use:

`cage.monkay_conditional_update_copy(use_wrapper=False)`

You can also do the inverse by calling, when having `use_wrapper_for_reads=False` (default):
`cage.monkay_conditional_update_copy(use_wrapper=True)`
6 changes: 0 additions & 6 deletions docs/settings.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
# Settings


## Setting settings forward

Sometimes you have some packages which should work independently but
Expand All @@ -22,7 +21,6 @@ monkay = Monkay(
globals(),
settings_path=os.environ.get("MONKAY_CHILD_SETTINGS", "foo.test:example") or ""
)

```

Main
Expand All @@ -36,10 +34,8 @@ monkay = Monkay(
settings_path=os.environ.get("MONKAY_MAIN_SETTINGS", "foo.test:example") or ""
)
child.monkay.settings = lambda: monkay.settings

```


## Lazy settings setup

Like when using a settings forward it is possible to activate the settings later by assigning a string, a class or an settings instance
Expand Down Expand Up @@ -86,8 +82,6 @@ attribute and are cached.
Functions get evaluated on every access and should care for caching in case it is required (for forwards the caching
takes place in the main settings).



## Forwarder

Sometimes you have an old settings place and want to forward it to the monkay one.
Expand Down
42 changes: 30 additions & 12 deletions monkay/cages.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ class Undefined: ...
class Cage(Generic[T]):
monkay_context_var: ContextVar[tuple[int, T] | type[Undefined]]
monkay_deep_copy: bool
monkay_use_wrapper_for_reads: bool
monkay_update_fn: Callable[[T, T], T] | None
monkay_original: T
monkay_original_last_update: int
Expand All @@ -40,7 +41,8 @@ def __new__(
# for e.g. locks
original_wrapper: AbstractContextManager = nullcontext(),
update_fn: Callable[[T, T], T] | None = None,
self_register: bool = True,
use_wrapper_for_reads: bool = False,
skip_self_register: bool = False,
package: str | None = "",
) -> Cage:
if package == "" and globals_dict.get("__spec__"):
Expand All @@ -57,7 +59,7 @@ def __new__(
if obj is Undefined:
obj = globals_dict[name]
assert obj is not Undefined
if self_register and isinstance(obj, Cage):
if not skip_self_register and isinstance(obj, Cage):
return obj
context_var_name = context_var_name.format(name=name)
obj_type = type(obj)
Expand All @@ -76,6 +78,7 @@ def __new__(
context_var_name, default=Undefined
)
monkay_cage_instance.monkay_deep_copy = deep_copy
monkay_cage_instance.monkay_use_wrapper_for_reads = use_wrapper_for_reads
monkay_cage_instance.monkay_update_fn = update_fn
monkay_cage_instance.monkay_original = obj
monkay_cage_instance.monkay_original_last_update = 0
Expand All @@ -84,7 +87,7 @@ def __new__(
)
monkay_cage_instance.monkay_original_wrapper = original_wrapper

if self_register:
if not skip_self_register:
globals_dict[name] = monkay_cage_instance
return monkay_cage_instance

Expand All @@ -97,31 +100,46 @@ def _(self, *args: Any, **kwargs: Any):
return _

def monkay_refresh_copy(
self, *, obj: T | type[Undefined] = Undefined, _monkay_dict: dict | None = None
self,
*,
obj: T | type[Undefined] = Undefined,
use_wrapper: bool | None = None,
_monkay_dict: dict | None = None,
) -> T:
"""Sets the contextvar."""
if _monkay_dict is None:
_monkay_dict = super().__getattribute__("__dict__")
if use_wrapper is None:
use_wrapper = _monkay_dict["monkay_use_wrapper_for_reads"]
if obj is Undefined:
obj = (
copy.deepcopy(_monkay_dict["monkay_original"])
if _monkay_dict["monkay_deep_copy"]
else copy.copy(_monkay_dict["monkay_original"])
)
with _monkay_dict["monkay_original_wrapper"] if use_wrapper else nullcontext():
obj = (
copy.deepcopy(_monkay_dict["monkay_original"])
if _monkay_dict["monkay_deep_copy"]
else copy.copy(_monkay_dict["monkay_original"])
)
_monkay_dict["monkay_context_var"].set((_monkay_dict["monkay_original_last_update"], obj))
return cast(T, obj)

def monkay_conditional_update_copy(self, *, _monkay_dict: dict | None = None) -> T:
def monkay_conditional_update_copy(
self, *, use_wrapper: bool | None = None, _monkay_dict: dict | None = None
) -> T:
if _monkay_dict is None:
_monkay_dict = super().__getattribute__("__dict__")
if use_wrapper is None:
use_wrapper = _monkay_dict["monkay_use_wrapper_for_reads"]
tup = _monkay_dict["monkay_context_var"].get()
if tup is Undefined:
obj = self.monkay_refresh_copy(_monkay_dict=_monkay_dict)
elif (
_monkay_dict["monkay_update_fn"] is not None
and tup[0] != _monkay_dict["monkay_original_last_update"]
):
obj = _monkay_dict["monkay_update_fn"](obj, _monkay_dict["monkay_original"])
obj = self.monkay_refresh_copy(obj=obj, _monkay_dict=_monkay_dict)
with _monkay_dict["monkay_original_wrapper"] if use_wrapper else nullcontext():
obj = _monkay_dict["monkay_update_fn"](tup[1], _monkay_dict["monkay_original"])
obj = self.monkay_refresh_copy(
obj=obj, _monkay_dict=_monkay_dict, use_wrapper=use_wrapper
)
else:
obj = tup[1]
return obj
Expand Down
2 changes: 2 additions & 0 deletions tests/targets/cages_preloaded.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
def load():
from . import cages_preloaded_fn # noqa
Empty file.
47 changes: 41 additions & 6 deletions tests/test_cages.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import sys
from contextvars import ContextVar
from pathlib import Path
from threading import Lock

import pytest

Expand Down Expand Up @@ -32,17 +33,20 @@ def test_cages_overwrite():
def test_cages_preload_and_register():
d = {}
assert "tests.targets.module_prefixed" not in sys.modules
assert "tests.targets.module_full_preloaded1" not in sys.modules
assert "tests.targets.module_full_preloaded1_fn" not in sys.modules
assert "tests.targets.cages_preloaded" not in sys.modules
assert "tests.targets.cages_preloaded_fn" not in sys.modules
cage = Cage(
d,
target_ro,
name="target_ro",
preloads=["tests.targets.module_prefixed", "tests.targets.module_full_preloaded1:load"],
preloads=[
"tests.targets.module_prefixed",
"tests.targets.cages_preloaded:load",
],
)
assert "tests.targets.module_prefixed" in sys.modules
assert "tests.targets.module_full_preloaded1" in sys.modules
assert "tests.targets.module_full_preloaded1_fn" in sys.modules
assert "tests.targets.cages_preloaded" in sys.modules
assert "tests.targets.cages_preloaded_fn" in sys.modules
assert isinstance(d["target_ro"], Cage)
assert d["target_ro"] is cage
assert isinstance(d["_target_ro_ctx"], ContextVar)
Expand All @@ -61,7 +65,38 @@ def test_cages_retrieve_with_name():
globals(),
name="target_ro",
context_var_name="foo_cages_retrieve_with_name_ctx",
self_register=False,
skip_self_register=True,
)
assert type(globals()["target_ro"]) is not Cage
assert isinstance(globals()["foo_cages_retrieve_with_name_ctx"], ContextVar)


@pytest.mark.parametrize("read_lock", [True, False])
def test_cages_wrapper_for_non_existing(read_lock):
lock = Lock()

def update_fn(context_content, original):
assert lock.locked() == read_lock
return original + context_content

cage = Cage(
globals(),
[],
name="target_cages_wrapper",
context_var_name=f"foo_cages_wrapper_ctx{read_lock}",
skip_self_register=True,
original_wrapper=lock,
use_wrapper_for_reads=read_lock,
update_fn=update_fn,
)
assert type(globals()["target_ro"]) is not Cage
assert isinstance(globals()[f"foo_cages_wrapper_ctx{read_lock}"], ContextVar)
assert cage == []
cage.append("b")
assert cage == ["b"]

with cage.monkay_with_original() as original:
original.append("a")
assert cage == ["a", "b"]
cage.append("c")
assert cage == ["a", "b", "c"]

0 comments on commit 4647956

Please sign in to comment.