UnicodeDecodeError when reading Python 2 objects from memcache

[dave-2]

Expected Behavior

memcache.get() returns an object from memcache, even if it was added to memcache using the python27 runtime.

Actual Behavior

Traceback (most recent call last):
  ... <truncated> ...
  File "/srv/services/secrets_svc.py", line 59, in GetSecrets
    secrets = memcache.get(GLOBAL_KEY)
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/google/appengine/api/memcache/__init__.py", line 583, in get
    results = rpc.get_result()
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/google/appengine/api/apiproxy_stub_map.py", line 648, in get_result
    return self.__get_result_hook(self)
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/google/appengine/api/memcache/__init__.py", line 652, in __get_hook
    value = _decode_value(returned_item.value,
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/google/appengine/api/memcache/__init__.py", line 289, in _decode_value
    return do_unpickle(value)
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/google/appengine/api/memcache/__init__.py", line 425, in _do_unpickle
    return unpickler.load()
UnicodeDecodeError: 'ascii' codec can't decode byte 0x93 in position 55: ordinal not in range(128)

Steps to Reproduce the Problem

1. Call memcache.set() on an ndb.Model object in python27.
2. Call memcache.get() to get the object in python39.

This reproduces even with MEMCACHE_USE_CROSS_COMPATIBLE_PROTOCOL set in app.yaml. This sets the pickling protocol to 2, which only affects pickling and not unpickling, since the protocol is autodetected upon unpickling.

env_variables:
  MEMCACHE_USE_CROSS_COMPATIBLE_PROTOCOL: "2"

Workaround

There is a workaround by setting the encoding to 'bytes' in the memcache unpickler. It must be 'bytes' instead of 'latin1' because the ndb.Model deserializer expects a bytes object.

import functools
import six

from google.appengine.api import memcache

unpickler = functools.partial(six.moves.cPickle.Unpickler, encoding='bytes')
memcache.setup_client(memcache.Client(unpickler=unpickler))

The Python docs and bug tracker indicate that the encoding argument of pickle.Unpickler() is used to help with the awkwardness in differences in str between Python 2 and 3.
https://docs.python.org/3/library/pickle.html#pickle.Unpickler
https://bugs.python.org/issue22005
https://stackoverflow.com/questions/28218466/unpickling-a-python-2-object-with-python-3/28218598#28218598

Specifications

• Version: appengine-python-standard==0.3.1 and appengine-python-standard==1.0.0
• Platform: python39

[pnico]

does this happen with any ndb.Model? I thought this was only an issue with datetime/date objects?

[dave-2]

does this happen with any ndb.Model? I thought this was only an issue with datetime/date objects?

I'm not sure if it's any ndb.Model. I encountered it with a Model containing 3 properties of type ndb.StringProperty(required=True) and nothing else.

[pnico]

what?? oh no.. I thought ok maybe DateTimeProperty.. but yeah, no 😄
I don't know how to "escalate" an issue here but that sounds like a big problem.

it doesn't seem like encoding='bytes' is really acceptable either, or is it? won't that break other stuff that needs to be unicode? does this happen with the normal, automated memcache caching ndb does or only when you do it manually like this?

Otherwise maybe Python 3 ndb just needs its own memcache namespace or something, it would double memcache use during the transition but it should work..?

Edit: if the automated ndb caching works, and this is only when manually storing things in memcache, I think the best way might be to just use different memcache keys for your Python 2 and Python 3 deployments. But it does raise questions about other things, like deferred, pickling ndb.Key objects with string ids, etc

[pnico]

Can you post a small example with code that shows this issue? Edit: actually I'm wondering about the string stored in the property - if it's something that can't be decoded to utf-8

[dave-2]

Sorry, I won't be able to at the moment, though I think the issue description contains enough information to create a small test case.