Add_docs method fails on second call

I’m using the add_docs method in Python in the following line of code:
res = rs.Collection.add_docs(name=collection_name__, docs=data__)
In the script I’m using there is a loop that would add some docs (a list of dicts) in each step. When I run the script directly everything works as expected. Having deployed the application with docker i’m facing an issue though.
On the first step of the loop the method call works as expected. On the second step it ends up in an error however. The process is stuck for 1 minute until the error is thrown.
Working around the issue I tried using a larger VI and placing the code that calls the add_docs method outside of the loop. Both would still produce the error though.
I’m using the rockset package version 0.7.116.
Please find the error below:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/lib/python3.8/http/client.py", line 1347, in getresponse
    response.begin()
  File "/usr/local/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/local/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/local/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
  File "/usr/local/lib/python3.8/ssl.py", line 1241, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/local/lib/python3.8/ssl.py", line 1099, in read
    return self._sslobj.read(len, buffer)
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "single_model_performance_and_selection.py", line 115, in write_to_rockset_function
    res = rs.Collection.add_docs(name=collection_name__, docs=data__)
  File "/usr/local/lib/python3.8/site-packages/rockset/client.py", line 454, in add_docs
    return c.add_docs(docs, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/rockset/collection.py", line 445, in add_docs
    DocumentsApi(self.client).add(
  File "/usr/local/lib/python3.8/site-packages/rockset/swagger_client/api/documents_api.py", line 57, in add
    (data) = self.add_with_http_info(workspace, collection, body, **kwargs)  # noqa: E501
  File "/usr/local/lib/python3.8/site-packages/rockset/swagger_client/api/documents_api.py", line 133, in add_with_http_info
    return self.api_client.call_api(
  File "/usr/local/lib/python3.8/site-packages/rockset/client.py", line 316, in call_api
    result = self.api_client.call_api(resource, method, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/rockset/swagger_client/api_client.py", line 324, in call_api
    return self.__call_api(resource_path, method,
  File "/usr/local/lib/python3.8/site-packages/rockset/swagger_client/api_client.py", line 156, in __call_api
    response_data = self.request(
  File "/usr/local/lib/python3.8/site-packages/rockset/swagger_client/api_client.py", line 366, in request
    return self.rest_client.POST(url,
  File "/usr/local/lib/python3.8/site-packages/rockset/swagger_client/rest.py", line 269, in POST
    return self.request("POST", url,
  File "/usr/local/lib/python3.8/site-packages/rockset/swagger_client/rest.py", line 162, in request
    r = self.pool_manager.request(
  File "/usr/local/lib/python3.8/site-packages/urllib3/request.py", line 78, in request
    return self.request_encode_body(
  File "/usr/local/lib/python3.8/site-packages/urllib3/request.py", line 170, in request_encode_body
    return self.urlopen(method, url, **extra_kw)
  File "/usr/local/lib/python3.8/site-packages/urllib3/poolmanager.py", line 375, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/site-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/site-packages/urllib3/packages/six.py", line 769, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/lib/python3.8/http/client.py", line 1347, in getresponse
    response.begin()
  File "/usr/local/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/local/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/local/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
  File "/usr/local/lib/python3.8/ssl.py", line 1241, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/local/lib/python3.8/ssl.py", line 1099, in read
    return self._sslobj.read(len, buffer)
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

If further information is needed please let me know.
Any hints on how to possibly solve this are appreciated, thanks!

Hi @aaron -

Does something like this help: python - requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')) - Stack Overflow

If not, please let me know, and I’ll investigate further.

–n

Hey @aaron

touching base- did this help?

Hi @nadine
thanks for the quick reply. The proposed fix did not solve my issue though.
As outlined in the description, if I run the script directly it works as expected. If I deploy the application using docker it only works for the first time. All subsequent tries to save the data to rockset are causing the error.

Hey @aaron

I’m still unsure what’s going on- but I have something that does work. Can you try this and compare it to the file you do have? When I run this script, I don’t get any errors.

To Run:
• Put the DockerFile and test.py in the same directory
• Put your API Key in the DockerFile
docker build -f DockerFile -t docker-test .
docker run docker-test:latest

Here’s the DockerFile

FROM python:3.8-slim

ENV ROCKSET_APIKEY=YOUR APIKEY
ENV COLLECTION=loadtest
ENV API=api.rs2.usw2.rockset.com

ADD test.py /

RUN pip install --no-cache-dir rockset six

CMD [ "python", "./test.py" ]

Here’s the test file: test.py

from rockset import Client, Q, F
from datetime import datetime
import random
import string
import os

client = Client(api_server=os.getenv('API'), api_key=os.getenv('ROCKSET_APIKEY'))
collection = client.Collection.retrieve(os.getenv('COLLECTION'))

for x in range(0,10):
	body = {}
	body['id'] = x
	body['time'] = datetime.now()
	body['data'] = ''.join([random.choice(string.ascii_letters) for i in range(1000)])
	response = collection.add_docs([body])
	print(response)

This was able to work for me. Please let me know if this helps-

Best,
n

*** edit- updated the DockerFile with
FROM python:3.8-slim

it wasn’t showing up in the text because of the spacing.

Thanks for the suggestion.
I implemented a workaround now using the timeout parameter in add_docs. Whenever the function would hit the error I can catch it before the full minute has passed through this parameter. If I hit the timeout I simply try inserting again and it works on the second try.

Best,
Aaron

1 Like