Kafka events with avro timestamp logical type

I have a topic with a timestamp avro logical type.

      "name": "updated_date",
      "type": {
        "logicalType": "timestamp-millis",
        "type": "long"

I sent some events with today’s date but the values which i got in my collection are in 1970-01-20

event in the topic


Date in rocketset :


are the avro logical types supported in rockset? Am I doing something wrong?

Thank you for you help

1 Like

Hi @mourad.d !
Can you try doing a SQL transform to coerce the LONG to a timestamp with TIMESTAMP_MILLIS? We are going to investigate to see if it’s supported on our side by default.

Hi @mourad.d , I guess you are using some avro converter when setting up your kafka connect as specified here: Kafka ?

I guess the transformation of that timestamp from long to string might be done in the avro converter instead by of rockset. But I might be wrong.

If you could provide more information on how you setup the kafka connect, that would be helpful for us to debug.

my connector config

name = RocksetSinkConnectorConnector_0
connector.class = rockset.RocksetSinkConnector
tasks.max = 10
key.converter = org.apache.kafka.connect.storage.StringConverter
value.converter = io.confluent.connect.avro.AvroConverter
topics = test
rockset.apiserver.url = https://api.euc1a1.rockset.com
rockset.integration.key = kafka://mykey@api.euc1a1.rockset.com
rockset.task.threads = 5
format = AVRO
value.converter.schema.registry.url = http://schema-registry:8081
1 Like

shows that updated_date type is timestamp

oh nvm. I misunderstood the question. Please follow @nadine 's suggestion.

Hi @nadine,

When I add the transformation, no event is received

SELECT id as _id, TIMESTAMP_MILLIS(updated_date) as updated_date, * FROM _input

Investigating- one moment.

@mourad.d - can you confirm the topic is getting data?

yes - created a second collection without the transformation and I can see data coming

1 Like

thanks- we’re looking into it

@mourad.d - do we have your permission to create a support key for your account? We’re going to investigate. Your explicit permission allows us to investigate your account.

Hi @nadine
yes, you have the permission

1 Like

Thanks @mourad.d - we are looking into it.

@mourad.d I’m going to follow up Monday with our engineering team. I’ll work on response for you early next week.

HI @mourad.d !

We have investigated, and it’s an issue on our side. We are working on a fix.

The workaround:
You can use the following sql transformation. This will break when we roll out the actual fix, but if you want to use it now, I can update this thread when the fix happens.

TIMESTAMP_MICROS(UNIX_MICROS(updated_date) * 1000)

Thanks for posting! Let me know if this resolves your issue!