I have a topic with a timestamp avro logical type.
I sent some events with today’s date but the values which i got in my collection are in 1970-01-20
event in the topic
Date in rocketset :
are the avro logical types supported in rockset? Am I doing something wrong?
Thank you for you help
Hi @mourad.d !
Can you try doing a SQL transform to coerce the LONG to a timestamp with
TIMESTAMP_MILLIS? We are going to investigate to see if it’s supported on our side by default.
Hi @mourad.d , I guess you are using some avro converter when setting up your kafka connect as specified here: Kafka ?
I guess the transformation of that timestamp from long to string might be done in the avro converter instead by of rockset. But I might be wrong.
If you could provide more information on how you setup the kafka connect, that would be helpful for us to debug.
my connector config
name = RocksetSinkConnectorConnector_0
connector.class = rockset.RocksetSinkConnector
tasks.max = 10
key.converter = org.apache.kafka.connect.storage.StringConverter
value.converter = io.confluent.connect.avro.AvroConverter
topics = test
rockset.apiserver.url = https://api.euc1a1.rockset.com
rockset.integration.key = kafka://firstname.lastname@example.org
rockset.task.threads = 5
format = AVRO
value.converter.schema.registry.url = http://schema-registry:8081
shows that updated_date type is
oh nvm. I misunderstood the question. Please follow @nadine 's suggestion.
When I add the transformation, no event is received
SELECT id as _id, TIMESTAMP_MILLIS(updated_date) as updated_date, * FROM _input
Investigating- one moment.
@mourad.d - can you confirm the topic is getting data?
yes - created a second collection without the transformation and I can see data coming
thanks- we’re looking into it
@mourad.d - do we have your permission to create a support key for your account? We’re going to investigate. Your explicit permission allows us to investigate your account.
yes, you have the permission
Thanks @mourad.d - we are looking into it.
@mourad.d I’m going to follow up Monday with our engineering team. I’ll work on response for you early next week.
HI @mourad.d !
We have investigated, and it’s an issue on our side. We are working on a fix.
You can use the following sql transformation. This will break when we roll out the actual fix, but if you want to use it now, I can update this thread when the fix happens.
TIMESTAMP_MICROS(UNIX_MICROS(updated_date) * 1000)
Thanks for posting! Let me know if this resolves your issue!