osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[jira] [Created] (FLINK-10806) Support consume from earliest offset with a new topic


Jiayi Liao created FLINK-10806:
----------------------------------

             Summary: Support consume from earliest offset with a new topic
                 Key: FLINK-10806
                 URL: https://issues.apache.org/jira/browse/FLINK-10806
             Project: Flink
          Issue Type: Improvement
          Components: Kafka Connector
    Affects Versions: 1.6.2
            Reporter: Jiayi Liao
            Assignee: Jiayi Liao


In KafkaConsumerBase, we discover the TopicPartitions and compare them with the restoredState. It's reasonable when a topic's partitions scaled. However, if we add a new topic which has too much data and restore the Flink program, the data of the new topic will be consumed from the start, which may not be what we want.  I think this should be an option for developers.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)