› › ›

Tutorial: Update existing data stream

To update the lifecycle of an existing data stream you do the following actions:

Set a data stream’s lifecycle

To add or to change the retention period of your data stream you can use the PUT lifecycle API.

You can set infinite retention period, meaning that your data should never be deleted. For example:

resp = client.indices.put_data_lifecycle(
    name="my-data-stream",
)
print(resp)

response = client.indices.put_data_lifecycle(
  name: 'my-data-stream',
  body: {}
)
puts response

const response = await client.indices.putDataLifecycle({
  name: "my-data-stream",
});
console.log(response);

PUT _data_stream/my-data-stream/_lifecycle
{ }

An empty payload means that your data stream is still managed but the data will never be deleted. Managing a time series data stream such as logs or metrics enables Elasticsearch to better store your data even if you do not use a retention period.

Or you can set the retention period of your choice. For example:

resp = client.indices.put_data_lifecycle(
    name="my-data-stream",
    data_retention="30d",
)
print(resp)

response = client.indices.put_data_lifecycle(
  name: 'my-data-stream',
  body: {
    data_retention: '30d'
  }
)
puts response

const response = await client.indices.putDataLifecycle({
  name: "my-data-stream",
  data_retention: "30d",
});
console.log(response);

PUT _data_stream/my-data-stream/_lifecycle
{
  "data_retention": "30d" 
}

The retention period of this data stream is set to 30 days. This means that Elasticsearch is allowed to delete data that is older than 30 days at its own discretion.

The changes in the lifecycle are applied on all backing indices of the data stream. You can see the effect of the change via the explain API:

resp = client.indices.explain_data_lifecycle(
    index=".ds-my-data-stream-*",
)
print(resp)

response = client.indices.explain_data_lifecycle(
  index: '.ds-my-data-stream-*'
)
puts response

const response = await client.indices.explainDataLifecycle({
  index: ".ds-my-data-stream-*",
});
console.log(response);

GET .ds-my-data-stream-*/_lifecycle/explain

The response will look like:

{
  "indices": {
    ".ds-my-data-stream-2023.04.19-000002": {
      "index": ".ds-my-data-stream-2023.04.19-000002",  
      "managed_by_lifecycle": true,                           
      "index_creation_date_millis": 1681919221417,
      "time_since_index_creation": "6.85s",             
      "lifecycle": {
        "enabled": true,
        "data_retention": "30d"                         
      }
    },
    ".ds-my-data-stream-2023.04.17-000001": {
      "index": ".ds-my-data-stream-2023.04.17-000001",  
      "managed_by_lifecycle": true,                           
      "index_creation_date_millis": 1681745209501,
      "time_since_index_creation": "48d",               
      "rollover_date_millis": 1681919221419,
      "time_since_rollover": "6.84s",                   
      "generation_time": "6.84s",                       
      "lifecycle": {
        "enabled": true,
        "data_retention": "30d"                         
      }
    }
  }
}

	The name of the backing index.
	This index is managed by a data stream lifecycle.
	The time that has passed since this index has been created.
	The data retention for this index is at least 30 days, as it was recently updated.
	The name of the backing index.
	This index is managed by the built-in data stream lifecycle.
	The time that has passed since this index has been created.
	The time that has passed since this index was rolled over.
	The time that will be used to determine when it’s safe to delete this index and all its data.
	The data retention for this index as well is at least 30 days, as it was recently updated.

Remove lifecycle for a data stream

To remove the lifecycle of a data stream you can use the delete lifecycle API. As consequence, the maintenance operations that were applied by the lifecycle will no longer be applied to the data stream and all its backing indices. For example:

resp = client.indices.delete_data_lifecycle(
    name="my-data-stream",
)
print(resp)

response = client.indices.delete_data_lifecycle(
  name: 'my-data-stream'
)
puts response

const response = await client.indices.deleteDataLifecycle({
  name: "my-data-stream",
});
console.log(response);

DELETE _data_stream/my-data-stream/_lifecycle

You can then use the explain API again to see that the indices are no longer managed.

resp = client.indices.explain_data_lifecycle(
    index=".ds-my-data-stream-*",
)
print(resp)

response = client.indices.explain_data_lifecycle(
  index: '.ds-my-data-stream-*'
)
puts response

const response = await client.indices.explainDataLifecycle({
  index: ".ds-my-data-stream-*",
});
console.log(response);

GET .ds-my-data-stream-*/_lifecycle/explain

{
  "indices": {
    ".ds-my-data-stream-2023.04.19-000002": {
      "index": ".ds-my-data-stream-2023.04.19-000002",  
      "managed_by_lifecycle": false                           
    },
    ".ds-my-data-stream-2023.04.17-000001": {
      "index": ".ds-my-data-stream-2023.04.19-000001",  
      "managed_by_lifecycle": false                           
    }
  }
}

	The name of the backing index.
	Indication that the index is not managed by the data stream lifecycle.
	The name of another backing index.
	Indication that the index is not managed by the data stream lifecycle.

© 2023-2025 Elasticsearch
As of September 2024, Elasticsearch is available under a choice of three licenses: the Server Side Public License (SSPL), the Elastic License, or the AGPLv3 (OSI approved).
Elasticsearch and the Elasticsearch logo are trademarks of Elasticsearch B.V., registered in the U.S. and in other countries.
https://www.elastic.co/guide/en/elasticsearch/reference/8.17/tutorial-manage-existing-data-stream.html