Elasticsearch version 7.0.0-beta1
Also see Breaking changes in 7.0.
Breaking changes
- Audit
- Authentication
- Authorization
-
- Remove implicit index monitor privilege #37774
- CCR
- CRUD
- Features/Ingest
- Features/Java Low Level REST Client
- Features/Watcher
- Infra/Logging
- Infra/Packaging
- Machine Learning
- Mapping
- Network
- Ranking
- Search
- Security
- ZenDiscovery
Breaking Java changes
- Features/Java Low Level REST Client
- Infra/Core
Deprecations
- Aggregations
- Analysis
- Audit
- Features/Indices APIs
- Features/Ingest
- Features/Java High Level REST Client
- Features/Watcher
- Infra/Core
- Infra/Scripting
- Machine Learning
-
- Adding ml_settings entry to HLRC and Docs for deprecation_info #38118
- [ML] Datafeed deprecation checks #38026 (issue: #37932)
- [ML] Remove "8" prefixes from file structure finder timestamp formats #38016
- [ML] Adjust structure finder for Joda to Java time migration #37306
- [ML] Resolve 7.0.0 TODOs in ML code #36842 (issue: #29963)
- Mapping
-
- Deprecate types in rollover index API #38039 (issue: #35190)
- Deprecate types in get field mapping API #37667 (issue: #35190)
- Deprecate types in the put mapping API. #37280 (issues: #29453, #37285)
- Support include_type_name in the field mapping and index template APIs. #37210
- Deprecate types in create index requests. #37134 (issues: #29453, #37285)
- Deprecate use of the _type field in aggregations. #37131 (issue: #36802)
- Deprecate reference to _type in lookup queries #37016 (issue: #35190)
- Deprecate the document create endpoint. #36863
- Deprecate types in index API #36575 (issues: #35190, #35790)
- Deprecate types in update APIs #36225
- Search
New features
- Authentication
- Authorization
- CCR
- Features/ILM
- Geo
-
- geotile_grid implementation #37842 (issue: #30240)
- [GEO] Fork Lucene’s LatLonShape Classes to local lucene package #36794
- [Geo] Integrate Lucene’s LatLonShape (BKD Backed GeoShapes) as default
geo_shapeindexing approach #36751 (issue: #35320) - [Geo] Integrate Lucene’s LatLonShape (BKD Backed GeoShapes) as default
geo_shapeindexing approach #35320 (issue: #32039)
- Machine Learning
-
- ML: Adds set_upgrade_mode API endpoint #37837
- Mapping
- SQL
- Search
Enhancements
- Aggregations
-
- Add Composite to AggregationBuilders #38207 (issue: #38020)
- Allow nested fields in the composite aggregation #37178 (issue: #28611)
- Remove single shard optimization when suggesting shard_size #37041 (issue: #32125)
- Use List instead of priority queue for stable sorting in bucket sort aggregator #36748 (issue: #36322)
- Keys are compared in BucketSortPipelineAggregation so making key type… #36407
- Allocation
- Analysis
- Audit
- Authentication
-
- Security: propagate auth result to listeners #36900 (issue: #30794)
- Security: reorder realms based on last success #36878
- Improve error message for 6.x style realm settings #36876 (issues: #30241, #36026)
- Change missing authn message to not mention tokens #36750
- Invalidate Token API enhancements - HLRC #36362 (issue: #35388)
- Enhance Invalidate Token API #35388 (issues: #34556, #35115)
- Authorization
- CCR
-
- Concurrent file chunk fetching for CCR restore #38495
- Tighten mapping syncing in ccr remote restore #38071 (issues: #36879, #37887)
- Do not allow put mapping on follower #37675 (issue: #30086)
- Added ccr to xpack usage infrastructure #37256 (issue: #37221)
- [CCR] FollowingEngine should fail with 403 if operation has no seqno assigned #37213
- [CCR] Added auto_follow_exception.timestamp field to auto follow stats #36947
- [CCR] Add time since last auto follow fetch to auto follow stats #36542 (issues: #33007, #35895)
- CRUD
-
- Add Seq# based optimistic concurrency control to UpdateRequest #37872 (issues: #10708, #36148)
- Introduce ssl settings to reindex from remote #37527 (issues: #29755, #37287)
- Use Sequence number powered OCC for processing updates #37308 (issues: #10708, #36148)
- Document Seq No powered optimistic concurrency control #37284 (issues: #10708, #36148)
- Enable IPv6 URIs in reindex from remote #36874
- Rename seq# powered optimistic concurrency control parameters to ifSeqNo/ifPrimaryTerm #36757 (issues: #10708, #36148)
- Expose Sequence Number based Optimistic Concurrency Control in the rest layer #36721 (issues: #10708, #36148)
- Add doc’s sequence number + primary term to GetResult and use it for updates #36680 (issues: #10708, #36148)
- Add seq no powered optimistic locking support to the index and delete transport actions #36619 (issues: #10708, #36148)
- Set acking timeout to 0 on dynamic mapping update #31140 (issues: #30672, #30844)
- Distributed
-
- Recover retention leases during peer recovery #38435 (issue: #37165)
- Lift retention lease expiration to index shard #38380 (issues: #37165, #37963, #38070)
- Introduce retention lease background sync #38262 (issue: #37165)
- Allow shards of closed indices to be replicated as regular shards #38024 (issue: #33888)
- Expose retention leases in shard stats #37991 (issue: #37165)
- Introduce retention leases versioning #37951 (issue: #37165)
- Soft-deletes policy should always fetch latest leases #37940 (issues: #37165, #37375)
- Sync retention leases on expiration #37902 (issue: #37165)
- Ignore shard started requests when primary term does not match #37899 (issue: #33888)
- Move update and delete by query to use seq# for optimistic concurrency control #37857 (issues: #10708, #36148, #37639)
- Introduce retention lease serialization #37447 (issues: #37165, #37398)
- Add run under primary permit method #37440 (issue: #37398)
- Introduce retention lease syncing #37398 (issue: #37165)
- Introduce retention lease persistence #37375 (issue: #37165)
- Add validation for retention lease construction #37312 (issue: #37165)
- Introduce retention lease expiration #37195 (issue: #37165)
- Introduce shard history retention leases #37167 (issue: #37165)
- [Close Index API] Add unique UUID to ClusterBlock #36775
- [Close Index API] Mark shard copy as stale if needed during shard verification #36755
- [Close Index API] Propagate tasks ids between Freeze, Close and Verify Shard actions #36630
- Always initialize the global checkpoint #34381
- Engine
-
- Specialize pre-closing checks for engine implementations #38702
- Ensure that max seq # is equal to the global checkpoint when creating ReadOnlyEngines #37426
- Enable Bulk-Merge if all source remains #37269
- Rename setting to enable mmap #37070 (issue: #36668)
- Add hybridfs store type #36668
- Introduce time-based retention policy for soft-deletes #34943 (issue: #34908)
- handle AsyncAfterWriteAction failure on primary in the same way as failures on replicas #31969 (issues: #31716, #31755)
- Features/CAT APIs
- Features/Features
- Features/ILM
-
- Ensure ILM policies run safely on leader indices #38140 (issue: #34648)
- Skip Shrink when numberOfShards not changed #37953 (issue: #33275)
- Inject Unfollow before Rollover and Shrink #37625 (issue: #34648)
- Add set_priority action to ILM #37397 (issue: #36905)
- [ILM] Add Freeze Action #36910 (issue: #34630)
- Features/Indices APIs
-
- New mapping signature and mapping string source fixed. #37401
- Features/Ingest
- Features/Java High Level REST Client
- Features/Monitoring
-
- Adding mapping for hostname field #37288
- Features/Stats
-
- Stats: Add JVM dns cache expiration config to JvmInfo #36372
- Features/Watcher
- Infra/Core
- Infra/Logging
-
- Trim the JSON source in indexing slow logs #38081 (issue: #38080)
- Optimize warning header de-duplication #37725 (issues: #35754, #37530, #37597, #37622)
- Remove warn-date from warning headers #37622 (issues: #35754, #37530, #37597)
- Add some deprecation optimizations #37597 (issues: #35754, #37530)
- Only update response headers if we have a new one #37590 (issues: #35754, #37530)
- Infra/Packaging
- Infra/Scripting
- Infra/Settings
-
- Separate out validation of groups of settings #34184
- License
- Machine Learning
-
- Move ML Optimistic Concurrency Control to Seq No #38278 (issues: #10708, #36148)
- [ML] Add explanation so far to file structure finder exceptions #38191 (issue: #29821)
- ML: Add reason field in JobTaskState #38029 (issue: #34431)
- [ML] Add _meta information to all ML indices #37964
- ML: Add upgrade mode docs, hlrc, and fix bug #37942
- [ML] Tighten up use of aliases rather than concrete indices #37874
- ML: Add support for single bucket aggs in Datafeeds #37544 (issue: #36838)
- [ML] Create the ML annotations index #36731 (issues: #26034, #33376)
- [ML] Merge the Jindex master feature branch #36702 (issue: #32905)
- [FEATURE][ML] Add cluster setting to enable/disable config migration #36700 (issue: #32905)
- Mapping
- Network
- Recovery
-
- SyncedFlushService.getShardRoutingTable() should use metadata to check for index existence #37691 (issue: #33888)
- Make prepare engine step of recovery source non-blocking #37573 (issue: #37174)
- Make recovery source send operations non-blocking #37503 (issue: #37458)
- Prepare to make send translog of recovery non-blocking #37458 (issue: #37291)
- Make finalize step of recovery source non-blocking #37388 (issue: #37291)
- Make recovery source partially non-blocking #37291 (issue: #36195)
- Do not mutate RecoveryResponse #37204 (issue: #37174)
- Don’t block on peer recovery on the target side #37076 (issue: #36195)
- Reduce recovery time with compress or secure transport #36981 (issue: #33844)
- Translog corruption marker #33415 (issue: #31389)
- Rollup
-
- Replace the TreeMap in the composite aggregation #36675
- SQL
-
- SQL: Allow look-ahead resolution of aliases for WHERE clause #38450 (issue: #29983)
- SQL: Implement CURRENT_DATE #38175 (issue: #38160)
- SQL: Generate relevant error message when grouping functions are not used in GROUP BY #38017 (issue: #37952)
- SQL: Skip the nested and object field types in case of an ODBC request #37948 (issue: #37801)
- SQL: Add protocol tests and remove jdbc_type from drivers response #37516 (issues: #36635, #36882)
- SQL: Remove slightly used meta commands #37506 (issue: #37409)
- SQL: Describe aliases as views #37496 (issue: #37422)
- SQL: Make
FULLnon-reserved keyword in the grammar #37377 (issue: #37376) - SQL: Use declared source for error messages #37161
- SQL: Improve error message when unable to translate to ES query DSL #37129 (issue: #37040)
- [API] spelling: subtract #37055 (issue: #37035)
- [API] spelling: similar #37054 (issue: #37035)
- [API] spelling: input #37048 (issue: #37035)
- SQL: Enhance message for PERCENTILE[_RANK] with field as 2nd arg #36933 (issue: #36903)
- SQL: Preserve original source for each expression #36912 (issue: #36894)
- SQL: Extend the ODBC metric by differentiating between 32 and 64bit platforms #36753 (issue: #36740)
- SQL: Fix wrong appliance of StackOverflow limit for IN #36724 (issue: #36592)
- Search
-
- Tie break on cluster alias when merging shard search failures #38715 (issue: #38672)
- Add finalReduce flag to SearchRequest #38104 (issues: #37000, #37838)
- Streamline skip_unavailable handling #37672 (issue: #32125)
- Expose sequence number and primary terms in search responses #37639
- Add support for merging multiple search responses into one #37566 (issue: #32125)
- Allow field types to optimize phrase prefix queries #37436 (issue: #31921)
- Add support for providing absolute start time to SearchRequest #37142 (issue: #32125)
- Ensure that local cluster alias is never treated as remote #37121 (issues: #32125, #36997)
- [API] spelling: cacheable #37047 (issue: #37035)
- Add ability to suggest shard_size on coord node rewrite #37017 (issues: #32125, #36997, #37000)
- Skip final reduction if SearchRequest holds a cluster alias #37000 (issues: #32125, #36997)
- Add support for local cluster alias to SearchRequest #36997 (issue: #32125)
- Use SearchRequest copy constructor in ExpandSearchPhase #36772 (issue: #36641)
- Add raw sort values to SearchSortValues transport serialization #36617 (issue: #32125)
- Security
- Snapshot/Restore
-
- RestoreService should update primary terms when restoring shards of existing indices #38177 (issue: #33888)
- Allow open indices to be restored #37733
- Create specific exception for when snapshots are in progress #37550 (issue: #37541)
- SNAPSHOT: Make Atomic Blob Writes Mandatory #37168 (issues: #37011, #37066)
- SNAPSHOT: Speed up HDFS Repository Writes #37069
- Implement Atomic Blob Writes for HDFS Repository #37066 (issue: #37011)
- [API] spelling: repositories #37053 (issue: #37035)
- SNAPSHOT: Use CancellableThreads to Abort #35901 (issue: #21759)
- WIP: S3 client encryption #30513 (issues: #11128, #16843)
- Suggesters
- ZenDiscovery
-
- Add elasticsearch-node detach-cluster tool #37979
- Deprecate minimum_master_nodes #37868
- Step down as master when configured out of voting configuration #37802 (issue: #37712)
- Enforce cluster UUIDs #37775
- Bubble exceptions up in ClusterApplierService #37729
- Use m_m_nodes from Zen1 master for Zen2 bootstrap #37701
- Add tool elasticsearch-node unsafe-bootstrap #37696
- Report terms and version if cluster does not form #37473
- Bootstrap a Zen2 cluster once quorum is discovered #37463
- Zen2: Add join validation #37203
- Publish cluster states in chunks #36973
Bug fixes
- Aggregations
- Allocation
- Audit
- Authentication
- CCR
-
- Prevent CCR recovery from missing documents #38237
- Fix file reading in ccr restore service #38117
- Correct argument names in update mapping/settings from leader #38063
- Ensure changes requests return the latest mapping version #37633
- Do not set fatal exception when shard follow task is stopped. #37603
- Add fatal_exception field for ccr stats in monitoring mapping #37563
- Do not add index event listener if CCR disabled #37432
- When removing an AutoFollower also mark it as removed. #37402 (issue: #36761)
- [CCR] Make shard follow tasks more resilient for restarts #37239 (issue: #37231)
- [CCR] Resume follow Api should not require a request body #37217 (issue: #37022)
- [CCR] Report error if auto follower tries auto follow a leader index with soft deletes disabled #36886 (issue: #33007)
- Remote cluster license checker and no license info. #36837 (issue: #36815)
- Make CCR resilient against missing remote cluster connections #36682 (issues: #36255, #36667)
- [CCR] AutoFollowCoordinator and follower index already created #36540 (issue: #33007)
- CRUD
- Distributed
-
- TransportVerifyShardBeforeCloseAction should force a flush #38401 (issues: #33888, #37961)
- Fix limit on retaining sequence number #37992 (issue: #37165)
- Close Index API should force a flush if a sync is needed #37961 (issues: #33888, #37426)
- Force Refresh Listeners when Acquiring all Operation Permits #36835
- Replaced the word shards with replicas in an error message. (#36234) #36275 (issue: #36234)
- Engine
- Features/ILM
- Features/Indices APIs
- Features/Ingest
-
- Support unknown fields in ingest pipeline map configuration #38352 (issue: #36938)
- Ingest node - user_agent, move device parsing to an object #38115 (issues: #37329, #38094)
- ingest: fix on_failure with Drop processor #36686 (issue: #36151)
- ingest: support default pipelines + bulk upserts #36618 (issue: #36219)
- Features/Java High Level REST Client
-
- Update IndexTemplateMetadata to allow unknown fields #38448 (issue: #36938)
-
if_seq_noandif_primary_termparameters aren’t wired correctly in REST Client’s CRUD API #38411 - Update Rollup Caps to allow unknown fields #38339 (issue: #36938)
- Fix ILM explain response to allow unknown fields #38054 (issue: #36938)
- Fix ILM status to allow unknown fields #38043 (issue: #36938)
- Fix ILM Lifecycle Policy to allow unknown fields #38041 (issue: #36938)
- Update authenticate to allow unknown fields #37713 (issue: #36938)
- Update verify repository to allow unknown fields #37619 (issue: #36938)
- Update get users to allow unknown fields #37593 (issue: #36938)
- Update Execute Watch to allow unknown fields #37498 (issue: #36938)
- Update Put Watch to allow unknown fields #37494 (issue: #36938)
- Update Delete Watch to allow unknown fields #37435 (issue: #36938)
- Fix rest reindex test for IPv4 addresses #37310
- Fix weighted_avg parser not found for RestHighLevelClient #37027 (issue: #36861)
- Features/Java Low Level REST Client
-
- Fix potential IllegalCapacityException in LLRC when selecting nodes #37821
- Features/Monitoring
- Features/Watcher
- Geo
- Infra/Core
- Infra/Packaging
- Infra/REST API
- Infra/Scripting
-
- Fix Painless void return bug #38046
- Infra/Settings
- Machine Learning
-
- [ML] Report index unavailable instead of waiting for lazy node #38423
- ML: Fix error race condition on stop _all datafeeds and close _all jobs #38113 (issue: #37959)
- [ML] Update ML results mappings on process start #37706 (issue: #37607)
- [ML] Prevent submit after autodetect worker is stopped #37700 (issue: #37108)
- [ML] Fix ML datafeed CCS with wildcarded cluster name #37470 (issue: #36228)
- [ML] Update error message for process update #37363
- [ML] Wait for autodetect to be ready in the datafeed #37349 (issues: #36810, #37227)
- [ML] Stop datafeeds running when their jobs are stale #37227 (issue: #36810)
- [ML] Order GET job stats response by job id #36841 (issue: #36683)
- [ML] Make GetJobStats work with arbitrary wildcards and groups #36683 (issue: #34745)
- Mapping
-
- Treat put-mapping calls with
_docas a top-level key as typed calls. #38032 - Correct deprec log in RestGetFieldMappingAction #37843 (issue: #37667)
- Restore a noop _all metadata field for 6x indices #37808 (issue: #37429)
- Make sure PutMappingRequest accepts content types other than JSON. #37720
- Make sure to use the resolved type in DocumentMapperService#extractMappings. #37451 (issue: #36811)
- MAPPING: Improve Precision for scaled_float #37169 (issue: #32570)
- Make sure to accept empty unnested mappings in create index requests. #37089
- Stop automatically nesting mappings in index creation requests. #36924
- Rewrite SourceToParse with resolved docType #36921 (issues: #35790, #36769)
- Treat put-mapping calls with
- Network
- Ranking
-
- QueryRescorer should keep the window size when rewriting #36836
- Recovery
-
- RecoveryMonitor#lastSeenAccessTime should be volatile #36781
- Rollup
- SQL
-
- SQL: Prevent grouping over grouping functions #38649 (issue: #38308)
- SQL: Relax StackOverflow circuit breaker for constants #38572 (issue: #38571)
- SQL: Fix issue with IN not resolving to underlying keyword field #38440 (issue: #38424)
- SQL: change the Intervals milliseconds precision to 3 digits #38297 (issue: #37423)
- SQL: Fix esType for DATETIME/DATE and INTERVALS #38179 (issue: #38051)
- SQL: Added SSL configuration options tests #37875 (issue: #37711)
- SQL: Fix casting from date to numeric type to use millis #37869 (issue: #37655)
- SQL: Fix BasicFormatter NPE #37804
- SQL: Return Intervals in SQL format for CLI #37602 (issues: #29970, #36186, #36432)
- SQL: fix object extraction from sources #37502 (issue: #37364)
- SQL: Fix issue with field names containing "." #37364 (issue: #37128)
- SQL: Fix bug regarding alias fields with dots #37279 (issue: #37224)
- SQL: Proper handling of COUNT(field_name) and COUNT(DISTINCT field_name) #37254 (issue: #30285)
- SQL: fix COUNT DISTINCT filtering #37176 (issue: #37086)
- SQL: Fix issue with wrong NULL optimization #37124 (issue: #35872)
- SQL: Fix issue with complex expression as args of PERCENTILE/_RANK #37102 (issue: #37099)
- SQL: Handle the bwc Joda ZonedDateTime scripting class in Painless #37024 (issue: #37023)
- SQL: Fix bug regarding histograms usage in scripting #36866
- SQL: Fix issue with always false filter involving functions #36830 (issue: #35980)
- SQL: protocol returns ISO 8601 String formatted dates instead of Long for JDBC/ODBC requests #36800 (issue: #36756)
- SQL: Enhance Verifier to prevent aggregate or grouping functions from #36799 (issue: #36798)
- SQL: Fix translation of LIKE/RLIKE keywords #36672 (issues: #36039, #36584)
- SQL: Scripting support for casting functions CAST and CONVERT #36640 (issue: #36061)
- SQL: Concat should be always not nullable #36601 (issue: #36169)
- SQL: Fix issue with complex HAVING and GROUP BY ordinal #36594 (issue: #36059)
- Search
-
- Look up connection using the right cluster alias when releasing contexts #38570
- Fix fetch source option in expand search phase #37908 (issue: #23829)
- Change
rationaltosaturationin script_score #37766 (issue: #37714) - Throw if two inner_hits have the same name #37645 (issue: #37584)
- Ensure either success or failure path for SearchOperationListener is called #37467 (issue: #37185)
-
query_stringshould use indexed prefixes #36895 - Avoid duplicate types deprecation messages in search-related APIs. #36802
- Security
- Snapshot/Restore
- Suggesters
- Task Management
-
- Un-assign persistent tasks as nodes exit the cluster #37656
- ZenDiscovery
Regressions
- Infra/Core
- Mapping
Upgrades
© 2023-2025 Elasticsearch
As of September 2024, Elasticsearch is available under a choice of three licenses: the Server Side Public License (SSPL), the Elastic License, or the AGPLv3 (OSI approved).
Elasticsearch and the Elasticsearch logo are trademarks of Elasticsearch B.V., registered in the U.S. and in other countries.
https://www.elastic.co/guide/en/elasticsearch/reference/7.17/release-notes-7.0.0-beta1.html