Skip to content

Commit fed919f

Browse files
jmikolajmikola
authored
Support search indexes in mappings and SchemaManager (#2630)
* Model search index mappings * Store default search index name in ClassMetadata * Support search indexes in SchemaManager * Support search indexes in CLI commands Adds default implementations for "process" methods in AbstractCommand, which throw BadMethodCallException. Renames internal methods in ShardCommand to no longer override "index" base methods, since sharding methods in SchemaManager do much more than process indexes. * Update phpstan baseline for unmodeled search index structs The fields struct is recursive, which is not supported by phpstan. The analyzers struct may be technically possible to model, but the complexity isn't worth the effort. * Require driver 1.17+ for search index APIs * Add skip-search-indexes option to schema CLI commands Currently, commands can either process all definitions (default behavior) or specify individual definitions. This allows the commands to rely on default behavior (e.g. $createOrder) but omit processing of search indexes, which may be more stringent requirements. Note: this is similar to the disable-validators option that already existed in UpdateCommand; however, #2634 suggests renaming that if additional "skip" options are introduced. * Update baseline for Psalm 5.24.0 * SearchIndex annotation docs * Search indexes chapter
1 parent 29c65ce commit fed919f

23 files changed

+1148
-97
lines changed

‎.github/workflows/continuous-integration.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ jobs:
3838
- dependencies: "lowest"
3939
php-version: "8.1"
4040
mongodb-version: "5.0"
41-
driver-version: "1.11.0"
41+
driver-version: "1.17.0"
4242
topology: "server"
4343
symfony-version: "stable"
4444
# Test with highest dependencies

‎composer.json

+2-2
Original file line numberDiff line numberDiff line change
@@ -22,15 +22,15 @@
2222
],
2323
"require": {
2424
"php": "^8.1",
25-
"ext-mongodb": "^1.11",
25+
"ext-mongodb": "^1.17",
2626
"doctrine/cache": "^1.11 || ^2.0",
2727
"doctrine/collections": "^1.5 || ^2.0",
2828
"doctrine/event-manager": "^1.0 || ^2.0",
2929
"doctrine/instantiator": "^1.1 || ^2",
3030
"doctrine/persistence": "^3.2",
3131
"friendsofphp/proxy-manager-lts": "^1.0",
3232
"jean85/pretty-package-versions": "^1.3.0 || ^2.0.1",
33-
"mongodb/mongodb": "^1.10.0",
33+
"mongodb/mongodb": "^1.17.0",
3434
"psr/cache": "^1.0 || ^2.0 || ^3.0",
3535
"symfony/console": "^5.4 || ^6.0 || ^7.0",
3636
"symfony/deprecation-contracts": "^2.2 || ^3.0",

‎docs/en/reference/annotations-reference.rst

+54
Original file line numberDiff line numberDiff line change
@@ -1073,6 +1073,60 @@ Optional attributes:
10731073
*/
10741074
private $cart;
10751075
1076+
@SearchIndex
1077+
------------
1078+
1079+
This annotation is used to specify :ref:`search indexes <search_indexes>` for
1080+
`MongoDB Atlas Search <https://www.mongodb.com/docs/atlas/atlas-search/>`__.
1081+
1082+
The attributes correspond to arguments for
1083+
`MongoDB\Collection::createSearchIndex() <https://www.mongodb.com/docs/php-library/current/reference/method/MongoDBCollection-createSearchIndex/>`__.
1084+
Excluding ``name``, attributes are used to create the
1085+
`search index definition <https://www.mongodb.com/docs/manual/reference/command/createSearchIndexes/#search-index-definition-syntax>`__.
1086+
1087+
Optional attributes:
1088+
1089+
-
1090+
``name`` - Name of the search index to create, which must be unique to the
1091+
collection. Defaults to ``"default"``.
1092+
-
1093+
``dynamic`` - Enables or disables dynamic field mapping for this index.
1094+
If ``true``, the index will include all fields with
1095+
`supported data types <https://www.mongodb.com/docs/atlas/atlas-search/define-field-mappings/#std-label-bson-data-chart>`__.
1096+
If ``false``, the ``fields`` attribute must be specified. Defaults to ``false``.
1097+
-
1098+
``fields`` - Associative array of `field mappings <https://www.mongodb.com/docs/atlas/atlas-search/define-field-mappings/>`__
1099+
that specify the fields to index (keys). Required only if dynamic mapping is disabled.
1100+
-
1101+
``analyzer`` - Specifies the `analyzer <https://www.mongodb.com/docs/atlas/atlas-search/analyzers/>`__
1102+
to apply to string fields when indexing. Defaults to the
1103+
`standard analyzer <https://www.mongodb.com/docs/atlas/atlas-search/analyzers/standard/>`__.
1104+
-
1105+
``searchAnalyzer`` - Specifies the `analyzer <https://www.mongodb.com/docs/atlas/atlas-search/analyzers/>`__
1106+
to apply to query text before the text is searched. Defaults to the
1107+
``analyzer`` attribute, or the `standard analyzer <https://www.mongodb.com/docs/atlas/atlas-search/analyzers/standard/>`__.
1108+
if both are unspecified.
1109+
-
1110+
``analyzers`` - Array of `custom analyzers <https://www.mongodb.com/docs/atlas/atlas-search/analyzers/custom/>`__
1111+
to use in this index.
1112+
-
1113+
``storedSource`` - Specifies document fields to store for queries performed
1114+
using the `returnedStoredSource <https://www.mongodb.com/docs/atlas/atlas-search/return-stored-source/>`__
1115+
option. Specify ``true`` to store all fields, ``false`` to store no fields,
1116+
or a `document <https://www.mongodb.com/docs/atlas/atlas-search/stored-source-definition/#std-label-fts-stored-source-document>`__
1117+
to specify individual fields to include or exclude from storage. Defaults to ``false``.
1118+
-
1119+
``synonyms`` - Array of `synonym mapping definitions <https://www.mongodb.com/docs/atlas/atlas-search/synonyms/>`__
1120+
to use in this index.
1121+
1122+
.. note::
1123+
1124+
Search indexes have some notable differences from `@Index`_. They may only
1125+
be defined on document classes. Definitions will not be incorporated from
1126+
embedded documents. Additionally, ODM will **NOT** translate field names in
1127+
search index definitions. Database field names must be used instead of
1128+
mapped field names (i.e. PHP property names).
1129+
10761130
@ShardKey
10771131
---------
10781132

‎docs/en/reference/search-indexes.rst

+188
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,188 @@
1+
.. _search_indexes:
2+
3+
Search Indexes
4+
==============
5+
6+
In addition to standard :ref:`indexes <indexes>`, ODM allows you to define
7+
search indexes for use with `MongoDB Atlas Search <https://www.mongodb.com/docs/atlas/atlas-search/>`__.
8+
Search indexes may be queried using the `$search <https://www.mongodb.com/docs/atlas/atlas-search/aggregation-stages/search/>`__
9+
and `$searchMeta <https://www.mongodb.com/docs/atlas/atlas-search/aggregation-stages/searchMeta/>`__
10+
aggregation pipeline stages.
11+
12+
Search indexes have some notable differences from regular
13+
:ref:`indexes <indexes>` in ODM. They may only be defined on document classes.
14+
Definitions will not be incorporated from embedded documents. Additionally, ODM
15+
will **NOT** translate field names in search index definitions. Database field
16+
names must be used instead of mapped field names (i.e. PHP property names).
17+
18+
Search Index Options
19+
--------------------
20+
21+
Search indexes are defined using a more complex syntax than regular
22+
:ref:`indexes <indexes>`.
23+
24+
ODM supports the following search index options:
25+
26+
-
27+
``name`` - Name of the search index to create, which must be unique to the
28+
collection. Defaults to ``"default"``.
29+
-
30+
``dynamic`` - Enables or disables dynamic field mapping for this index.
31+
If ``true``, the index will include all fields with
32+
`supported data types <https://www.mongodb.com/docs/atlas/atlas-search/define-field-mappings/#std-label-bson-data-chart>`__.
33+
If ``false``, the ``fields`` attribute must be specified. Defaults to ``false``.
34+
-
35+
``fields`` - Associative array of `field mappings <https://www.mongodb.com/docs/atlas/atlas-search/define-field-mappings/>`__
36+
that specify the fields to index (keys). Required only if dynamic mapping is disabled.
37+
-
38+
``analyzer`` - Specifies the `analyzer <https://www.mongodb.com/docs/atlas/atlas-search/analyzers/>`__
39+
to apply to string fields when indexing. Defaults to the
40+
`standard analyzer <https://www.mongodb.com/docs/atlas/atlas-search/analyzers/standard/>`__.
41+
-
42+
``searchAnalyzer`` - Specifies the `analyzer <https://www.mongodb.com/docs/atlas/atlas-search/analyzers/>`__
43+
to apply to query text before the text is searched. Defaults to the
44+
``analyzer`` attribute, or the `standard analyzer <https://www.mongodb.com/docs/atlas/atlas-search/analyzers/standard/>`__.
45+
if both are unspecified.
46+
-
47+
``analyzers`` - Array of `custom analyzers <https://www.mongodb.com/docs/atlas/atlas-search/analyzers/custom/>`__
48+
to use in this index.
49+
-
50+
``storedSource`` - Specifies document fields to store for queries performed
51+
using the `returnedStoredSource <https://www.mongodb.com/docs/atlas/atlas-search/return-stored-source/>`__
52+
option. Specify ``true`` to store all fields, ``false`` to store no fields,
53+
or a `document <https://www.mongodb.com/docs/atlas/atlas-search/stored-source-definition/#std-label-fts-stored-source-document>`__
54+
to specify individual fields to include or exclude from storage. Defaults to ``false``.
55+
-
56+
``synonyms`` - Array of `synonym mapping definitions <https://www.mongodb.com/docs/atlas/atlas-search/synonyms/>`__
57+
to use in this index.
58+
59+
Additional documentation for defining search indexes may be found in
60+
`search index definition <https://www.mongodb.com/docs/manual/reference/command/createSearchIndexes/#search-index-definition-syntax>`__
61+
within the MongoDB manual.
62+
63+
Static Mapping
64+
--------------
65+
66+
`Static mapping <https://www.mongodb.com/docs/atlas/atlas-search/define-field-mappings/#static-mappings>`__
67+
can be used to configure indexing of specific fields within a document.
68+
69+
The following example demonstrates how to define a search index using static
70+
mapping.
71+
72+
.. configuration-block::
73+
74+
.. code-block:: php
75+
76+
<?php
77+
78+
/**
79+
* @Document
80+
* @SearchIndex(
81+
* name="usernameAndAddresses",
82+
* fields={
83+
* "username"={
84+
* {"type"="string"},
85+
* {"type"="autocomplete"},
86+
* },
87+
* "addresses"={"type"="embeddedDocuments", "dynamic"=true},
88+
* },
89+
* )
90+
*/
91+
class User
92+
{
93+
/** @Id */
94+
private $id;
95+
96+
/** @Field(type="string") */
97+
private $username;
98+
99+
/** @EmbedMany(targetDocument=Address::class) */
100+
private $addresses;
101+
102+
// ...
103+
}
104+
105+
.. code-block:: xml
106+
107+
<doctrine-mongo-mapping xmlns="http://doctrine-project.org/schemas/orm/doctrine-mongo-mapping"
108+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
109+
xsi:schemaLocation="http://doctrine-project.org/schemas/orm/doctrine-mongo-mapping
110+
http://doctrine-project.org/schemas/orm/doctrine-mongo-mapping.xsd">
111+
112+
<document name="Documents\User">
113+
<search-indexes>
114+
<search-index name="usernameAndAddresses">
115+
<field name="username" type="string" />
116+
<field name="username" type="autocomplete" />
117+
<field name="addresses" type="embeddedDocuments" dynamic="true" />
118+
</search-index>
119+
</search-indexes>
120+
121+
<!-- ... -->
122+
</document>
123+
</doctrine-mongo-mapping>
124+
125+
The ``username`` field will indexed both as a string and for autocompletion.
126+
Since the ``addresses`` field uses an :ref:`embed-many <embed_many>`
127+
relationship, it must be indexed using the ``embeddedDocuments`` type; however,
128+
embedded documents within the array are permitted to use dynamic mapping.
129+
130+
Dynamic Mapping
131+
---------------
132+
133+
`Dynamic mapping <https://www.mongodb.com/docs/atlas/atlas-search/define-field-mappings/#dynamic-mappings>`__
134+
can be used to automatically index fields with
135+
`supported data types <https://www.mongodb.com/docs/atlas/atlas-search/define-field-mappings/#std-label-bson-data-chart>`__
136+
within a document. Dynamically mapped indexes occupy more disk space than
137+
statically mapped indexes and may be less performant; however, they may be
138+
useful if your schema changes or for when experimenting with Atlas Search
139+
140+
.. note::
141+
142+
Atlas Search does **NOT** dynamically index embedded documents contained
143+
within arrays (e.g. :ref:`embed-many <embed_many>` relationships). You must
144+
use static mappings with the `embeddedDocument <https://www.mongodb.com/docs/atlas/atlas-search/field-types/embedded-documents-type/>`__
145+
field type.
146+
147+
The following example demonstrates how to define a search index using dynamic
148+
mapping:
149+
150+
.. configuration-block::
151+
152+
.. code-block:: php
153+
154+
<?php
155+
156+
/**
157+
* @Document
158+
* @SearchIndex(dynamic=true)
159+
*/
160+
class BlogPost
161+
{
162+
/** @Id */
163+
private $id;
164+
165+
/** @Field(type="string") */
166+
private $title;
167+
168+
/** @Field(type="string") */
169+
private $body;
170+
171+
// ...
172+
}
173+
174+
.. code-block:: xml
175+
176+
<doctrine-mongo-mapping xmlns="http://doctrine-project.org/schemas/orm/doctrine-mongo-mapping"
177+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
178+
xsi:schemaLocation="http://doctrine-project.org/schemas/orm/doctrine-mongo-mapping
179+
http://doctrine-project.org/schemas/orm/doctrine-mongo-mapping.xsd">
180+
181+
<document name="Documents\BlogPost">
182+
<search-indexes>
183+
<search-index dynamic="true" />
184+
</search-indexes>
185+
186+
<!-- ... -->
187+
</document>
188+
</doctrine-mongo-mapping>

‎docs/en/sidebar.rst

+1
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@
2424
reference/bidirectional-references
2525
reference/complex-references
2626
reference/indexes
27+
reference/search-indexes
2728
reference/inheritance-mapping
2829
reference/embedded-mapping
2930
reference/trees

‎doctrine-mongo-mapping.xsd

+91
Original file line numberDiff line numberDiff line change
@@ -100,6 +100,7 @@
100100
<xs:element name="lifecycle-callbacks" type="odm:lifecycle-callbacks" minOccurs="0" />
101101
<xs:element name="also-load-methods" type="odm:also-load-methods" minOccurs="0" />
102102
<xs:element name="indexes" type="odm:indexes" minOccurs="0" />
103+
<xs:element name="search-indexes" type="odm:search-indexes" minOccurs="0" />
103104
<xs:element name="shard-key" type="odm:shard-key" minOccurs="0" />
104105
<xs:element name="read-preference" type="odm:read-preference" minOccurs="0" />
105106
<xs:element name="schema-validation" type="odm:schema-validation" minOccurs="0" />
@@ -466,6 +467,96 @@
466467
</xs:choice>
467468
</xs:complexType>
468469

470+
<xs:complexType name="search-indexes">
471+
<xs:choice maxOccurs="unbounded">
472+
<xs:element name="search-index" type="odm:search-index" maxOccurs="unbounded" />
473+
</xs:choice>
474+
</xs:complexType>
475+
476+
<!-- https://www.mongodb.com/docs/manual/reference/command/createSearchIndexes/ -->
477+
<xs:complexType name="search-index">
478+
<xs:choice minOccurs="0" maxOccurs="unbounded">
479+
<xs:element name="field" type="odm:search-index-field" minOccurs="0" maxOccurs="unbounded" />
480+
<!-- Note: custom analyzers are intentionally unsupported in XML -->
481+
<xs:element name="synonym" type="odm:search-index-synonym" minOccurs="0" maxOccurs="unbounded" />
482+
<xs:element name="stored-source" type="odm:search-index-stored-source" minOccurs="0" maxOccurs="1" />
483+
</xs:choice>
484+
485+
<xs:attribute name="name" type="xs:string" />
486+
<xs:attribute name="dynamic" type="xs:boolean" />
487+
<xs:attribute name="analyzer" type="xs:string" />
488+
<xs:attribute name="searchAnalyzer" type="xs:string" />
489+
<xs:attribute name="storedSource" type="xs:boolean" />
490+
</xs:complexType>
491+
492+
<!-- https://www.mongodb.com/docs/atlas/atlas-search/define-field-mappings/ -->
493+
<xs:complexType name="search-index-field">
494+
<xs:choice minOccurs="0" maxOccurs="unbounded">
495+
<!-- https://www.mongodb.com/docs/atlas/atlas-search/field-types/document-type/ -->
496+
<!-- https://www.mongodb.com/docs/atlas/atlas-search/field-types/embedded-documents-type/ -->
497+
<xs:element name="field" type="odm:search-index-field" minOccurs="0" maxOccurs="unbounded" />
498+
<!-- https://www.mongodb.com/docs/atlas/atlas-search/field-types/string-type/ -->
499+
<xs:element name="multi" type="odm:search-index-field" minOccurs="0" maxOccurs="unbounded" />
500+
</xs:choice>
501+
502+
<xs:attribute name="name" type="xs:string" use="required" />
503+
<xs:attribute name="type" type="xs:string" use="required" />
504+
<!-- https://www.mongodb.com/docs/atlas/atlas-search/field-types/autocomplete-type/ -->
505+
<xs:attribute name="maxGrams" type="xs:integer" />
506+
<xs:attribute name="minGrams" type="xs:integer" />
507+
<xs:attribute name="tokenization" type="xs:string" />
508+
<xs:attribute name="foldDiacritics" type="xs:boolean" />
509+
<!-- https://www.mongodb.com/docs/atlas/atlas-search/field-types/document-type/ -->
510+
<!-- https://www.mongodb.com/docs/atlas/atlas-search/field-types/embedded-documents-type/ -->
511+
<xs:attribute name="dynamic" type="xs:boolean" />
512+
<!-- https://www.mongodb.com/docs/atlas/atlas-search/field-types/geo-type/ -->
513+
<xs:attribute name="indexShapes" type="xs:boolean" />
514+
<!-- https://www.mongodb.com/docs/atlas/atlas-search/field-types/knn-vector/ -->
515+
<xs:attribute name="dimensions" type="xs:integer" />
516+
<xs:attribute name="similarity" type="xs:string" />
517+
<!-- https://www.mongodb.com/docs/atlas/atlas-search/field-types/number-type/ -->
518+
<!-- https://www.mongodb.com/docs/atlas/atlas-search/field-types/number-facet-type/ -->
519+
<xs:attribute name="representation" type="xs:string" />
520+
<xs:attribute name="indexIntegers" type="xs:boolean" />
521+
<xs:attribute name="indexDoubles" type="xs:boolean" />
522+
<!-- https://www.mongodb.com/docs/atlas/atlas-search/field-types/string-type/ -->
523+
<xs:attribute name="analyzer" type="xs:string" />
524+
<xs:attribute name="searchAnalyzer" type="xs:string" />
525+
<xs:attribute name="indexOptions" type="xs:string" />
526+
<xs:attribute name="store" type="xs:boolean" />
527+
<xs:attribute name="ignoreAbove" type="xs:integer" />
528+
<xs:attribute name="norms" type="xs:string" />
529+
<!-- https://www.mongodb.com/docs/atlas/atlas-search/field-types/token-type/ -->
530+
<xs:attribute name="normalizer" type="xs:string" />
531+
</xs:complexType>
532+
533+
<!-- https://www.mongodb.com/docs/atlas/atlas-search/synonyms/ -->
534+
<xs:complexType name="search-index-synonym">
535+
<xs:attribute name="name" type="xs:string" use="required" />
536+
<xs:attribute name="analyzer" type="xs:string" use="required" />
537+
<xs:attribute name="sourceCollection" type="xs:string" use="required" />
538+
</xs:complexType>
539+
540+
<!-- https://www.mongodb.com/docs/atlas/atlas-search/stored-source-definition -->
541+
<xs:complexType name="search-index-stored-source">
542+
<xs:choice minOccurs="0" maxOccurs="unbounded">
543+
<xs:element name="field" type="odm:search-index-stored-source-field" minOccurs="0" maxOccurs="unbounded" />
544+
</xs:choice>
545+
546+
<xs:attribute name="type" type="odm:search-index-stored-source-type" use="required" />
547+
</xs:complexType>
548+
549+
<xs:complexType name="search-index-stored-source-field">
550+
<xs:attribute name="name" type="xs:string" use="required" />
551+
</xs:complexType>
552+
553+
<xs:simpleType name="search-index-stored-source-type">
554+
<xs:restriction base="xs:token">
555+
<xs:enumeration value="include" />
556+
<xs:enumeration value="exclude" />
557+
</xs:restriction>
558+
</xs:simpleType>
559+
469560
<xs:complexType name="shard-key">
470561
<xs:choice minOccurs="0" maxOccurs="unbounded">
471562
<xs:element name="key" type="odm:shard-key-key" maxOccurs="unbounded" />

0 commit comments

Comments
 (0)