NoSQL Protocol Module
NoSQL Protocol Module
The nosqlprotocol module allows a MariaDB server or cluster to be
used as the backend of an application using a MongoDB® client library.
Internally, all documents are stored in a table containing two columns;
an id column for the object id and a doc column for the document itself.
When the MongoDB® client application issues MongoDB protocol commands,
either directly or indirectly via the client library, they are transparently
converted into the equivalent SQL and executed against the MariaDB backend.
The MariaDB responses are then in turn converted into the format expected by
the MongoDB® client library and application.
There are a number of with which the behavior
of nosqlprotocol can be adjusted. A minimal configuration looks
like:
nosqlprotocol.user and nosqlprotocol.password specify the
credentials that will be used when accessing the backend database or
cluster. Note that the same credentials will be used for all connecting
MongoDB® clients.
Since nosqlprotocol is a listener, there must be a service to which
the client requests will be sent. Nosqlprotocol places no limitations
on what filters, routers or backends can be used.
To configure the same listener with MaxCtrl, the parameters must be passed in a
JSON object in the following manner:
All the parameters that the nosqlprotocol module takes must be passed in the
same JSON object.
A complete example can be found at the of this document.
Nosqlprotocol supports SCRAM authentication as implemented by MongoDB®.
The mechanisms SCRAM-SHA-1 and SCRAM-SHA-256 are both supported.
If nosqlprotocol has been setup so that no authentication is required, then
when connecting only the host and port should be provided, but neither a
username nor a password.
For instance, if the MongoDB Node.JS Driver is used, then the connection
string should look like:
Similarly, if the Mongo Shell is used, only the host and port should
be provided:
NoSQL and MariaDB Users
A MariaDB user consists of a name and a host part. A user 'user'@'%'
and a user 'user'@'127.0.0.1' are completely different. The host part
specifies where a user may connect from, with % being a wildcard that
matches all hosts. What data a user is allowed to access and modify is
specified by what privileges are granted to the user.
A NoSQL user is somewhat different. It is created in the context of a
particular database, so there may be a user userx in the database dbA
and different user with the same name userx in the database dbB. What
hosts a user may connect from can be restricted, but that is a property of
the user and not an implicit part of it. What data a user is allowed to
access and modify is specified by the roles that have been assigned to
the user.
From the above it should be clear that there is not a 1-to-1
correspondence between the concept of a user in NoSQL and the concept
of a user in MariaDB, but that some additional conventions are needed.
To make it possible to have different NoSQL users with the same name,
the database in whose context the user is created is prepended to the
user name, separated with a dot, when the MariaDB user is created.
This is perhaps easiest to illustrate using an example:
Currently there are two user accounts defined. Even though there is
a user bob, creating a NoSQL user bob succeeds.
If we now, from the MariaDB prompt, check the users we will see:
The MariaDB user corresponding to the NoSQL user bob, created in the
context of the database test, has test as a prefix.
The mariadb database
The fact that NoSQL users have the database embedded in the MariaDB
name may be inconvenient if the same data is accessed both as NoSQL
via nosqlprotocol and as SQL directly from MariaDB. It also makes
it impossible to use an existing MariaDB account from NoSQL.
To provide a solution for this problem, the database mariadb is treated
in a specific fashion. A user created in the context of the mariadb
database is created in the MariaDB server without the database prefix.
If we now try to create a user bob in the mariadb database it will fail,
because the user 'bob'@'%' exists already.
If we create a user with another name it will succeed.
And if we check the situation from MariaDB,
we will see that alice was created without a database prefix.
Roles and Privileges
When creating a user nosqlprotocol accepts all roles as predefined by
MongoDB®, but not all of them are translated into GRANT privileges.
The following table shows what privilege(s) a particular role is
converted to.
The following roles are shorthands for several other roles.
dbOwner differs from root in that the privileges of the former
apply only to a particular database, while the privileges of the
latter apply to all databases. However, the role root can
only be assigned to a user in the admin database.
In addition there are AnyDatabase versions of dbAdmin, read andreadWrite (e.g readAnyDatabase) that can be assigned to a user in
the admin database. If so, then the privilege is granted on *.*,
otherwise on <db>.*.
If the root role is assigned to a user in the admin database,
then the privileges are granted on *.*, otherwise on <db>.*.
Other pre-defined roles are recognized and stored in the local
nosqlprotocol account database, but they do not affect what privileges
are granted to the MariaDB user. Currently user-defined roles are
not supported.
Client Authentication
Authenticationwise nosqlprotocol can be used in three different ways:
If there is an anonymous user on the MariaDB server and if nosqlprotocol
is configured without a user/password, then all nosqlprotocol clients will
access the MariaDB server as anonymous users.
Note that the anonymous MariaDB user is only intended for testing and
should in general not be used, but deleted.
Shared Credentials
If nosqlprotocol is configured with
then each MongoDB® client will use those credentials when accessing the
MariaDB server. Note that from the perspective of the MariaDB server, it
is not possibe to distinguish between different MongoDB® clients.
Unique Credentials
If nosqlprotocol authentication has been taken into use and a MongoDB®
client authenticates, either when connecting or later, then the credentials
of MongoDB® client will be used when accessing the MariaDB server.
Note that even if nosqlprotocol authentication has been enabled, authentication
is not required, and if the MongoDB® client has not authenticated itself, the
credentials specified with nosqlprotocol.[user|password] (or the anonymous
user) will be used when accessing the MariaDB server.
Enforce Authentication
To enforce authentication, specify
in the configuration. If authentication is required, then any command
that requires access to the MariaDB server will fail, unless the client
has authenticated.
By default nosqlprotocol does no authorization. However, a nosqlprotocol
client is always subject to the authorization performed by the MariaDB
server.
When nosqlprotocol authorization is enabled by adding
to the configuration file, some commands will be subject to authorization,
by nosqlprotocol. The following table lists the commands and what role they
require.
It is important to note that even if nosqlprotocol authorization
is enabled, the MariaDB server has the final word. That is, even if the
roles of a user would be sufficient for a particular operation, if the
granted privileges are not, the operation will not succeed. There may
be a mismatch between roles and grants, for instance, if the wrong roles
were specified when the user was added, or if the grants have been
altered directly and not via nosqlprotocol.
Bootstrapping the Authentication/Authorization
The authentication/authorization can be bootstrapped explicitly
or implicitly. Bootstrapping explicitly provides more control, while
bootstrapping implicitly is much more convenient.
Explicit bootstrapping
In order to enable authorization you need to have NoSQL users and
those can be created with or added
with .
If you want to create a user, then you first need to configure
nosqlprotocol with credentials that are sufficient for creating a user:
At this point nosqlprotocol.authentication_required andnosqlprotocol.authorization_enabled should both be false. Note that
as those are their default values, they do not have to be specified.
Start MaxScale and connect to it with the MongoDB® command line client
Then create the user.
Alternatively you can add an existing user. Note that it should be
added to the mariadb database, unless it was created with the
convention of having the database as a prefix, e.g. db.bob.
Now you should shutdown MaxScale and add the entries
and start MaxScale.
The nosqlprotocol.user and nosqlprotocol.password can be removed but
as they will be ignored with nosqlprotocol.authentication_required=true
being present, it is not mandatory.
If you now try to create a user when not having been authenticated or
when authenticated as a user without the userAdmin role, the result
will be:
NOTE When a client authenticates, the password will not be
transferred in cleartext over the network, so, even without SSL,
it is not possible to gain access to a password by monitoring the
network traffic.
However, when a user is created or added (or the password is changed),
the password will be transferred in cleartext. To prevent eavesdropping,
create/add users when connecting over a domain socket, or use
Implicit bootstrapping
With implicit bootstrapping, you should first create the MariaDB
user that should appear as the initial NoSQL user. As explained , the concept of a user is somewhat
different in MariaDB and NoSQL, which means that certain factors
must be taken into account when creating the MariaDB user. Then
at first startup, nosqlprotocol will create the corresponding
NoSQL user, which will enable the authenticated and authorized
use of nosqlprotocol.
When MaxScale is started, if the following hold
nosqlprotocol.authentication_required andnosqlprotocol.authorization_enabled are true in the configuration
section of the nosqlprotocol listener,
nosqlprotocol.user and nosqlprotocol.password are provided, and
then, MaxScale will
wait until the master of the service pointed to by the listener
is available,
connect using the credentials specified in nosqlprotocol.user
and nosqlprotocol.password,
Immediately thereafter it is possible to connect to the nosqlprotocol
port with a MongoDB® client using the specified credentials.
Note that after the bootstrapping, nosqlprotocol will not use
the user and password settings and they can be removed.
Grants
When a NoSQL user is created using
the MariaDB grants are obtained from the specified NoSQL roles
as explained .
When implicitly creating a NoSQL user from an existing user in
MariaDB, the inverse operation must be performed. There are
many factors that affect what NoSQL roles the grants of a user
are translated into:
whether the user is a regular or admin user,
whether the privileges are on *.* or some specific db.*, and
the privileges themselves, e.g. SELECT, DELETE
In NoSQL, every user resides in a specific database. Note that
this does not mean that database would have to exist
in MariaDB.
When it comes to users, the database effectively means a scope,
which in the case of nosqlprotocol is handled by prefixing the
corresponding MariaDB user name with the database/scope name.
When creating a user to be used from NoSQL, there are three
options for the user's name:
The name can be of the format some_db.user_name wheresome_db can be anything (subject to the naming rules of
MariaDB), except admin or mariadb. In this case, the
user will be a regular user, who can access data in
databases that she has been granted access to.
The name can be of the format admin.user_name where admin
What database the privileges can be specified ON depends on
what kind of user is being created.
If it is a regular user, the privileges must be granted on a
specific database, such as \dbA``.*`. Note that there is no
dependency between this database and the (conceptual) database
the user resides in.
If it is an admin user, the privileges must be granted on
the *.* database.
In NoSQL, a role can be database specific or generic. However,
a generic role can only be assigned to a user in the admin
database. In practice this means that if the privileges are
on *.*, then the user must reside in the admin database
(e.g. admin.bob) or it is treated as an error.
The following table shows what privileges are required for a
role to be assigned. Note that ALL PRIVILEGES can be used as
well.
Only required if the user is an admin user.
The AnyDatabase version will be assigned, if the user is
an admin user.
If certain roles are assigned, then other roles will be
assigned as well.
If the roles dbAdmin, readWrite and userAdmin are
assigned, then dbOwner will be assigned as well.
If the roles dbAdminAnyDatabase, readWriteAnyDatabase
and userAdminAnyDatabase
Once the user has been created and the desired privileges have
been granted, the NoSQL listener should be configured as follows:
At MaxScale startup, the NoSQL user will then be created.
Examples
Admin User
We want the initial NoSQL user to be an administrator, with
full rights.
As we want an admin user, the name is prefixed with admin,
which will have that effect. And since it is an admin user, the
privileges are granted ON *.*.
Thereafter, we specify the following in the configuration file,
and start MaxScale.
As the creation of the initial user can be made only after the
monitor for the listener's service has marked one server as master,
whether the creation succeeded or not must be checked from MaxScale'
log file:
Under normal conditions, the bootstrapping will be almost
instantaneous.
It is now possible to connect using any MongoDB® client application.
Note that when connecting the user is passed as nosql_admin and not
as admin.nosql_admin. The fact that we want to authenticate against
the admin database is expressed by passing the database as the last
argument.
As can be seen, the user has the any roles on the admin
database, which means that all databases can be accessed and
modified, and that new users can be created.
Test User
We want the initial NoSQL user to be a user with limited rights,
intended to be used for testing.
As we want a user with limited rights, the name is not prefixed
with admin. The privileges are granted specifically on databasetest.*. Indeed, if *.* had been used, the creation of the initial
NoSQL user would have failed with an error. Here, the user is created
in the same database that the user is given access to, but it could
have been another one. Further, several GRANT statements could have
been used, had we wanted to give access to several databases.
Thereafter, we specify the following in the configuration file,
and start MaxScale.
As the creation of the initial user can be made only after the
monitor for the listener's service has marked one server as master,
whether the creation succeeded or not must be checked from MaxScale'
log file:
Under normal conditions, the bootstrapping will be almost
instantaneous.
It is now possible to connect using any MongoDB® client application.
Note that when connecting the user is passed as test_user and not
as test.test_user. The fact that we want to authenticate against
the test database is expressed by passing the database as the last
argument.
As can be seen, the user has the readWrite role on the test database,
which means that only the test database can be accessed and modified.
Since nosqlprotocol is a regular protocol module used in a listener,
the TLS/SSL support of listeners is available. Please see
for details.
NoSQL Account Database
So as to be able to connect to the MariaDB server on behalf of
clients, nosqlprotocol must know their password. As the password
is not transferred to nosqlprotocol during the authentication in
a way that could be used when logging into MariaDB, the password
must be stored when the user is created with
or added with .
Note that the password is not stored in cleartext but as three
different hashes; hashed with sha1 for use with MariaDB, salted
and hashed with sha1 for use with the SCRAM-SHA-1 authentication
mechanism (if that is enabled for the user) and salted and hashed
with sha256 for use with the SCRAM-SHA-256 authentication mechanism
(if that is enabled for the user).
The account information can be stored privately, in which case it
can be used only by a particular MaxScale instance, or in a_shared_ manner, in which case multiple MaxScale instances can
share the information and a user created/added on one instance
can be used on another.
In the private case, the account information of nosqlprotocol is
stored in an database
whose name is <libdir>/nosqlprotocol/<listener-name>-v1.db,
where <libdir> is the libdir of MaxScale, typically/var/lib/maxscale, <listener-name> is the name of the
listener section in the MaxScale configuration file, and -v1
a suffix for making schema evolution easier, should there be
a need for that.
For instance, given a configuration like
the account information will be stored in the file<libdir>/nosqlprotocol/NoSQL-Listener-v1.db.
Note that since the database name is derived from the listener
name, changing the name of the listener in the configuration
file will have the effect of making all accounts disappear.
To retain the accounts, the database file should also be
renamed.
At first startup, the nosqlprotocol directory and
the file NoSQL-Listener-v1.db will be created. They will
be created with file permissions that only allow MaxScale
access. At subsequent startups the permissions will be checked
and MaxScale will refuse to start if the permissions allow
access to others.
We strongly recommend that no manual modifications are made
to the database.
Note that we make no guarantees that the way in which the
account information is stored by nosqlprotocol will remain the
same even between maintenance releases. We do guarantee,
however, that even if the way in which the account information is
stored changes, existing account information will automatically
be converted and no manual intervention, such as re-creation of
accounts, will be needed.
In the shared case, the account information of nosqlprotocol
is stored in the cluster of the service in front of which the
NoSQL listener resides. The master of the cluster will be used
both for reading and writing data.
A table whose name is the same as the listener's name in the
MaxScale configuration will be created in the database
specified with the
parameter. If it is not specified explicitly, the default isnosqlprotocol. The name of the table will be the name of
the listener section in the MaxScale configuration file.
For instance, given a configuration like
the account information will be stored in the tablenosqlprotocol.NoSQL-Listener.
Note that since the table name is derived from the listener
name, changing the name of the listener in the configuration
file will have the effect of making all accounts disappear.
To retain the accounts, the table should also be renamed.
nosqlprotocol will create the table when needed, so the
user specified with
must have sufficient grants to be able to do that.
nosqlprotocol will store in the table, data that allow
any MaxScale to authenticate a MongoDB® client, irrespective
of which MaxScale instance was used when the user was created.
nosqlprotocol also stores in the table the SHA1 of a user's
password, to be able to authenticate against the MariaDB server.
Therefore it is strongly suggested to enable encryption key
management in MaxScale and to provide an authentication
key ID with so
that the data will be encrypted.
If shared authentication has been enabled with then and must also
be provided. With the
database name can optionally be changed, and with an
encryption key ID, using which the sensitive data is encrypted,
can optionally be provided.
Note that we make no guarantees that the table in which the
account information is stored by nosqlprotocol will remain the
same even between maintenance releases. We do guarantee,
however, that even if the way in which the account information is
stored changes, existing account information will automatically
be converted and no manual intervention, such as re-creation of
accounts, will be needed.
Nosqlprotocol fully supports wire protocol version 6 and only provides
rudimentary support for earlier wire protocol versions, but reports at
startup that it would support versions 0 to 6. The reason is that some
client libraries are buggy and use an old wire protocol version if the
server claims to support only version 6. Consequently, one should use a
client library version that at least supports wire protocol version 6.
As the goal of nosqlprotocol is to implement, to the extent that it
is feasible, the wire protocol and the database commands the way MongoDB®
implements them, it should be possible to use any language specific driver.
However, during the development of nosqlprotocol, the only client library
that has been verified to work is version 3.6 of MongoDB Node.JS Driver.
Using the following parameters, the behavior of nosqlprotocol can be
adjusted. As they are not generic listener parameters, but specific tonosqlprotocol they must be qualified with the nosqlprotocol-prefix.
For instance:
Specifies the user to be used when connecting to the backend, if the MongoDB®
client is not authenticated.
Specifies the password to be used when connecting to the backend, is the MongoDB®
client is not authenticated. Note that the same user/password combination will be
used for all unauthenticated MongoDB® clients connecting to the same listener port.
authentication_required
Specifies whether the client always must authenticate. If authentication is required,
it does not matter whether user and password have been specified, the client must
authenticate.
Authentication should not be required before users have been created with or added with ,
with authentication being optional and authorization being disabled.
NOTE: All client activity is always subject to authorization performed by the
MariaDB server.
authentication_shared
Specifies whether the NoSQL account information should be stored in a shared
manner or privately.
authentication_db
Specifies the database of the table where the NoSQL account information
is stored, if authentication_shared is true. If the database does not
exist, nosqlprotocol will attempt to create it, so either is should be
manually created or the used specified with authentication_user should
have the grants required to do so.
authentication_key_id
The encryption key ID, using which the NoSQL account information should be
encrypted with when stored in the MariaDB server. If an encryption key ID is
given, the encryption key manager in MaxScale must also be enabled.
The encryption key must be a 256-bit key. Keys of shorter length are rejected
as invalid encryption keys.
authentication_user
Mandatory: Yes, if authentication_shared is true.
Specifies the user to be used when modifying and accessing the NoSQL
account information stored in the MariaDB server.
authentication_password
Specifies the password of authentication_user.
authorization_enabled
Specifies whether nosqlprotocol itself should perform authorization in the context
of the commands , and . Authorization should not be enabled before users
have been created with or added with
with authorization being disabled.
NOTE: All client activity is always subject to authorization performed by the
MariaDB server.
Specifies the host to be used when a MariaDB user is created via nosqlprotocol.
By default all users are created as ...@'%', which means that it is possible to
connect to the MariaDB server from any host using the credentials of the created
user. For tighter security, the IP-address of the MaxScale host can be specified.
NOTE: This value does not specify from which host it is allowed to connect to
MaxScale.
on_unknown_command
Values: return_error, return_empty
Specifies what should happen in case a clients sends an unrecognized command.
Enumeration values:
return_error: An error document is returned.
return_empty: An empty document is returned.
log_unknown_command
Specifies whether an unknown command should be logged. This is primarily
for debugging purposes, to find out whether a client uses a command that
currently is not supported.
auto_create_databases
Specifies whether databases should automatically be created, as needed.
Note that setting this parameter to true, without also settingauto_create_tables to true, has no effect at all.
auto_create_tables
Specifies whether tables should automatically be created, as needed.
Note that this applies only if the relevant database already exists.
If a database should also be created if needed, then auto_create_databases
must also be set to true.
Specifies the length of the id column in tables that are automatically created.
ordered_insert_behavior
Enumeration values:
default: Each document is inserted using a separate INSERT, either in a
multi-statement or in a compound statement. Whether an error causes the remaining
insertions to be aborted, depends on the value of ordered specified in the
insert command.
atomic: If the value of ordered in the insert command is true
(the default) then all documents are inserted using a single
What combination of ordered_insert_behavior and ordered (in the insert command
document) is used, has an impact on the performance. Please see the discussion at .
Specifies how long a cursor can be idle, that is, not accessed, before it is
automatically closed.
Values: none, in, out, back
Specifies what should be logged as notice messages.
Enumeration values:
in: The incoming protocol command is logged.
out: The outgoing SQL sent to the backend is logged.
So, specify
to have the incoming command, the corresponding SQL sent to the backend
and the resulting response sent to the client logged.
Databases and Tables
By default, nosqlprotocol automatically creates databases as needed.
The default behavior can be changed by setting auto_create_databases to
false. In that case, databases must manually be created.
Each MongoDB® collection corresponds to a MariaDB table with the same name.
However, it is always possible to access a collection irrespective of whether
the corresponding table exists or not; it will simply appear to be empty.
Inserting documents into a collection, whose corresponding table does not
exist, succeeds, provided auto_create_tables is true, as the table will
in that case be created.
When nosqlprotocol creates a table, it uses a statement like
where the length of the VARCHAR is specified by the value of id_length,
whose default and minimum is 35.
NOTE If the tables are created manually, then the CREATE statement_must_ contain a similar AS-clause as the one above and should contain
a similar constraint.
Note that nosqlprotocol does not in any way verify that the table
corresponding to a collection being accessed or modified does indeed
have the expected columns id and doc of the expected types, but it
simply uses the table, which will fail if the layout is not the expected
one.
To reduce the risk for confusion, the recommendation is to use a specific
database for tables that contain documents.
The following operators are currently supported.
Query and Projection Operators
Comparison Query Operators
Logical Query Operators
Element Query Operators
$type
When $type is used, it will be converted into a condition involving one or more comparisons. The following subset
of types can be used in $type queries:
The "number" alias is supported and will match values whose MariaDB type isDOUBLE or INTEGER.
Evaluation Query Operators
Array Query Operators
$elemMatch
As arguments, only the operators $eq and $ne are supported.
Update Operators
Field Update Operators
Database Commands
The following commands are supported. At each command is specified
what fields are relevant for the command.
All non-listed fields are ignored; their presence or absence have no
impact, unless otherwise explicitly specified.
Aggregation Commands
The following fields are relevant.
The following fields are relevant.
Query and Write Operation Commands
The following fields are relevant.
Each element of the deletes array contains the following fields:
The following fields are relevant.
All other fields are ignored.
Projection
The projection parameter determines which fields are returned in the matching documents.
The projection parameter takes a document of the following form:
If a projection document is not provided or if it is empty, the entire document
will be returned.
Embedded Field Specification
For fields in an embedded documents, the field can be specified using:
dot notation; e.g. "field.nestedfield": <value>
In particular, specifying fields in embedded documents using nested form
is not supported.
_id Field Projection
The _id field is included in the returned documents by default unless you
explicitly specify _id: 0 in the projection to suppress the field.
Inclusion or Exclusion
A projection cannot contain both include and exclude specifications,
with the exception of the _id field:
In projections that explicitly include fields, the _id field is the only field that can be explicitly excluded.
In projections that explicitly excludes fields, the _id field is the only field that can be explicitly include; however, the _id field is included by default.
NOTE Currently _id is the only field that can be excluded, and only
if other fields are explicitly included.NOTE Currently exclusion of other fields but _id is not supported.
Filtering by _id
Note that there is a significant difference between
and
In the former case the generated WHERE clause will be
and in the latter
That is, in the former case the indexed column id will be used, in the
latter it will not.
The following fields are relevant.
All other fields are ignored.
The following fields are relevant.
The following fields are relevant.
The insert command inserts one or more documents into the table whose
name is the same as that of the collection. If the option auto_create_tables
is true, then the table is created if it does not already exist. If the
value is false, then the insert will fail unless the table already exists.
The following fields are relevant.
ordered
The impact of ordered is dependent upon the value of ordered_insert_behavior.
default
In this case ordered has the same impact as in MongoDB®. That is, if the value
is true, then when an insert of a document fails, return without inserting any
remaining documents listed in the inserts array. If false, then when an insert
of a document fails, continue to insert the remaining documents.
atomic
If ordered is true, then all documents will be inserted using a single
INSERT command. That is, if the insertion of any document fails, for instance,
due to a duplicate id, then no document will be inserted. If ordered is false,
then the behavior is identical with that of default.
Performance
What combination of ordered_insert_behavior and ordered is used, has an
impact on the performance.
Of these, atomic + true is the fastest and atomic|default + false the slowest,
being roughly twice as slow. The performance of 'default + true' is halfway between
the two.
The following fields are relevant.
The following fields are relevant.
All other fields are ignored.
Update Statements
Each element of the updates array is an update statement document.
Each document contains the following fields:
Note that currently it is possible to set multi to true in conjunction
with a replacement-style update, even though MongoDB® rejects that.
All other fields are ignored, with the exception of upsert that if present
with the value of true will cause the command to fail.
Behavior
Currently only updating using update operator expressions or with a_replacement document_ is supported. In particular, updating using anaggregation pipeline is not supported.
Update with an Update Operator Expressions document
The update statement field u can accept a document that only contains expressions. For example:
In this case, the update command updates only the corresponding fields in the document.
Update with a Replacement Document
The update statement field u field can accept a replacement document,
i.e. the document contains only field:value expressions. For example:
In this case, the update command replaces the matching document with the update document.
The update command can only replace a single matching document; i.e. the multi field
cannot be true.
Note If the replacement document contains an _id field, it will be ignored and the
document id will remain non-changed while the document otherwise is replaced. This is
different from MongoDB® where the presence of the _id field in the replacement document
causes an error, if the value is not the same as it is in the document being replaced.
Authentication Commands
The following fields are relevant.
If you are not logged in and using authentication, logout has no effect.
Note that in order to be logged out, the logging out must be done while
using the same database that was used when you logged on.
Always returns
User Management Commands
Creates a new MariaDB user and adds an entry to the local nosqlprotocol
account database.
The following fields are relevant.
The MariaDB user will be created as '<db>.<user>'@'%' where <db> is
the name of the NoSQL database in whose context the user is created, and<user> the value of the createUser field. For instance, with the
following command
the MariaDB user 'myDatabase.user1'@'%' will be created.
The elements of the roles array are converted into privileges
as explained in .
In practice the creation is performed as follows:First the MariaDB user is created. Then the privileges are granted.
Finally the local nosqlprotocol account database is updated.
If the granting of privileges fails, an attempt will be made to
drop the user.
dropAllUsersFromDatabase
Drops all users from the local nosqlprotocol account database and
the corresponding MariaDB users.
The following fields are relevant.
If no users can be dropped, e.g. due to an authorization error,
then an error will be returned. If even a single user can be dropped
the returned document tells how many were dropped, which does not
necessarily indicate that all users were dropped.
The following fields are relevant.
The user will first be dropped from the MariaDB server and if
that succeeds also from the local nosqlprotocol account database.
grantRolesToUser
This command adds more roles to a NoSQL user, which may imply
that additional privileges are granted to the corresponding MariaDB
user.
Note that roles assigned to different databases will result in separate
GRANT statements, which means that it is possible that some succeed and
others do not.
revokeRolesFromUser
This command removes roles from an NoSQL user, which may imply
that privileges are revoked from the corresponding MariaDB user.
Note that roles to be removed from different databases will result in
separate REVOKE statements, which means that it is possible that some
succeed and others do not.
This command updates the information about a particular user.
Changes to customData or mechanisism are made only to the local
nosqlprotocol database, but changes to pwd or roles require
the MariaDB server to be updated.
This command returns information about one or more users.
The following fields are relevant.
The returned information depends the valie of usersInfo:
Note that users may always view their own information. Otherwise the user must
have the userAdmin or userAdminAnyDatabase role.
If showCredentials is true, the returned object(s) will contain amariadb: { password: "*..."} field, where password is theSHA1(SHA1()) value of the password used when logging to MariaDB.
That is, the same string that is found in the password column in
the mysql.user table.
Replication Commands
The following fields are relevant.
replSetGetStatus
The following fields are relevant.
All other fields are ignored.
This command will always return the document
Sessions Commands
The following fields are relevant.
The following document will always be returned:
Administration Commands
The following fields are relevant.
Currently, capped collections and views are not supported. Consequently,
specifying that the collection should be capped or that it should be a
view on another collection, will cause the command to fail.
The following fields are relevant.
NOTE Currently it is not possible to create indexes, but the command
will nonetheless return success, provide the index specification passes
some rudimentary sanity checks. Note also that the collection will be
created if it does not exist.
The following fields are relevant.
The following fields are relevant.
The following fields are relevant.
NOTE Currently it is not possible to create indexes and thus there
will never be any indexes that could be dropped. However, provided the
specfied collection exists, dropping indexes will always succeed except
for an attempt to drop the built-in _id_ index.
The following fields are relevant.
The response will always be
The following fields are relevant.
listCollections
The following fields are relevant.
Note that the command lists all collections (that is, tables) that are found
in the current database. The listed collections may or may not be suitable
for being accessed using nosqlprotocol.
The following fields are relevant.
The following fields are relevant.
NOTE As it currently is not possible to actually create indexes,
although an attempt to do so using createIndexes will succeed, the
result will always only contain information about the built-in
index _id_.
renameCollection
The following fields are relevant.
The following fields are relevant.
Any kind of parameter is accepted and the response will always be:
Diagnostic Commands
The following fields are relevant.
The command returns a document containing the stable fields. In addition, there is a field maxscale whose value is the MaxScale version, expressed as a string.
The following fields are relevant.
The command will return a document of the expected layout, but the content is only rudimentary.
The following fields are relevant.
The following fields are relevant.
The command returns a document of the correct format, but no actual log data will be returned.
The following fields are relevant.
The following fields are relevant.
The following fields are relevant.
The following fields are relevant.
The following fields are relevant.
The command does not actually perform any validation but for checking
that the collection exists. The response will contain in nrecords
the current number of documents/rows it contains.
The following fields are relevant.
This is an internal command, implemented only because the Mongo Shell uses it.
Free Monitoring Commands
getFreeMonitoringStatus
The following fields are relevant.
The following document will always be returned:
MaxScale Specific Commands
Definition
mxsAddUser
The mxsAddUser command adds an existing MariaDB user to the local
nosqlprotocol account database. Use if the
MariaDB user should be created as well.
Note that the mxsAddUser command does not check that the user exists
or that the specified roles are compatible with the grants of the user.
Syntax
The 'mxsAddUser' command has the following syntax:
Command Fields
The command has the following fields:
The value of mxsAddUser should be the name (without the host part) of
an existing user in the MariaDB server and the value of pwd should be
that user's password in cleartext.
The roles array should contain roles that a compatible with the
grants of the user. Please check
for a discussion on how to map roles map to grants.
Returns
If the addition of the user succeeds, the command returns a document
with the single field ok whose value is 1.
If there is a failure of some kind, the command returns an error document
mxsCreateDatabase
Definition
mxsCreateDatabase
The 'mxsCreateDatabase' command creates a new database and must be run
against the admin database.
Syntax
The 'mxsCreateDatabase' has the following syntax:
Command Fields
The command takes the following fields:
Returns
If database creation succeeds, the command returns a document with the
single field ok whose value is 1.
If the database creation fails, the command returns an error document.
Definition
mxsDiagnose
The mxsDiagnose command provides diagnostics for any other command; that is, how
MaxScale will handle that command.
Syntax
The mxsDiagnose command has the following syntax:
Command Fields
The command takes the following fields:
Returns
The command returns a document that contains diagnostics of the command
provided as argument. For example:
kind specifies of what kind the command is; an immediate command is one for
which MaxScale autonomously can generate the response, a single command is one
where the command will cause a single SQL statement to be sent to the backend, and
a multi command is one where potentially multiple SQL statements will be sent to
the backend.
If the command is immediate then there will be a field response containing
the actual response of the command, if the command is single then there will be
a field sql containing the actual statement that would have been sent to the backend,
and if the command is multi then there will be a field sql containing an array
of statements that would have been sent to the backend.
If an error occurs while the command is being diagnosed, then there will be noresponse field but an error field whose value is an error document. Note that
the value of ok will always be 1.
Definition
mxsGetConfig
The mxsGetConfig command returns the current configuration of the session
and must be run against the 'admin' database.
Syntax
The mxsGetConfig has the following syntax:
Command Fields
The command takes the following fields:
Returns
The command returns a document that contains the current configuration of
the session. For example:
Definition
mxsRemoveUser
The mxsRemoveUser removes a user from the local nosqlprotocol account
database. Use if the MariaDB user should be dropped
as well.
Syntax
The 'mxsRemoveUser' command has the following syntax:
Command Fields
The command has the following fields:
Returns
If the removal of the user succeeds, the command returns a document
with the single field ok whose value is 1.
If there is a failure of some kind, the command returns an error document
Definition
mxsSetConfig
The mxsSetConfig command changes the configuration of the session
and must be run against the 'admin' database.
Note that the changes only affect the current session and are not
persisted.
Syntax
The mxsSetConfig has the following syntax:
Command Fields
The command takes the following fields:
The document takes the following fields:
Returns
The command returns a document that contains the changed configuration of
the session. For example:
Definition
mxsUpdateUser
The mxsUpdateUser command updates a user in the local nosqlprotocol
account database. Use to update MariaDB user
as well.
Note that the mxsUpdateUser command does not check that the changed
data is compatible e.g. with the grants of the corresponding MariaDB
user.
Syntax
The 'mxsUpdateUser' command has the following syntax:
Command Fields
The command has the following fields:
The roles array should contain roles that a compatible with the
grants of the user. Please check
for a discussion on how to map roles map to grants.
Returns
If the updating of the user succeeds, the command returns a document
with the single field ok whose value is 1.
If there is a failure of some kind, the command returns an error document
When a document is created, an id of type ObjectId will be autogenerated by
the MongoDB® client library. If the id is provided explicitly, by assigning a
value to the _id field, the value must be an ObjectId, a string or an
integer.
Currently 30% of the tests in the
test-suite pass.
The following is a minimal setup for getting nosqlprotocol up and
running. It is assumed the reader knows how to configure MaxScale for
normal use. If not, please start with the .
Note that as nosqlprotocol is the first component in the MaxScale
routing chain, it can be used with all routers and filters.
Configuring MaxScale
In the following it is assumed that MaxScale already has been configured
for normal use and that there exists a service [TheService].
The values the_user and the_password must be replaced with the
actual credentials to be used for every MongoDB® client that connects.
If MaxScale is now started, the following entry should appear in the
log file.
MongoDB® Shell
The mongo Shell is a powerful tool with which to access and manipulate a
MongoDB database. It is part of the MongoDB® package. Having the native
MongoDB database installed is convenient, as it makes it easy to ascertain
whether a problem is due to nosqlprotocol not fully implementing something
or due to the API not being used in the correct fashion.
With the mongo shell, all that is needed is to invoke it with the port_nosqlprotocol_ is listening on:
If the shell prompt appears, then a connection was successfully
established and the shell can be used.
The db variable is implicitly available, and refers by default to
the test database.
The command inserted a document into the collection called collection.
The table corresponding to that collection is created implicitly because
the default value of auto_create_tables is true. Here, the object id
is specified explicitly, but there is no need for that, as one will be
created if needed.
To check whether the documents was inserted into the collection, thefind command can be issued:
As can be seen, the document was indeed inserted into the collection
With the mysql shell, the content of the actual table can be checked.
The collection collection is represented by a table collection with
the two colums id and doc. id is a virtual column whose content is
the value of the _id field of the document in the doc column.
All MongoDB® commands that mongdbprotocol support (but for the ones that
do not require database access), basically access or manipulate the
content in the doc column using the of MariaDB.
From within the mongo shell itself it is easy to find out just what SQL
a particular MongoDB command is translated into.
For instance, the SQL that the insert command with which the document was
added can be found out like:
Similarily, the SQL of the find command can be find out like:
The returned SQL can be directly pasted at the mysql prompt, which is
quite convenient in case the MongoDB® command does not behave as expected.
MongoDB® Node.JS Driver
As all client libraries implement and depend on the MongoDB® wire protocol,
all client libraries should work with nosqlprotocol. However, the
only client library that has been used and that has been verified to work
is version 3.6 of the MongoDB Node.JS Driver.
In principle, the only thing that needs to be altered in an existing
program using the library is to change the uri string that typically
is something like
to
with the assumption that the default nosqlprotocol port is used.
In practice, additional modifications may be needed since nosqlprotocol
does not implement all commands and does not in all cases implement the
full functionality of the commands that it supports.
Inserting a Document
Store the following into a file called insert.js.
Then, run the program like
As the id is not explicitly provided, it will not be the same.
Finding a Document
Store the following into a file called find.js.
Then, run the program like
CC BY-SA / Gnu FDL