Endpoints

Magic contains a whole range of endpoints, or “middleware” parts, that the system itself relies upon to function. You can play around with these endpoints by using the “Endpoints” menu item in your dashboard, ensure you show your system endpoints, while filtering on “magic/system”. Most of these endpoints are for internal use through the Magic dashboard, and should as a general rule of thumb not be consumed directly by you - But some of these endpoints are useful for things such as implementing authentication and authorisation in your own code. Below is a screenshot of how Magic allows you to browse and try your endpoints through its “Endpoints” menu item.

Magic endpoints menu item

Notice, all endpoints that requires authorization of some sort assumes a valid JWT token is transmitted in the Authorization HTTP header as a “Bearer” type of token, and if not, the user will not be allowed to invoke the endpoint, and an HTTP status code of 401 will be returned. To retrieve a JWT token use the magic/system/auth/authenticate endpoint documented further down in this document.

Authentication and authorisation endpoints

These are the endpoints related to the authentication and authorisation parts of Magic. You can find their Hyperlambda files in the “/system/auth/” folder. These endpoints are typically useful for you as you implement your own authentication logic in your own code, and most of these endpoints are intended for you to consume yourself as you see fit.

GET magic/system/auth/authenticate

This endpoint allows you to authenticate towards your Magic backend with a username and password combination. It’s mostly a thin layer on top of your [magic.auth.authenticate] slot that you can read about further down in this document. It requires two query arguments being as follows.

The endpoint can be invoked by anyone, and does not have any authorisation requirements. The endpoint will return a JWT token you can use for consecutive requests towards your backend, authorising you to invoke endpoints your user is authorised to invoking. Notice, Magic relies upon the JWT tokens being transmitted as “Bearer” tokens in the “Authorization” HTTP header, implying you’ll have to ensure the resulting JWT token from invoking the above endpoint is attached to your HTTP requests as such in later requests. This endpoint can be invoked by anyone, including non-authenticated clients.

This endpoints is intended for you to consume from your own code.

GET magic/system/auth/auto-auth

This endpoint always returns “off” unless you’ve configured Magic to use automatic LDAP authentication by following this recipe. The idea with this endpoint is to use the frontend to check if automatic authentication has been turned on, and if so, simply invoking the authenticate endpoint directly without any username or password combination, which works only if you’ve enabled LDAP authentication, and your frontend is deployed on a local domain where LDAP credentials are transmitted to the backend. The endpoint does not require any arguments. The endpoint can be invoked by anyone, and does not have any authorisation requirements. The endpoint does not require any arguments.

Notice, this endpoint is considered obsolete and might be removed in future versions of Magic.

PUT magic/system/auth/change-password

This endpoint allows an existing user to change his or her password. It can only change the password of the currently authenticated user, and takes the new password as the following payload. This endpoint can be invoked by anyone as long as you have authenticated towards your backend previously. This endpoint can also be invoked with a “change password” type of JWT token, generated by the backend if the caller forgot his or her password, and requested to change his or her password somehow.

{
  "password": "new-password"
}

This endpoints is intended for you to consume from your own code.

GET magic/system/auth/endpoints

This endpoint returns the authorisation requirements for all endpoints in the system. It does not require any arguments. It returns one item for each Hyperlambda endpoint in the system, with its associated verb, and a list of roles the user must belong to in order to invoke the endpoint, if the endpoint requires authorisation. The endpoint can be invoked by anyone, and does not have any authorisation requirements itself. Notice, this endpoint caches its result for 5 minutes, implying changes done to the authorisation requirements of your endpoints will not be accessible for clients before 5 minutes after your changes have been applied, unless you explicitly delete the cache item for the endpoint.

This endpoints is intended for you to consume from your own code.

GET magic/system/auth/impersonate

This endpoint allows you to generate a JWT token on behalf of another user, by providing a username allowing you to impersonate the other user in the system. The endpoint requires the following argument(s).

This endpoint can only be invoked by a root user. This endpoint is useful when debugging user related errors, and/or supporting/helping users who for some reasons needs help, allowing you to use the system within the context of the user you impersonate.

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/auth/reset-password

This endpoint allows you to generate a reset password JWT token on behalf of another user, by providing a username, returning a “reset password” JWT token that can only be used to reset the user’s password. The endpoint requires the following argument(s).

This endpoint can only be invoked by a root user. However there exists another endpoint you can see further down in this document allowing a user to have Magic send him a “forgot password email”.

Notice - This endpoint is not intended for you to consume in your own code.

PUT magic/system/auth/imprison

This endpoint allows you to “imprison” a user in your system. When a user is imprisoned, the user cannot login to the system for as long as the user is “imprisoned”. This endpoint can only be invoked by a root user, and the endpoint relies upon scheduled tasks to release users imprisoned, so the endpoint will not function correctly unless scheduled tasks are enabled in the system. The endpoint takes the following JSON payload.

{
  "username": "user-to-imprison",
  "releaseDate": "2022-01-31T05:44:52.439Z"
}

The release date is the date when the user should be released from his “imprisonment”. This endpoint is considered obsolete and might be removed in a future version of Magic.

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/auth/refresh-ticket

This endpoint allows an already authenticated user to retrieve a new JWT token with an expiration date further into the future. The idea of the endpoint is to allow for an authenticated user to non-stop constantly invoke this endpoint some few minutes before his existing JWT token expires, to retrieve a new JWT token, preventing the user from being thrown out of the backend as his or her existing token expires. The endpoint does not take any arguments, but can only be invoked by an already authenticated user, implying you’ll need to pass in your JWT token to it in the Authorization HTTP header as a “Bearer” token or the endpoint will return “Access denied”.

This endpoints is intended for you to consume from your own code.

POST magic/system/auth/register

This endpoint allows a user to register in your backend, which implies creating a new user. The endpoint requires the username to be a valid email address. If you want users to confirm their email address as they register, you’ll have to configure your SMTP settings. Configuring your SMTP settings can be done by reading the following parts. In addition you need to provide a frontendUrl to the endpoint that will be used by the email template as a “confirm email address link” is sent to the user to dynamically create a link the user must click to confirm his or her email address. The endpoint requires the payload below to be transmitted by the client. Notice, the frontendUrl is only used if the system has been configured with an SMTP server, resulting in that the endpoint invocation tries to send the registered user an email to have the user confirm his or her email address. If you don’t care about having users confirm their email address, and/or haven’t configured SMTP settings, you don’t have to supply the frontendUrl argument.

When a user registers using this endpoint, a new user will be created, but the user will be added to the “unconfirmed” role, resulting in that the user have no access to any parts of the system at all before his or her email address has been confirmed. When his or her email address is confirmed, the “unconfirmed” user/role association is deleted, and the user is added to the “guest” role. The guest role still doesn’t have access to much of the system, so if the user is to gain access to more of the system, this must be manually added by a root user after the user has confirmed his or her email address.

{
  "username": "john@doe.com",
  "password": "some-password",
  "frontendUrl": "https://your-frontend-url.com",
  "template": "/some/path/to-some-email-template.html",
  "subject": "Subject line of registration email",
  "extra": {
    "name": "John Doe"
  }
}

The above arguments implies the following.

The [extra] field is optional and if not supplied will be ignored. This field declares additional extra information associated with the created user, and can be anything you wish, such as for instance “phone”, “name”, etc. Notice, what type of extra information the system accepts is configured as a comma separated list in your “appsettings.json” file, and can be edited in the “Config” menu item, by adding a key named “extra” beneath your “registration” configuration object. By default the only accepted extra information you can associate with users is “name”, but if you for instance want to allow users to provide their phone number as they are registering in addition to their names, you can modify this section as follows.

{
  "magic": {
    "auth": {
      "registration": {
        "extra": "name,phone"
      }
    }
  }
}

Whatever extra information you pass in to this endpoint during registration, will be associated with the user and returned as the user authenticates.

This endpoint does not require the caller to be authenticated and can be invoked by anyone. If you have configured the system to send “confirm email address” emails, the URL transmitted to the user to confirm his or her email address will resemble the following, except the query parameters will be URL encoded of course.

https://your-frontend-url.com?
token=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855&
username=john@doe.com&
url=https://your-backend-url.com

The above arguments in the URL sent to the user’s email address implies the following.

This endpoint allows you to create a frontend page somewhere where you accept the above token query parameter, the username query parameter, and the url query parameter, for then to invoke the verify-email endpoint found below to verify the user’s email address, passing in the token value found in your query parameters above as the token argument to the verify-email endpoint. Notice, Magic’s dashboard will correctly handle these URL arguments if you supply the root-URL/domain to your Magic dashboard as your frontendUrl when invoking the endpoint, but this will of course result in the user being brought onwards to the Magic dashboard’s frontend, which might not be what you wish for in your own apps. The “register” endpoint can return one of the following values if it successfully executes.

You can override the email template used to send user a “verify email address” email. The default email template used can be found at /system/auth/email-templates/register.html. However, if you create your own email template you’ll have to make sure you keep the dynamically substituted url parts in your own custom template. See how the href parameter for the default email template is constructed to understand how this works.

This endpoints is intended for you to consume from your own code.

POST magic/system/auth/verify-email

This endpoint allows a registered user to verify his or her email address, and is typically occurring after a user has registered and the system has sent the user an email to confirm his or her email address. The endpoint does not require authorisation, but takes the following payload.

{
  "username": "john@doe.com",
  "token": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
}

Notice, the above token must be the same token generated by Magic as the user registered, and sent to the user on his or her email.

This endpoints is intended for you to consume from your own code.

POST magic/system/auth/send-reset-password-link

This endpoint sends a “reset password email” to a registered user. Like the register endpoint, the endpoint requires a frontendUrl field, in addition to of course a username. The username must be an email address. This endpoint requires that you have configured Magic’s SMTP settings such that it can send an email to the user allowing him or her to reset the password. Below is an example payload.

{
  "username": "john@doe.com",
  "frontendUrl": "https://your-frontend-url.com",
  "template": "/some/path/to-some-email-template.html",
  "subject": "Subject line of change password email"
}

The above arguments implies the following.

The endpoint does not require the caller to be authenticated and can be invoked by anyone, but the endpoint can only be invoked for a user who’s username is a valid email address.

This endpoints is intended for you to consume from your own code.

GET magic/system/auth/verify-ticket

This endpoint allows a frontend to verify an existing JWT token, and it will return “success” if the JWT token is valid and can be used in consecutive invocations requiring authorisation somehow. The endpoint can be invoked by any user as long as the user is authenticated. The endpoint does not take any arguments, but can only be invoked by an already authenticated user, implying you’ll need to pass in your JWT token to it in the Authorization HTTP header as a “Bearer” token. If the JWT token is not valid the endpoint will return a 401 status code with “Access denied” as its message.

This endpoints is intended for you to consume from your own code.

Bazar endpoints

These are endpoints related to the Bazar somehow, allowing you to install new Bazar modules, and check which Bazar modules have already been installed. The Bazar is a micro service “AppStore” for your Magic server, allowing your Magic server to install backend modules on the fly, resulting in having some backend micro service installed into your backend, without interrupting normal usage.

Notice - None of these endpoints are really intended to be consumed by your own code, but only for internal usage by Magic itself.

GET magic/system/bazar/app-manifests

This endpoint returns a list of Bazar apps that have been installed in your backend. Each item will contain a date of installation, the username of the user who installed the app, the name and version of the app, etc. Most importantly each Bazar manifest item will also contain a “token”, which is typically required as the app is to be updated. Below is an example of what the endpoint might return if it only has one Bazar item installed. The endpoint can only be invoked by a root user and it requires no arguments.

[
  {
    "name": "Babel Fish",
    "version": "v1.0.0",
    "token": "3b65f514-2e30-4ca2-bdcf-96a2d2727bc1",
    "module_name": "babelfish",
    "installed_by": "root",
    "installed": "2022-01-31T06:03:01.669Z"
  }
]

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/bazar/can-install

This endpoint requires a Magic version number, such as “v10.0.15”, and returns either “SUCCESS” or “ERROR”, depending upon whether or not the current version of Magic is above or beneath the specified version number. It is typically used to verify that a Bazar app can be installed, to verify the Magic version is high enough for the Bazar item to be successfully installed. The endpoint can only be invoked by a root user. The endpoint requires the following argument(s).

This endpoint is considered obsolete and might be removed in a future version of Magic.

Notice - This endpoint is not intended for you to consume in your own code.

POST magic/system/bazar/download-from-bazar

This endpoint downloads a Bazar item as a ZIP file from the specified URL, and unzips the file into the backend. The endpoint can only be invoked by a root user and requires the following payload.

{
  "url": "https://some-bazar.com/download?app=some-app.zip",
  "name": "some-module"
}

The Bazar item will be downloaded from the specified URL and assumed to be a ZIP file. The ZIP file will then be unzipped, and unzipped into your “modules” folder as the specified “name”. Notice, any previously installed modules with the same name will be automatically deleted, and the endpoint will not install the module, but only unzip it and create its folder structure. To install the module after it has been unzipped make sure you invoke the /system/file-system/install endpoint.

Notice, this endpoint is considered obsolete and will be changed in a future version of Magic.

Notice - This endpoint is not intended for you to consume in your own code.

POST magic/system/bazar/download-from-url

This endpoint downloads a file from its specified [url] and saves it to the specified [folder] in your backend. The endpoint can only be invoked by a root user, and requires the following payload.

{
  "folder": "foo",
  "url": "foo"
}

The above arguments implies the following.

The [url] field needs to be a valid URL returning a file, with the Content-Disposition HTTP header correctly applied, since the Content-Disposition header’s “filename” value becomes the name of the file on your server.

Notice - This endpoint is not intended for you to consume in your own code.

These are the endpoints related to your server side cache, and allows you to query your cache, purge your cache entirely, or delete a single cache item on the server. Internally Magic is using server-side cache sparsingly for some expensive operations, such as retrieving meta data from your database. Sometimes you need to manually purge your cache, and/or delete individual cache items on your server, to re-retrieve data invalidating your cache in the process. This section provides information about such operations for such scenarios.

Notice - None of these endpoints are really intended to be consumed in your own code, but only for internal usage by Magic itself.

GET magic/system/cache/list

This endpoint returns the number of cache items on your server, optionally allowing you to page with the following query parameters.

The endpoint can only be invoked by a root user.

Notice - This endpoint is to be considered obsolete and might change in a future version of Magic.

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/cache/count

This endpoint returns the number of cache items in total on your server, optionally starting out with the “filter” query parameter as their key. The endpoint can only be invoked by a root user. The endpoint requires the following argument(s).

Notice - This endpoint is to be considered obsolete and might change in a future version of Magic.

Notice - This endpoint is not intended for you to consume in your own code.

DELETE magic/system/cache/delete

This endpoint requires an [id] as a query parameter, and will delete the associated server side cache item. It can only be invoked by a root user. The endpoint requires the following argument(s).

Notice - This endpoint is to be considered obsolete and might change in a future version of Magic.

Notice - This endpoint is not intended for you to consume in your own code.

DELETE magic/system/cache/empty

This endpoint completely empties your server side cache, optionally taking a “filter” query parameter, which if specified will ensure only cache items starting with the specified filter key will be deleted. The endpoint can only be invoked by a root user. The endpoint requires the following argument(s).

Notice - This endpoint is to be considered obsolete and might change in a future version of Magic.

Notice - This endpoint is not intended for you to consume in your own code.

These endpoints are related to configuration of your system, and allows you to read and modify configuration settings as you see fit. These endpoints are typically not intended for being used directly by your code, but for the most parts only used during initialisation of your Magic server, and/or as you change configuration settings in Magic.

Notice - None of these endpoints are really intended to be consumed in your own code, but only for internal usage by Magic itself.

GET magic/system/config/load

This endpoint loads your configuration settings and returns it to the caller. The endpoint can only be invoked by a root user and does not require any arguments.

Notice - This endpoint is not intended for you to consume in your own code.

POST magic/system/config/save

This endpoint allows you to save your configuration settings. The specified paylod will in its entirety overwrite your existing “appsettings.json” file. The endpoint can only be invoked by a root user. The endpoint can accept any type of JSON payload, however whatever you supply to the endpoint will in its entirety be persisted in your backend as your “appsettings.json” file, implying if you supply bogus data, your backend will no longer function correctly.

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/config/status

This endpoint returns the setup status of your system, implying if the system has been correctly initialised. The endpoint requires no argument. The response object will resemble the following.

{
  "config_done": true,
  "magic_crudified": true,
  "server_keypair": true
}

The fields in the above payload implies the following.

If any of the above fields does not return true, the frontend dashboard will guide you through the process of finishing the configuration process before allowing you to gain access to other parts of your Magic server. The endpoint can only be invoked by a root user.

Notice - This endpoint is not intended for you to consume in your own code.

POST magic/system/config/setup

This endpoint sets up your system, and requires the following payload.

{
  "password": "yyy",
  "settings": "*"
}

The above fields implies the following.

The endpoint can only be invoked by a root user, and will setup Magic to use the specified connection string found in your [settings] field, verify the database server exists and can be connected to, create the magic database if not existing from before, and create a root user account in the users table for you with the specified [password].

The endpoint can only be invoked once and will throw an exception if the system has previously been setup - Which Magic determines by checking the auth:secret value of your existing “appsettings.json” file, and if it’s not the default value of “THIS-IS-NOT-A-GOOD-SECRET-PLEASE-CHANGE-IT”, Magic will assume it’s been previously setup and throw an exception.

If the magic database already exists in your database server, it will only run its migration scripts, and replace the existing root user’s password with whatever you provided as a payload to the endpoint invocation. This implies that if you forget your root user’s password the easiest way to change it is in fact to manually change the auth:secret back to its original value which was “THIS-IS-NOT-A-GOOD-SECRET-PLEASE-CHANGE-IT”, for then to log out and login again, which will allow you to run through the configuration process again, overwriting your root user’s existing password.

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/config/version-compare

This endpoint requires two version numbers and return -1, 0 or 1 depending upon which of the two versions comes before the other one. It’s useful for validating the backend is at least at some specific version number or higher, before for instance installing modules and such requiring a specific version. The versions it expects to perform this comparison must be in the form of vn.n.n, e.g. “v10.14.33”. Refer to the [version] slot further down in this document to understand how a Magic version provides semantic meaning with its version entities. The query parameters the endpoint requires are as follows.

If [version_1] is lower than [version_2] the endpoint will return -1. If the versions are the same the endpoint will return 0, if [version_1] is higher it will return 1.

Notice - This endpoint is considered obsolete and might be removed in future versions of Magic.

Notice - This endpoint is not intended for you to consume in your own code.

These endpoints allows you to among other things generate CRUD HTTP backend modules, and is at the heart of the “CRUDifier” in Magic. In addition these endpoints allows you to generate SQL based HTTP endpoints, and/or frontends using a template of some sort. These endpoints are not intended for being directly invoked by your code, but rather from the Magic dashboard as it is creating HTTP backend endpoints for you, and/or frontends based upon your backend.

Backend CRUD operations

This section describes the parts that generates backend HTTP endpoints for you, typically wrapping a database table with CRUD operations.

POST magic/system/crudifier/crudify

This endpoint creates one CRUD HTTP backend endpoint for you, wrapping the specified database table with the specified HTTP verb/CRUD-operation. The endpoint requires the following payload.

{
  "verbose": true,
  "join": true,
  "databaseType": "xxx",
  "moduleName": "xxx",
  "database": "xxx",
  "table": "xxx",
  "moduleUrl": "xxx",
  "returnId": true,
  "template": "xxx",
  "verb": "xxx",
  "auth": "xxx",
  "log": "xxx",
  "overwrite": true,
  "validators": "xxx",
  "cache": 42,
  "publicCache": true,
  "cqrs": true,
  "cqrsAuthorisation": "xxx",
  "cqrsAuthorisationValues": "xxx",
  "args": "*",
  "conditions": "*"
}

The above fields implies the following.

The CRUD endpoints to HTTP verb mappings this endpoint generates are as follows.

This endpoint can only be invoked by a root user. This endpoint is to be considered obsolete and will probably change in a future version of Magic.

Notice - This endpoint is not intended for you to consume in your own code.

POST magic/system/crudifier/custom-sql

This endpoint creates an SQL based HTTP backend endpoint for you. It’s similar to the above endpoint, but much simpler in syntax. It requires the following payload.

{
  "databaseType": "xxx",
  "authorization": "xxx",
  "moduleName": "xxx",
  "database": "xxx",
  "arguments": "xxx",
  "verb": "xxx",
  "endpointName": "xxx",
  "sql": "xxx",
  "overwrite": true,
  "isList": true
}

The arguments implies the following.

The endpoint can only be invoked by a root user.

Notice - This endpoint is not intended for you to consume in your own code.

Frontend generator

This part describes the frontend generator allowing you to create some sort of frontend, wrapping your previously created backend endpoints.

GET magic/system/crudifier/templates

This endpoint returns all frontend templates the system can use for generating frontends. It requires no arguments, but can only be invoked by a root user. This is used to allow the user to select a frontend template as he or she wants to generate a frontend of some sort.

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/crudifier/template

This endpoint returns Markdown describing the specified template with a humanly understandable description, implying it returns the README.md file associated with the template specified as the [name] query parameter. As the user is selecting a template, typically displaying the result of invoking this endpoint makes sense, since it allows the user to understand what the template does for him or her. The endpoint can only be invoked by a root user. The endpoint requires the following query argument(s).

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/crudifier/template-args

This endpoint returns the additional custom arguments the specified frontend template can handle. It requires one query parameter being [name] of template you wish to retrieve arguments for. The return value of invocations to this endpoint becomes the [args] collection for invocations to the above endpoint. The endpoint can only be invoked by a root user. The endpoint requires the following query argument(s).

Notice - This endpoint is not intended for you to consume in your own code.

POST magic/system/crudifier/generate-frontend

This endpoint generates a frontend based upon a template, and either puts the result into a local folder on the server, or returns as a ZIP file download to the client. The endpoint requires the following payload.

{
  "templateName": "xxx",
  "apiUrl": "xxx",
  "frontendUrl": "xxx",
  "email": "xxx",
  "name": "xxx",
  "copyright": "xxx",
  "endpoints": "*",
  "deployLocally": true,
  "args": "*"
}

The above arguments implies the following.

The endpoint can only be invoked by a root user.

Notice - This endpoint is not intended for you to consume in your own code.

These are endpoints related to cryptography somehow, implying among other things the cryptographically secured HTTP lambda implementation. To understand this feature read the following article that describes them from a high level perspective. However, the basic foundation for this feature is to allow for cryptographically signing a Hyperlambda payload, for then to transmit the payload to another Magic server, for then to have that Magic server securely execute the specified Hyperlambda, without risking compromising the server in any ways. In addition this section also describes how to associate a user with a cryptography key pair, allowing users to sign in with zero username/password requirements, etc.

PUT magic/system/crypto/associate-user

Associates a specified public cryptography key with a specified user. The endpoints requires the following payload.

{
  "keyId": 42,
  "username": "foo"
}

The [keyId] is the ID of the public cryptography key, and the [username] is the username of the user you want to associate with the public key. Invoking this endpoint allows the specified user to authenticate towards the backend using a “cryptography challenge”, implying zero username/password, allowing the user to authenticate using his private cryptography key instead, by giving the client a “cryptography challenge” the client is assumed to be able to cryptographically sign using his or her own private key.

Notice - This endpoint is not intended for you to consume in your own code.

PUT magic/system/crypto/deassociate-user

The opposite of the above endpoint, allowing a root user to de-associate a user with the specified public key. Payload to endpoint is as follows.

{
  "keyId": 42
}

Whatever users are associated with the above public key will be de-associated with their existing key association, and no longer be able to use cryptography challenges to authenticate towards the backend.

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/crypto/challenge

Returns a new cryptography challenge to the client, allowing him to use it to cryptographically sign the specified challenge, submit the cryptographically signed result to another endpoint, to generate a JWT token that authorises the client on behalf of the user associated with the public key. The challenge is is valid for 300 seconds, implying the client needs to be able to cryptographically sign the result, and submit the cryptographically signed result to the coupled endpoint within 5 minutes to be able to authenticate as the user associated with his cryptography key. The endpoint can be invoked by anyone, and does not require any argument(s).

This endpoints is intended for you to consume from your own code.

POST magic/system/crypto/challenge

Coupled with the above endpoint, this endpoint allows the user to use his private cryptography key to authenticate towards the backend without having to supply his or her username/password combination. After having cryptographically signed the challenge returned from the above endpoint, the user can submit the signed challenge to this endpoint, resulting in a JWT token being returned, allowing the user to invoke endpoints afterwards within the context of the user associated with the cryptography key pair.

If the invocation to this endpoint is successful, a valid JWT token will be returned, allowing the client to use this JWT token to invoke other endpoints authorised as the user associated with the public cryptography key on the server using the JWT token. The endpoint requires the following payload.

{
  "challenge": "cryptographically-signed-challenge"
}

The above argument implies the following.

This endpoints is intended for you to consume from your own code.

PATCH magic/system/crypto/eval-id

Executes a cryptographically secured lambda invocation. See cryptographically secured lambda invocations to understand what this implies. The payload is assumed to be binary in the form of application/octet-stream, assumed to be a cryptographically signed Hyperlambda snippet, and only after the signature has been confirmed to belong to a public key in the database being enabled and allowed to create such HTTP invocations, the Hyperlambda will be executed within the [whitelist] declaration associated with the key the payload was signed with.

Notice - This endpoint is not intended for you to consume in your own code, but rather used indirectly through the [magic.crypto.http.eval] slot.

POST magic/system/crypto/generate-keypair

This endpoint generates a new server cryptography key pair, both its public key and its private key, and takes a backup of the old key, making the newly generated key the default server key. The endpoint requires the following payload.

{
  "strength": 4096,
  "seed": "qwertyuiop",
  "subject": "John Doe",
  "email": "john@doe.com",
  "domain": "https://johndoe.com"
}

The above arguments implies the following.

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/crypto/get-fingerprint

This endpoint returns the fingerprint for the specified public key. The public key must be base64 encoded DER format. The endpoint requires one query parameter being its [key], which is the key the invocation will return the fingerprint for. The endpoint can only be invoked by a root user and it requires the following argument(s).

Notice - This endpoint is not intended for you to consume in your own code.

POST magic/system/crypto/import

This endpoint allows any user to import his or her public key to the backend. By default the key will not be enabled, but disabled, and needs to be enabled by an administrator before it can be used to create cryptographically secured lambda invocations. The payload for the endpoint is as follows.

{
  "subject": "xxx",
  "email": "xxx",
  "domain": "xxx",
  "content": "xxx"
}

The above arguments implies the following.

Notice, currently Magic only supports RSA keys base64 encoded in DER format.

This endpoints is intended for you to consume from your own code.

GET magic/system/crypto/public-key-raw

Returns the server’s public key as base64 encoded DER format. This endpoint can be invoked by anyone, allowing others to import your server’s public key into their own database. The endpoint requires no arguments.

This endpoints is intended for you to consume from your own code.

GET magic/system/crypto/public-key

This endpoint is similar to the above endpoint, but instead of returning the public key of your server in raw base64 encoded DER format, it returns the following response.

{
  "fingerprint": "d090-3322-fcfa-c064-27e3-fd45-d5bc-1035-5c3f-13c8-7c53-1d03-ecb7-ad71-2ea2-cce1",
  "publicKey": "MIIBIjANBgkq... snip ...3QIDAQAB",
  "keyFormat": "base64/DER",
  "fingerprintFormat": "SHA256-fingerprint",
  "keyType": "RSA"
}

The above fields implies the following.

This endpoint can be invoked by anyone and does not require authentication. The endpoint requires no argument(s).

This endpoints is intended for you to consume from your own code.

GET magic/system/crypto/user-association

This endpoint returns the username associated with the specified public cryptography key ID. The endpoint can only be invoked by a root account, and requires [keyId] as a query parameter, being the ID of the public key as taken from the crypto_keys table from your magic database. The endpoint requires the following query arguments(s).

Notice - This endpoint is not intended for you to consume in your own code.

Diagnostic endpoints

These endpoints are diagnostic endpoints, allowing you to diagnose your backend, and/or run assumptions to sanity check your current installation. An assumption is a high level integration test specifically created to verify that some HTTP endpoint returns what is “assumed” given a specific payload of some sort. Notice, assumptions can also be manually ceated as lambda objects, to execute arbitrary Hyperlambda code, throwing exceptions if assumptions are incorrect.

POST magic/system/diagnostics/create-test

This endpoint creates a new assumption test. The endpoint can only be invoked by a root user and requires the following payload.

{
  "filename": "foo",
  "verb": "foo",
  "url": "foo",
  "status": 42,
  "description": "foo",
  "payload": "foo",
  "response": "foo",
  "produces": "foo"
}

The above arguments implies the following.

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/diagnostics/execute-test

This endpoint executes the specified assumption test, given through its [root_url] and [test_file] query parameters. The endpoint can only be invoked by a root user. The endpoint requires the following query argument(s).

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/diagnostics/system-information

This endpoint returns system related information to the caller, such as Magic backend version, number of tasks, log items count, etc. The endpoint can only be invoked by a root user. The endpoint does not require any arguments. The endpoint can only be invoked by a root user.

Notice - This endpoint is not intended for you to consume in your own code.

These endpoints are related to the file system somehow and allows you to upload and download files, create and delete folders, etc in your Magic backend. These endpoints are arguably the foundation of Hyper IDE, allowing you to edit your files on your server. Most of these endpoints can only be invoked by a root user, and you would typically not consume these endpoints yourself directly from your own frontends.

GET magic/system/file-system/download-folder

This endpoint allows the caller to download the specified folder from the backend as a ZIP file. The endpoint can only be invoked by a root account, and requires the caller to specify [folder] being an existing folder in the Magic backend. The endpoint requires the following query argument(s).

Notice - This endpoint is not intended for you to consume in your own code.

DELETE magic/system/file-system/file

This endpoints deletes the specified [file] in the backend. The endpoint can only be invoked by a root user. The endpoint requires the following query argument(s).

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/file-system/file

This endpoint returns the specified [file] to the caller. The endpoint can only be invoked by a root user. The endpoint requires the following query argument(s).

Notice - This endpoint is not intended for you to consume in your own code.

PUT magic/system/file-system/file

This endpoint creates or updates an existing file on the server. The endpoint requires its payload as multipart/form-data, where [folder] becomes the folder of where to save the file, and the attached [file] becomes the file to save, assuming the file has a name specified as a part of its MIME entity as its Content-Disposition header’s filename. The endpoint can only be invoked by a root user.

Notice - This endpoint is not intended for you to consume in your own code.

DELETE magic/system/file-system/folder

This endpoint deletes the specified [folder] in your backend. The endpoint can only be invoked by a root user. The endpoint requires the following query argument(s).

Notice - This endpoint is not intended for you to consume in your own code.

PUT magic/system/file-system/folder

This endpoint creates a new folder in your backend given the specified payload.

{
  "folder": "/foo/bar/"
}

The endpoint can only be invoked by a root user.

Notice - This endpoint is not intended for you to consume in your own code.

PUT magic/system/file-system/install

This endpoint installs a new module already assumed to be unzipped somehow inside your modules folder, by recursively executing all files found beneath “magic.startup” folders within your module’s primary folder, and/or within folders directly within the main folder of your module’s folder. The endpoint requires the following payload.

{
  "folder": "/modules/bar/",
  "app_version": "v15.0.11",
  "name": "bar",
  "token": "xyz"
}

The above arguments implies the following.

The endpoint can only be invoked by a root user.

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/file-system/list-folders-recursively

This endpoint lists all folders recursively starting from the specified [folder] argument. The endpoint can only be invoked by a root user. The endpoint requires the following query argument(s).

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/file-system/list-files-recursively

This endpoint returns all files recursively from within the specified [folder] folder. The endpoint can only be invoked by a root user. The endpoint requires the following query argument(s).

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/file-system/list-files

This endpoint lists all files within the specified [folder] folder, optionally containing the specified [filter] criteria. The endpoint can only be invoked by a root user. The endpoint requires the following argument(s).

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/file-system/list-folders

This endpoint lists all folders within the specified [folder] folder. The endpoint can only be invoked by a root user. The endpoint requires the following argument(s).

Notice - This endpoint is not intended for you to consume in your own code.

POST magic/system/file-system/rename

This endpoint renames a file or a folder. It requires the following payload.

{
  "oldName": "xxx",
  "newName": "yyy"
}

Notice, the endpoint can rename both files or folders, but can only be invoked by a root user. The [oldName] is the full path of the file’s or folder’s current name. The [newName] is its new full name or path.

Notice - This endpoint is not intended for you to consume in your own code.

PUT magic/system/file-system/unzip

This endpoint unzips the specified [file] ZIP file in place. The endpoint requires the following payload.

{
  "file": "/foo/bar.zip"
}

The endpoint can only be invoked by a root user.

Notice - This endpoint is not intended for you to consume in your own code.

These are log related endpoints, allowing you to retrieve information related to your log. By default your log items are persisted in the magic database table called log_entries, but there exists alternative log service implementations allowing you to among other things persist log entries into a NoSQL/CQL based database of your choice, such as ScyllaDB or Cassandra. What features the log has varies from which service is used for persisting log items.

GET magic/system/log/count

This endpoint counts the total number of log items in your backend. The endpoint can only be invoked by a root user.

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/log/get

This endpoint returns the specified [id] log item, including its meta data. The endpoint can only be invoked by a root user. The endpoint requires the following query argument(s).

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/log/list

This endpoint returns log items from your backend, and can optionally allow you to page with the following query parameters.

Notice - This endpoint is not intended for you to consume in your own code.

POST magic/system/log/log-loc

This endpoint allows you to log lines of code generated by either some sort of frontend scaffolding process, or CRUD backend process. The endpoint can only be invoked by a root user, and requires the following payload.

{
  "loc": 42,
  "type": "backend|frontend",
  "name": "Name of module/frontend"
}

The above arguments implies the following.

Notice - This endpoint is not intended for you to consume in your own code.

POST magic/system/log/log

This endpoint allows you to log some arbitrary log item, and takes the following payload.

{
  "type": "foo",
  "content": "foo",
  "meta": {
    "foo1": "bar1",
    "foo2": "bar2",
    "foo3": "bar3"
  }
}

The above arguments implies the following.

The endpoint can be invoked by any user but requires the user to be authenticated towards the backend and transmit his JWT token in the Authorization HTTP header as a “Bearer” token.

This endpoints is intended for you to consume from your own code.

These endpoints allows you to retrieve meta data about your databases, and/or connection strings, in addition to execute SQL towards a specific database, etc. These endpoints are the foundation for the “SQL” menu item in the Magic dashboard.

GET magic/system/sql/connection-strings

This endpoint returns all database connection strings existing in the backend, being of type [databaseType], where the type can be one of ‘mysql’, ‘mssql’ or ‘pgsql’. The endpoint can only be invoked by a root user.

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/sql/databases

This endpoint returns meta data associated with the specified [databaseType] and [connectionString] combination, such as all databases, all tables within each database, all columns, and any foreign keys existing between columns. The endpoint can only be invoked by a root user. The endpoint requires the following query argument(s).

Notice, the endpoint is fairly expensive to execute, and therefor its result is cached on the server with the key of “magic.sql.databases.xxx.yyy” where “xxx” is the database type and “yyy” is the connection string name.

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/sql/default-database-type

This endpoint returns your default database type, in addition to all databases types the system currently supports. An example of invoking the endpoint can be found below.

{
  "options": [
    "mssql",
    "mysql",
    "pgsql"
  ],
  "default": "mysql"
}

The above result implies that MySQL is the default database type, while the system still has support for both ‘mssql’ and ‘pgsql” in addition to ‘mysql’. The endpoint can only be invoked by a root user.

Notice - This endpoint is not intended for you to consume in your own code.

POST magic/system/sql/evaluate

This endpoint allows you to evaluate the specified SQL towards your database. The endpoint requires the following payload.

{
  "databaseType": "foo",
  "database": "foo",
  "sql": "foo",
  "safeMode": true,
  "batch": true
}

The above arguments implies the following.

The endpoint can only be invoked by a root user.

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/sql/list-files

This endpoint lists all SQL files associated with the specified [databaseType] query parameter. These are template files, stored by the user or distributed by the system by default. The endpoint can only be invoked by a root user. The endpoint requires no argument(s).

PUT magic/system/sql/save-file

This endpoint saves a template SQL snippet file in your backend, and requires the following payload.

{
  "databaseType": "foo",
  "filename": "foo",
  "content": "foo"
}

The above arguments implies the following.

Your SQL snippet files are saved in your backend inside of your “/etc/xxx/templates/” folder, where “xxx” is your database type such as for instance ‘pgsql’, ‘mysql’ or ‘mssql’.

Notice - This endpoint is not intended for you to consume in your own code.

These are endpoints related to administrating tasks, and/or due dates for executing tasks. To understand the task scheduler and how it works please refer to the following article.

GET magic/system/tasks/list

This endpoint returns a list of all your tasks, optionally matching the specified query parameters.

All the above arguments are optional, and the endpoint can only be invoked by a root user. Notice, this endpoint should be considered obsolete and might change in a future version of Magic.

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/tasks/get

This endpoint returns the declaration for the specified [name] task, where name is the ID or name of your task. This endpoint can only be invoked by a root user, and it requires the following query argument(s).

Notice, this endpoint should be considered obsolete and will change in a future version of Magic.

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/tasks/count

Returns the number of tasks in your system, optionally matching the specified [filter] filtering argument. This endpoint can only be invoked by a root user. The endpoint requires the following argument(s).

Notice, this endpoint should be considered obsolete and might change in a future version.

Notice - This endpoint is not intended for you to consume in your own code.

POST magic/system/tasks/create

This endpoint creates a new task in your backend. The endpoints requires the following payload.

{
  "id": "foo",
  "description": "foo",
  "hyperlambda": "foo"
}

The above arguments implies the following.

This endpoint can only be invoked by a root user.

Notice - This endpoint is not intended for you to consume in your own code.

POST magic/system/tasks/update

This endpoint updates an existing task, and requires the following payload.

{
  "id": "foo",
  "description": "foo",
  "hyperlambda": "foo"
}

The above arguments are the same as when creating a new task and implies the following.

Notice - This endpoint is not intended for you to consume in your own code.

DELETE magic/system/tasks/delete

This endpoint deletes a previously persisted task with the specified [id]. This endpoint can only be invoked by a root user. The endpoint requires the following query argument(s).

Notice - This endpoint is not intended for you to consume in your own code.

POST magic/system/tasks/due/add

This endpoints adds a due date, or a repetition pattern, to a previously persisted task. It requires the following payload.

{
  "id": "foo",
  "due": "2022-01-31T11:16:09.492Z",
  "repeats": "foo"
}

The above arguments implies the following.

To understand how the [repetition] works see the documentation for magic.lambda.scheduler. This endpoint can only be invoked by a root user.

Notice - This endpoint is not intended for you to consume in your own code.

DELETE magic/system/tasks/due/delete

This endpoint deletes the specified [id] due/repeats instance in your backend. This endpoint can only be invoked by a root user. The endpoint requires the following query argument(s).

Notice - This endpoint is not intended for you to consume in your own code.

These are terminal related endpoints, and/or socket connection points. Magic has support for subscribing to web socket messages using SignalR, in addition to executing Hyperlambda files over the same socket connection. This part of the documentation describes the endpoints related to such socket operations in regards to the integrated “Terminal” component. The terminal component again allows for executing terminal or bash commands in your backend, and see the result of the command in your frontend.

SOCKET magic/system/terminal/command

This socket URL allows you to transmit a command to a previously opened terminal instance on your server. The socket endpoints can only be invoked by a root user. The endpoints requires the following payload.

{
  "cmd": "cd",
  "channel": "xyz"
}

The [cmd] above is a terminal or bash command, such as ls or mkdir - While the [channel] above is the name of a previously created channel in your backend. To create a unique channel you can use the gibberish endpoint. This endpoint can only be invoked by a root user.

Notice - This endpoint is not intended for you to consume in your own code.

SOCKET magic/system/terminal/start

This socket URL allows you to create a new terminal instance on your server, allowing you to transmit bash command over. The socket endpoint requires the following payload.

{
  "channel": "foo",
  "folder": "/modules/"
}

The above [channel] becomes a unique reference for future commands transmitted to your server, while the [folder] becomes the initial default folder from where to run your terminal within on your server. This endpoint can only be invoked by a root user.

Notice - This endpoint is not intended for you to consume in your own code.

SOCKET magic/system/terminal/stop

This socket URL allows you to stop a previously created terminal instance on your server. It requires the following payload.

{
  "channel": "foo"
}

The above [channel] is a channel previously created using the start socket endpoint. This endpoint can only be invoked by a root user.

Notice - This endpoint is not intended for you to consume in your own code.

Misc endpoints

These are endpoints not belonging to a particular category of endpoints, but still a part of the Magic middleware, and used by the frontend dashboard somehow.

GET magic/system/emails/email

This endpoint returns the email address and full name of the root user. The endpoint can only be invoked by a root user.

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/diagnostics/assumptions

This endpoint returns all assumptions associated with the specified [verb] [endpoint] combination. This endpoint can only be invoked by a root user. The endpoint requires the following query argument(s).

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/endpoints/list

This endpoint returns all endpoints in the system, their meta data such as description, input arguments, and resulting response if possible. The endpoint can only be invoked by a root user. The endpoints requires no argument(s).

Notice - This endpoint is not intended for you to consume in your own code.

POST magic/system/evaluator/evaluate

This endpoint executes the specified Hyperlambda and returns the result of the invocation back to caller. The endpoint can only be invoked by a root user and takes the following payload.

{
  "hyperlambda": "/* ... Some Hyperlambda here ... */"
}

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/evaluator/vocabulary

This endpoint returns the Hyperlambda vocabulary from the backend, implying which Hyperlambda slots exists on the server. The endpoint can only be invoked by a root user.

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/ide/macro

This endpoint returns a Hyper IDE macro definition to caller, allowing the caller to see which arguments the macro can handle, and other meta data associated with the macro. Typically the macro will return something resembling the following.

{
  "name": "Creates a download file endpoint",
  "description": "Macro for Hyper IDE that creates Markdown documentation for a module's endpoints, and stores the documentation in an ENDPOINTS.md file at root of the module's folder.",
  "arguments": [
    {
      "name": "module",
      "type": "string",
      "description": "Name of module to create a 'download file endpoint' for",
      "mandatory": true
    }
  ]
}

The endpoint can only be invoked by a root user.

Notice - This endpoint is not intended for you to consume in your own code.

POST magic/system/ide/execute-macro

This endpoint executes the specified Hyper IDE macro with the specified arguments. The endpoint requires the following payload.

{
  "macro": "/full-path-of-macro.hl",
  "args": "*"
}

The [macro] is the full filename of the macro to execute, while the [args] depends upon the macro itself, and differs according to what macro is being executed. See the above endpoint for how to retrieve arguments related to a specific macro. The endpoint can only be invoked by a root user.

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/images/generate-qr

This endpoint generates a QR code, and returns to the caller as Content-Type of image/png. The endpoint can be invoked by anyone and does not require authentication to be invoked. The endpoint requires the following query parameters.

This endpoints is intended for you to consume from your own code.

GET magic/system/misc/gibberish

This endpoint returns cryptographically secured random characters to the caller, optionally taking [min] and [max] as query parameters, being the minimum length and maximum length of the returned “gibberish”. With gibberish we imply CSRNG characters. This endpoint can be invoked by anyone and requries the following argument(s).

This endpoints is intended for you to consume from your own code.

POST magic/system/sockets/publish

This endpoint allows you to publish a socket message. The endpoint requires the following payload.

{
  "client": "foo",
  "roles": "foo",
  "groups": "foo",
  "name": "foo",
  "message": "*"
}

The endpoint can only be invoked by a root user. The above arguments implies the following.

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/sockets/count-users

Returns number of currently connected socket users, having subscribed to one or more messages from the backend. The endpoint optionally takes a [filter] query parameter, allowing you to filter users returned by it. The endpoint can only be invoked by a root user. The endpoint requires the following argument(s).

Notice - This endpoint is not intended for you to consume in your own code.

GET magic/system/sockets/list-users

Returns the currently connected socket users form the backend, optionally matching the optional [filter] argument. The endpoint can only be invoked by a root user. The endpoint requires the following argument(s).

Notice - This endpoint is not intended for you to consume in your own code.