Dynamic Hyperlambda slots

Magic creates the following dynamic slots during startup. Notice, most of these slots are there exclusively to make sure the middleware of Magic works correctly, and are not intended to be used directly by you in your own Hyperlambda code, unless explicitly stated otherwise, and/or you’re extending Magic or replacing parts of its core with your own custom logic.

These are slots related to authentication and authorisation, and allows you to authenticate, impersonate, change password of users, and other parts related to the “auth” parts of the system. Notice, you would rarely if ever use these slots directly, since the system contains several endpoints helping you out with the consumption of this logic. However, to be complete, we’ve chosen to document these none the less.

[magic.auth.authenticate]

This slot authenticates a user towards your specific authentication and authorisation logic, which implies looking up the username in the magic.users database table, and ensuring the password is correct. If both the username exists, and the password is correct, the slot will return a valid JWT token, allowing you to use it as a token authorising you to act on behalf of the user, and/or return the JWT token to the client. Below is an example assuming your root password is “admin”. Try to execute the following code with your own root user’s password to see how it works.

signal:magic.auth.authenticate
   username:root
   password:admin

The above would return something resembling the following.

signal
   ticket:eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.xxx

Notice, the authentication and authorisation logic in Magic is extendible, and allows you to use for instance LDAP or Windows SSO authentication. To understand how to apply this refer to this documentation. The slot can also be given two optional arguments.

If you invoke the slot with the [check-password] argument being boolean false, you do not need to supply a password, and in fact the password will simply be ignored if you do. This allows you to “impersonate” a user without knowing the user’s password. Below is an example.

signal:magic.auth.authenticate
   username:root
   check-password:bool:false

If you execute the above Hyperlambda, you will see the slot succeeds, and returns a valid JWT token for your root user, even though you did not supply a password to it. This obviously has security consequences, and you should never in any ways what so ever expose this slot invocation to a user whom is not already authenticated as “root” in your backend. However, for a root user, it allows for “impersonating” another user, to for instance debug, and/or view sites, within the context of another user, by creating a JWT token wrapping some other user, and use this as if the root user was that other user - Which is useful sometimes for helping users whom are stuck with something and need help, and/or debug issues related to specific users in your backend. The magic/system/auth/generate-token endpoint is using this slot to generate JWT token allowing you to impersonate users in your backend.

The [reset-password] argument generates a “reset password” token, that can only be used to reset a user’s password, and is typically quite useful for “reset password forms” that users can use to reset their own passwords, by sending a reset password link to the user’s registered email address, etc. Notice, this argument must be accompanied by a [check-password] argument being set to false. Below is an example of usage.

signal:magic.auth.authenticate
   username:root
   check-password:bool:false
   reset-password:bool:true

The above will result in a valid JWT token - However, if you run the token through for instance JWT.io, you will notice the user only belongs to the “reset-password” role, in addition to that it contains a hashed version of the user’s existing password. This role is an extremely restricted role and can only be used to change the user’s password by invoking the [magic.auth.change-password] slot, which is done by the magic/system/auth/change-password endpoint. The reason why the old password is sent in hashed format, is to prevent the same reset password JWT token from being used twice, which would obviously be a security concern. Notice, only use the [rest-password] logic for functions that for instance sends a reset password link on email to the user’s registered email address, or something similar, to prevent malicious users from high jacking other user accounts.

Magic contains endpoints wrapping all of the above slots, so you would typically never need to manually use these slots yourself, unless you’re building your own custom authentication system on top of Magic.

[magic.auth.change-password]

Allows a user to change his or her password. This slot requires the user to be already authenticated and having a valid JWT token. An example of changing your own password to “admin2” can be found below.

signal:magic.auth.change-password
   password:admin2

If you invoke the above Hyperlambda, and log out from Magic, you’ll need to use “admin2” as your password to authenticate again. Notice, this slot will evaluate within the context of the “current user”, implying the only person who can change his or her password is the user currently being authenticated. To allow for having a user whom have forgotten his or her password to change his or her password, see the [magic.auth.authenticate] slot above, and specifically the [reset-password] argument, allowing you to generate a “reset password” type of JWT token that you can send to the registered user in for instance an email or something, to allow for users to reset their passwords when they have forgotten their current passwords.

Yet again, Magic contains helper endpoints for these types of operations, and you would probably never want to directly invoke the slots yourself, but rather use the helper endpoints from your own frontend if you wish to use this type of functionality. See the endpoints section further down in this document for details.

[magic.auth.create-user]

This slot allows you to create a new user in Magic, with the specified username/password combination, belonging to the specified role(s). Below is an example of usage.

signal:magic.auth.create-user
   username:foo
   password:bar
   roles
      .:guest

The above creates a new user called “foo” with the password of “bar” belonging to the “guest” role. If you execute the above Hyperlambda for then to go to your “Auth” menu item, you will find this user. Notice, you might want to delete the user after having verified it exists.

[magic.auth.ensure-role]

This slot allows you to ensure that the specified role exists, and if not, create it. This is a useful slot when you create your own Hyperlambda modules, and your module depends upon some role existing in the system. Typically you’d invoke this slot from one of your module’s startup files, since these are executed both during startup and during installation of your module. Below is an example of usage.

signal:magic.auth.ensure-role
   role:foo-role
   description:This is the foo role

If you execute the above Hyperlambda and open up your “Auth” menu item afterwards, you will see how you now have a “foo-role” in your system.

These slots are related to cryptography usage in your system, and most of these are specifically related to cryptographically secured lambda invocations, allowing you to transmit and execute Hyperlambda from your server to some other server, having your Hyperlambda securely executed on the destination server. Yet again you would rarely if ever consume (most of) these slots directly yourself, but rather use them indirectly. To understand cryptographically secured lambda invocations you can refer to this part of our documentation.

[magic.crypto.get-public-key]

This slot returns a registered public key from your public cryptography key database, in addition to its domain, vocabulary access right, etc. Below is an example of usage. Notice, you’ll have to change the [fingerprint] of the invocation below to a key that actually exists in your system, which you can do by opening up your “Crypto” menu item and for instance copy and paste the fingerprint of your own server’s key.

signal:magic.crypto.get-public-key
   fingerprint:d090-3322-fcfa-c064-27e3-fd45-d5bc-1035-5c3f-13c8-7c53-1d03-ecb7-ad71-2ea2-cce1

The above would result in something resembling the following.

signal
   vocabulary
      add
      return
      get-nodes
      vocabulary
      slots.vocabulary
   id:int:3
   public_key:MIIBIjANBxyz
   domain:"http://localhost:5000"
   email:info@aista.com
   enabled:bool:true

To understand the structure and purpose of the above properties of a public key refer to the following documentation.

[magic.crypto.get-server-private-key]

This slot returns your server’s private cryptography key, allowing you to use it for cryptography operations, such as for instance encrypting or signing some package intended for being sent to some recipient. Below is example usage.

signal:magic.crypto.get-server-private-key

The above wold return something resembling the following.

signal
   private-key:MIIEvAIBxyz....rwKw==
   fingerprint:d090-3322-fcfa-c064-27e3-fd45-d5bc-1035-5c3f-13c8-7c53-1d03-ecb7-ad71-2ea2-cce1

Notice, your server’s private key needs to stay private, and you should be very careful with it, since among other things it allows others to impersonate your server’s root account, and also execute arbitrary Hyperlambda on your server. To understand the structure and purpose of your server’s private key refer to the following documentation.

[magic.crypto.get-server-public-key]

This slot returns your server’s public key, and contrary to the private key, this key should preferably be shared with as many recipients as possible, since it among other things allows others to encrypt data transmitted to your server, and/or verify that data originates from your server, and not some malicious “main in the middle”. Example usage can be found below.

signal:magic.crypto.get-server-public-key

The above wold result in something resembling the following.

signal
   key:MIIBIjANBxyz.....
   fingerprint:d090-3322-fcfa-c064-27e3-fd45-d5bc-1035-5c3f-13c8-7c53-1d03-ecb7-ad71-2ea2-cce1

To understand the structure and purpose of your server’s public key refer to the following documentation.

[magic.crypto.http.eval]

This slots allows you to cryptographically secured transmit Hyperlambda from your server to another Magic server, and have your Hyperlambda securely executed on the server of your choice, assuming the other party have given you the rights to execute your Hyperlambda object. Example usage can be found below.

guid.new
unwrap:x:+/**/.request-id
signal:magic.crypto.http.eval
   url:"http://localhost:5000/magic/system/crypto/eval-id"
   .lambda
      .request-id:x:@guid.new
      vocabulary
      slots.vocabulary
      add:x:./*/return
         get-nodes:x:@vocabulary/*
         get-nodes:x:@slots.vocabulary/*
      return

The above would result in something resembling the following.

signal:int:200
   headers
      Date:"Sun, 30 Jan 2022 07:48:36 GMT"
      Server:Kestrel
      Content-Length:351
      Content-Type:application/octet-stream
   content:@":return
:slots.vocabulary
:get-nodes
:vocabulary
:add
"

To understand the structure and purpose of the above Hyperlambda refer to the following documentation.

[magic.crypto.http.has-invoked]

This slot allows you to determine if a cryptographically secured lambda invocation has been executed before. It requires a [request-id] only, and returns true or false depending upon whether or not a receipt of the request has been persisted into your magic.crypto_invocations database table. The purpose of the slot is to avoid “replay attacks” of cryptographically secured lambda invocations. To understand the structure and purpose of the above slot refer to the following documentation.

[magic.crypto.http.persist-invocation]

This slot persists a cryptographically secured lambda invocation, resulting in a receipt, preventing the same payload to be persisted twice. As most of the other related slots, it’s not intended to be directly consumed, but rather indirectly used by the other cryptographic slots that allows you to execute lambda objects that are cryptographically signed. But it takes the following arguments.

[magic.crypto.import-key]

This slot allows you to import a public cryptography key into your key database. It requires the following arguments.

By default this inserts a new key and associates it with the following slots.

Database meta traversal

All the following slots are a part of the database meta system, and allows you to retrieve meta data about your databases, such as database names, table names, columns names, and foreign keys for tables. These slots are heavily relied upon by the CRUDifier, as it creates a Hyperlambda backend for you, in order to determine which arguments your HTTP endpoints requires, and how to validate these during HTTP invocations towards your backend.

All these have at least 3 versions, one for MySQL, one for PostgreSQL, and a third for SQL Server, and their only difference is that the MySQL versions use “mysql” as its slot name, SQL Server uses “mssql” and PostgreSQL uses “pgsql”. Besides from this, they’re identical in regards to their APIs, which is the point, since it allows for these slots to be used transparently, or “polymorphistically” towards any underlaying database type. In the following section, we therefor document their MySQL version, implying you’ll have to exchange the “mysql” parts to test these towards a different database type.

[magic.db.mysql.databases]

This slot returns all databases the system has access to within the specified connection string. Example usage can be found below.

signal:magic.db.mysql.databases
   connection-string:generic

The above would result in a list of databases resembling the following.

signal
   ""
      db:information_schema
   ""
      db:magic
   ""
      db:mysql
   ""
      db:performance_schema
   ""
      db:sys

[magic.db.mysql.tables]

This slot returns all tables for the specified [connection-string]/[database] combination. Below is example usage.

signal:magic.db.mysql.tables
   connection-string:generic
   database:magic

The above would return all tables in your “magic” database, resulting in something resembling the following.

signal
   ""
      table:crypto_invocations
   ""
      table:crypto_keys
   ""
      table:log_entries
   ""
      table:magic_version
   ""
      table:roles
   ""
      table:task_due
   ""
      table:tasks
   ""
      table:users
   ""
      table:users_crypto_keys
   ""
      table:users_roles

[magic.db.mysql.columns]

This slot returns all columns for the specified [connection-string]/[database]/[table] combination. Below is example usage.

signal:magic.db.mysql.columns
   connection-string:generic
   database:magic
   table:users_roles

The above would return all columns in your “users_roles” table, resulting in something resembling the following.

signal
   ""
      name:role
      db:varchar(45)
      nullable:bool:false
      primary:bool:true
      automatic:bool:false
      hl:string
   ""
      name:user
      db:varchar(256)
      nullable:bool:false
      primary:bool:true
      automatic:bool:false
      hl:string

The fields implies the following.

[magic.db.mysql.foreign_keys]

This slot returns all foreign keys for the specified [connection-string]/[database]/[table] combination. Below is example usage.

signal:magic.db.mysql.foreign_keys
   connection-string:generic
   database:magic
   table:users_roles

The above would return all foreign keys in your “users_roles” table, resulting in something resembling the following.

signal
   ""
      column:role
      foreign_table:roles
      foreign_column:name
   ""
      column:user
      foreign_table:users
      foreign_column:username

In the above result we can see how the “role” column in the “users_roles” table for instance has a foreign key declaration pointing towards the “name” column in the “roles” table, and similarly for the “user” column pointing towards the “users” table’s “username” column. The resulting values implies the following.

Misc slots

These are slots that don’t belong to a specific category solving some problem related to your applications.

[magic.emails.send]

This slot allows you to send a templated email, where the email’s content can be found in a template file in your file system, allowing you to provide substitution values that are “braided” into the email before it’s being sent. The slot takes the following arguments.

Notice, this slot requires you to have configured your SMTP settings. To see how to do this refer to the magic.lambda.mail project. The [substitutes] arguments is a key/value collection matching {{key}} values in your template, replacing the content by the value of your key. If you’ve got a template such as the following.

Hello {{name}}, …

You can invoke it with the following Hyperlambda.

signal:magic.emails.send
   substitutes
      name:John Doe

The above will substitute the {{key}} parts in your template with “John Doe”. The [attachments] argument is a list of additional MIME entities as [entity] objects. Refer to magic.lambda.mime to understand how these can be created and parametrised. If you specify at least one such additional [attachments] object, the email will be sent as a “multipart/mixed” entity. The [template-file] argument is assumed to be the path of an email template file in your system, that can be either HTML or plain text. If you have an HTML file you need to set the [mime-type] argument to text/html - Otherwise you should set this argument to text/plain.

[magic.google.translate]

This slot allows you to translate something using Google Translate, and it requires the following arguments.

Below is example usage.

signal:magic.google.translate
   text:This is some piece of text that will be translated into Arabic
   src-lang:en
   dest-lang:ar

The above will result in the following after executing it.

signal:هذا جزء من النص سيتم ترجمته إلى اللغة العربية

The slots requires the two character ISO code for its source and destination language arguments. You can lookup this online if you wish.

[magic.io.file.load-recursively]

This slot loads all files recursively within the specified folder, and returns their filenames and their content. The following example Hyperlambda illustrates its usage.

signal:magic.io.file.load-recursively
   .:/modules/

The above will result in something resembling the following. Notice, the result below has been significantly snippet.

signal
   :/modules/README.md
      :"\n# Your modules folder\n\nThis ... snip ...\n"
   :/modules/magic/crypto_invocations-count.get.hl
      :@"..."

As you can see above the result is an array of filenames, where each filename has one child node, being the actual content of the file. The slot is used when for instance a frontend application is being generated based upon a template - In addition when you download a folder or something similar.

[magic.modules.ensure-database]

This slot ensures that a module’s database exists by invoking its associated create database SQL file. The slot requires the following arguments.

The slot basically checks to see if the specified [database] database exists, and if not, attempts to find its associated “xxx.yyy.sql” script within its folder, where “xxx” is the module name and “yyy” is the default database type. The slot will also automatically execute any migration scripts associated with the module. Notice, the “Ensure database” macro in “Hyper IDE” automatically takes care of wiring up invocations to this slot, creating the required structure in the process.

[magic.modules.install-module]

This slot installs a module that has been previously unzipped into your modules folder by making sure all startup files are executed. Typically you wouldn’t want to execute this directly yourself, but rely upon Magic’s middleware to invoke the slot. If you unzip a zip file inside of your “modules” folder, and/or download a bazar item, this slot will be automatically invoked.

[transformers.hash-password]

This slot is a “transformer” and typically used for input payloads originating from HTTP invocations, such as when creating a new user, etc. The following Hyperlambda illustrates its usage.

.arguments
   password:foo
eval:x:+
signal:transformers.hash-password
   reference:x:@.arguments/*/password

The idea of the slot is that it transforms in place the specified argument by reference. Implying executing the above Hyperlambda will result in something resembling the following.

.arguments
   password:$2b$10$ZjOhP3DecbgqYsTKknvLmODs1H1C6bfCheBug6iTk5TNmEckE9H3e

This makes it perfect for “transforming” input payloads containing passwords before persisting these into your database.