[WIP] update reva and refactor config

this pr tries to clarify the storage provider setup and configuration for the ocis storage extension

update to latest reva

reduce driver specific config to /home, /user, /public storages and the metadata storage which is not accessible using the gateway

use ocis driver

[x] requires https://github.com/cs3org/reva/pull/1225, especially https://github.com/cs3org/reva/pull/1225/commits/b93bc8226df1a6e99b3df64edc9733d6323cda08 which fixes bugs that allow the accounts service to provision accounts when using the ocis driver

[x] with https://github.com/cs3org/reva/pull/1225 the accounts service can create /meta/accounts ... however the default users don't seem to be created in there, which causes subsequent logins to fail because the system users for konnectd and reva don't  exist ... noone can login. AFAICT the accounts service needs to create the default accounts. fixed with https://github.com/cs3org/reva/pull/1225/commits/b93bc8226df1a6e99b3df64edc9733d6323cda08

[ ] for eos - and this is untested - the only thing that should be necessary is setting STORAGE_HOME_DRIVER=eoshome and STORAGE_USERS_DRIVER=eos. But since the docker compose starts all services at once we need to use ocis to stop and start the home and user storages:
docker-compose exec ocis ./bin/ocis kill storage-home
docker-compose exec -e STORAGE_HOME_DRIVER=eoshome ocis ./bin/ocis run storage-home
docker-compose exec ocis ./bin/ocis kill storage-users
docker-compose exec -e STORAGE_USERS_DRIVER=eos ocis ./bin/ocis run storage-users

The storages now start a dataprovider with the same driver, so an additional config for them is no longer necessary. Unfortunately, testing this is currently not possible because the reva user used by eos to access ldap ... does not exist when the new account service backend has not created the users ... this might also be related to ldap support not working.
:thinking:
I think there is a problem with the ldap setup step. https://github.com/owncloud-docker/eos-stack/blob/de26692b6d6544620b948fa8212a4fa57455ae1d/eos-ocis-dev/start-ldap#L10-L11 requires an LDAP_BINDDN ... which to be honest I have no clue how it is set because in all the owncloud orga we are not setting it for eos. even in owncloud-docker where https://github.com/owncloud-docker/compose-playground/blob/c563391876ed63b52ab2960df025947d4788a7fe/examples/eos-compose-acceptance-tests/docker-compose.yml#L91 looks most promising I assume it was intended for konnectd ...

but how did resolving users in the docker container with id einstein ever work ... :thinking: ... because we were not checking the credentials in glauth ... or accounts?? ... :zany_face:
IN any case I think we need to tell eos to use the LDAP_BINDDN and LDAP_BINDBW that match the accounts service user ... 95cb8724-03b2-11eb-a0a6-c33ef8ef53ad and we should probably configure it by default ... currently it is empty...
AFAICT this is good on its own but other wheels need fixing ...

Interesting, the tests fail because of permissions errors:
2020-10-06T14:39:50Z ERR srv/app/pkg/mod/github.com/cs3org/reva@v1.2.2-0.20201006093611-4a9be347ac29/internal/grpc/services/storageprovider/storageprovider.go:399 > permission denied error="error: permission denied: root/accounts"

root/accounts is of the form nodeid:basename so the accounts dir is probably not owned by the user that tries to access the accounts folder in the root of the storage. It is configured to use the ocis driver...

[ ] log which user ID is trying to access the folder

The problem is that the ocis driver checks the permissions. While the user is provided by the accounts service the root:accounts node ... Has no owner... It has not even been created... So we need a way to create the folder... Or tell the ocis storage driver who owns the storage... Hmmm

Even if we add a config option for the owner... We still need to create the accounts dir. Or is the accounts service trying to create the folder if it does not exist? @IljaN @refs @kulmann

Cool, it does: https://github.com/owncloud/ocis/blob/f10950d6e857c22fba7dfe2879bfd283677ec221/accounts/pkg/storage/cs3.go#L226

We could add a config option for the owner. I would prefer that over having a storage that can be added by anyone.
But having a single user ID as the owner limits access to a single user... Which means we would have to share the credentials of that user with all services that want to use the metadata storage.
We could use a map with /path -> owner ID mapping
It could be used to set the owner if a node has no owner set. Which still does not work for root. The storage could precreate these paths when it is initialized and set the owner.
The map would be similar to the storage registry rules...

hm, the account service is also trying to create accounts. but it fails:
2020-10-06T21:43:37+02:00 INF access token is already provided pkg=rhttp service=storage traceid=a05d4064c45adf0501fc17224928ef23
2020-10-06T21:43:37+02:00 DBG http routing: head=data tail=/accounts/4c510ada-c86b-4815-8820-42cdf82c3d51 svc=data pkg=rhttp service=storage
2020-10-06T21:43:37+02:00 DBG NodeFromPath() fn=/accounts/4c510ada-c86b-4815-8820-42cdf82c3d51 pkg=rhttp service=storage traceid=a05d4064c45adf0501fc17224928ef23
2020-10-06T21:43:37+02:00 DBG NodeFromPath() walk node={"Exists":true,"ID":"69afd7ba-55f2-4678-9e4c-aeaf14c56aca","Name":"accounts","ParentID":"root"} pkg=rhttp service=storage traceid=a05d4064c45adf0501fc17224928ef23
2020-10-06T21:43:37+02:00 DBG NodeFromPath() walk node={"Exists":false,"ID":"","Name":"4c510ada-c86b-4815-8820-42cdf82c3d51","ParentID":"69afd7ba-55f2-4678-9e4c-aeaf14c56aca"} pkg=rhttp service=storage traceid=a05d4064c45adf0501fc17224928ef23
2020-10-06T21:43:37+02:00 ERR error uploading file error="error: permission denied: 69afd7ba-55f2-4678-9e4c-aeaf14c56aca/4c510ada-c86b-4815-8820-42cdf82c3d51" pkg=rhttp service=storage traceid=a05d4064c45adf0501fc17224928ef23

Oh I can push changes to butonic:rename-storage. That's nice.

After renaming the environment variables the eos setup worked.
I'm not sure what I need to do for the ldap setup..

I've added comments to config settings that I've found in drone

diff --git a/docs/extensions/storage/configuration.md b/docs/extensions/storage/configuration.md
index f77e5bba..5f06d4f5 100644
--- a/docs/extensions/storage/configuration.md
+++ b/docs/extensions/storage/configuration.md
@@ -1,6 +1,6 @@
 ---
 title: "Configuration"
-date: "2020-10-05T21:57:18+0200"
+date: "2020-10-12T17:33:44+0200"
 weight: 20
 geekdocRepo: https://github.com/owncloud/ocis
 geekdocEditPath: edit/master/docs/extensions/storage
@@ -57,125 +57,170 @@ Usage: `storage [global options] command [command options] [arguments...]`

 ## Sub Commands

-### storage sharing
+### storage storage-home

-Start sharing service
+Start storage-home service

-Usage: `storage sharing [command options] [arguments...]`
+Usage: `storage storage-home [command options] [arguments...]`

---debug-addr | $STORAGE_SHARING_DEBUG_ADDR
-: Address to bind debug server. Default: `<IP_ADDRESS>:9151`.
+--debug-addr | $STORAGE_HOME_DEBUG_ADDR
+: Address to bind debug server. Default: `<IP_ADDRESS>:9156`.

---network | $STORAGE_SHARING_NETWORK
+--grpc-network | $STORAGE_HOME_GRPC_NETWORK
 : Network to use for the storage service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.

---protocol | $STORAGE_SHARING_PROTOCOL
-: protocol for storage service, can be 'http' or 'grpc'. Default: `grpc`.
+--grpc-addr | $STORAGE_HOME_GRPC_ADDR
+: Address to bind storage service. Default: `<IP_ADDRESS>:9154`.

---addr | $STORAGE_SHARING_ADDR
-: Address to bind storage service. Default: `<IP_ADDRESS>:9150`.
+--http-network | $STORAGE_HOME_HTTP_NETWORK
+: Network to use for the storage service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.

---url | $STORAGE_SHARING_URL
-: URL to use for the storage service. Default: `localhost:9150`.
+--http-addr | $STORAGE_HOME_HTTP_ADDR
+: Address to bind storage service. Default: `<IP_ADDRESS>:9155`.

---user-driver | $STORAGE_SHARING_USER_DRIVER
-: driver to use for the UserShareProvider. Default: `json`.
+--driver | $STORAGE_HOME_DRIVER
+: storage driver for home mount: eg. local, eos, owncloud, ocis or s3. Default: `ocis`.

---user-json-file | $STORAGE_SHARING_USER_JSON_FILE
-: file used to persist shares for the UserShareProvider. Default: `/var/tmp/ocis/shares.json`.
+--mount-path | $STORAGE_HOME_MOUNT_PATH
+: mount path. Default: `/home`.

---public-driver | $STORAGE_SHARING_PUBLIC_DRIVER
-: driver to use for the PublicShareProvider. Default: `json`.
+--mount-id | $STORAGE_HOME_MOUNT_ID
+: mount id. Default: `1284d238-aa92-42ce-bdc4-0b0000009162`.

-### storage storage-eos
+--expose-data-server | $STORAGE_HOME_EXPOSE_DATA_SERVER
+: exposes a dedicated data server. Default: `false`.

-Start storage-eos service
+--data-server-url | $STORAGE_HOME_DATA_SERVER_URL
+: data server url. Default: `http://localhost:9155/data`.

-Usage: `storage storage-eos [command options] [arguments...]`
+--http-prefix | $STORAGE_HOME_HTTP_PREFIX
+: prefix for the http endpoint, without leading slash. Default: `data`.

---debug-addr | $STORAGE_STORAGE_EOS_DEBUG_ADDR
-: Address to bind debug server. Default: `<IP_ADDRESS>:9159`.
+--enable-home | $STORAGE_HOME_ENABLE_HOME
+: enable the creation of home directories. Default: `true`.

---network | $STORAGE_STORAGE_EOS_NETWORK
+--gateway-endpoint | $STORAGE_GATEWAY_ENDPOINT
+: endpoint to use for the storage gateway service. Default: `localhost:9142`.
+
+--users-endpoint | $STORAGE_USERPROVIDER_ENDPOINT
+: endpoint to use for the storage service. Default: `localhost:9144`.
+
+### storage storage-public-link
+
+Start storage-public-link service
+
+Usage: `storage storage-public-link [command options] [arguments...]`
+
+--debug-addr | $STORAGE_PUBLIC_LINK_DEBUG_ADDR
+: Address to bind debug server. Default: `<IP_ADDRESS>:9179`.
+
+--network | $STORAGE_PUBLIC_LINK_GRPC_NETWORK
 : Network to use for the storage service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.

---protocol | $STORAGE_STORAGE_EOS_PROTOCOL
-: protocol for storage service, can be 'http' or 'grpc'. Default: `grpc`.
+--addr | $STORAGE_PUBLIC_LINK_GRPC_ADDR
+: Address to bind storage service. Default: `<IP_ADDRESS>:9178`.

---addr | $STORAGE_STORAGE_EOS_ADDR
-: Address to bind storage service. Default: `<IP_ADDRESS>:9158`.
+--mount-path | $STORAGE_PUBLIC_LINK_MOUNT_PATH
+: mount path. Default: `/public`.

---url | $STORAGE_STORAGE_EOS_URL
-: URL to use for the storage service. Default: `localhost:9158`.
+--gateway-endpoint | $STORAGE_GATEWAY_ENDPOINT
+: endpoint to use for the storage gateway service. Default: `localhost:9142`.

---driver | $STORAGE_STORAGE_EOS_DRIVER
-: storage driver for eos mount: eg. local, eos, owncloud, ocis or s3. Default: `eos`.
+### storage auth-basic

---mount-path | $STORAGE_STORAGE_EOS_MOUNT_PATH
-: mount path. Default: `/eos`.
+Start authprovider for basic auth

---mount-id | $STORAGE_STORAGE_EOS_MOUNT_ID
-: mount id. Default: `1284d238-aa92-42ce-bdc4-0b0000009158`.
+Usage: `storage auth-basic [command options] [arguments...]`

---expose-data-server | $STORAGE_STORAGE_EOS_EXPOSE_DATA_SERVER
-: exposes a dedicated data server. Default: `false`.
+--debug-addr | $STORAGE_AUTH_BASIC_DEBUG_ADDR
+: Address to bind debug server. Default: `<IP_ADDRESS>:9147`.

---data-server-url | $STORAGE_STORAGE_EOS_DATA_SERVER_URL
-: data server url. Default: `http://localhost:9160/data`.
+--auth-driver | $STORAGE_AUTH_DRIVER
+: auth driver: 'demo', 'json' or 'ldap'. Default: `ldap`.

-### storage users
+--auth-json | $STORAGE_AUTH_JSON
+: Path to users.json file.

-Start users service
+--network | $STORAGE_AUTH_BASIC_GRPC_NETWORK
+: Network to use for the storage auth-basic service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.

-Usage: `storage users [command options] [arguments...]`
+--addr | $STORAGE_AUTH_BASIC_GRPC_ADDR
+: Address to bind storage service. Default: `<IP_ADDRESS>:9146`.

---debug-addr | $STORAGE_SHARING_DEBUG_ADDR
-: Address to bind debug server. Default: `<IP_ADDRESS>:9145`.
+### storage auth-bearer
+
+Start authprovider for bearer auth
+
+Usage: `storage auth-bearer [command options] [arguments...]`
+
+--debug-addr | $STORAGE_AUTH_BEARER_DEBUG_ADDR
+: Address to bind debug server. Default: `<IP_ADDRESS>:9149`.
+
+--oidc-issuer | $STORAGE_OIDC_ISSUER
+: OIDC issuer. Default: `https://localhost:9200`.
+
+--oidc-insecure | $STORAGE_OIDC_INSECURE
+: OIDC allow insecure communication. Default: `true`.
+
+--oidc-id-claim | $STORAGE_OIDC_ID_CLAIM
+: OIDC id claim. Default: `preferred_username`.
+
+--oidc-uid-claim | $STORAGE_OIDC_UID_CLAIM
+: OIDC uid claim.

---network | $STORAGE_USERS_NETWORK
+--oidc-gid-claim | $STORAGE_OIDC_GID_CLAIM
+: OIDC gid claim.
+
+--network | $STORAGE_AUTH_BEARER_GRPC_NETWORK
 : Network to use for the storage service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.

---protocol | $STORAGE_USERS_PROTOCOL
-: protocol for storage service, can be 'http' or 'grpc'. Default: `grpc`.
+--addr | $STORAGE_AUTH_BEARER_GRPC_ADDR
+: Address to bind storage service. Default: `<IP_ADDRESS>:9148`.

---addr | $STORAGE_USERS_ADDR
-: Address to bind storage service. Default: `<IP_ADDRESS>:9144`.
+### storage frontend

---url | $STORAGE_USERS_URL
-: URL to use for the storage service. Default: `localhost:9144`.
+Start frontend service

---driver | $STORAGE_USERS_DRIVER
-: user driver: 'demo', 'json', 'ldap', or 'rest'. Default: `ldap`.
+Usage: `storage frontend [command options] [arguments...]`

---json-config | $STORAGE_USERS_JSON
-: Path to users.json file.
+--debug-addr | $STORAGE_FRONTEND_DEBUG_ADDR
+: Address to bind debug server. Default: `<IP_ADDRESS>:9141`.

---rest-client-id | $STORAGE_REST_CLIENT_ID
-: User rest driver Client ID.
+--transfer-secret | $STORAGE_TRANSFER_SECRET
+: Transfer secret for datagateway. Default: `replace-me-with-a-transfer-secret`.

---rest-client-secret | $STORAGE_REST_CLIENT_SECRET
-: User rest driver Client Secret.
+--webdav-namespace | $WEBDAV_NAMESPACE
+: Namespace prefix for the /webdav endpoint. Default: `/home/`.

---rest-redis-address | $STORAGE_REST_REDIS_ADDRESS
-: Address for redis server. Default: `localhost:6379`.
+--dav-files-namespace | $DAV_FILES_NAMESPACE
+: Namespace prefix for the webdav /dav/files endpoint. Default: `/users/`.

---rest-redis-username | $STORAGE_REST_REDIS_USERNAME
-: Username for redis server.
+--network | $STORAGE_FRONTEND_HTTP_NETWORK
+: Network to use for the storage service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.

---rest-redis-password | $STORAGE_REST_REDIS_PASSWORD
-: Password for redis server.
+--addr | $STORAGE_FRONTEND_HTTP_ADDR
+: Address to bind storage service. Default: `<IP_ADDRESS>:9140`.

---rest-id-provider | $STORAGE_REST_ID_PROVIDER
-: The OIDC Provider.
+--public-url | $STORAGE_FRONTEND_PUBLIC_URL
+: URL to use for the storage service. Default: `https://localhost:9200`.

---rest-api-base-url | $STORAGE_REST_API_BASE_URL
-: Base API Endpoint.
+--datagateway-prefix | $STORAGE_FRONTEND_DATAGATEWAY_PREFIX
+: datagateway prefix. Default: `data`.

---rest-oidc-token-endpoint | $STORAGE_REST_OIDC_TOKEN_ENDPOINT
-: Endpoint to generate token to access the API.
+--ocdav-prefix | $STORAGE_FRONTEND_OCDAV_PREFIX
+: owncloud webdav endpoint prefix.

---rest-target-api | $STORAGE_REST_TARGET_API
-: The target application.
+--ocs-prefix | $STORAGE_FRONTEND_OCS_PREFIX
+: open collaboration services endpoint prefix. Default: `ocs`.
+
+--gateway-url | $STORAGE_GATEWAY_ENDPOINT
+: URL to use for the storage gateway service. Default: `localhost:9142`.
+
+--upload-disable-tus | $STORAGE_FRONTEND_UPLOAD_DISABLE_TUS
+: Disables TUS upload mechanism. Default: `false`.
+
+--upload-http-method-override | $STORAGE_FRONTEND_UPLOAD_HTTP_METHOD_OVERRIDE
+: Specify an HTTP method (ex: POST) that clients should to use when uploading instead of PATCH.

 ### storage gateway

@@ -189,17 +234,14 @@ Usage: `storage gateway [command options] [arguments...]`
 --transfer-secret | $STORAGE_TRANSFER_SECRET
 : Transfer secret for datagateway. Default: `replace-me-with-a-transfer-secret`.

---network | $STORAGE_GATEWAY_NETWORK
+--network | $STORAGE_GATEWAY_GRPC_NETWORK
 : Network to use for the storage service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.

---protocol | $STORAGE_GATEWAY_PROTOCOL
-: protocol for storage service, can be 'http' or 'grpc'. Default: `grpc`.
-
---addr | $STORAGE_GATEWAY_ADDR
+--addr | $STORAGE_GATEWAY_GRPC_ADDR
 : Address to bind storage service. Default: `<IP_ADDRESS>:9142`.

---url | $STORAGE_GATEWAY_URL
-: URL to use for the storage service. Default: `localhost:9142`.
+--endpoint | $STORAGE_GATEWAY_ENDPOINT
+: endpoint to use for the storage service. Default: `localhost:9142`.

 --commit-share-to-storage-grant | $STORAGE_GATEWAY_COMMIT_SHARE_TO_STORAGE_GRANT
 : Commit shares to the share manager. Default: `true`.
@@ -213,41 +255,32 @@ Usage: `storage gateway [command options] [arguments...]`
 --disable-home-creation-on-login | $STORAGE_GATEWAY_DISABLE_HOME_CREATION_ON_LOGIN
 : Disable creation of home folder on login.

+--auth-basic-endpoint | $STORAGE_AUTH_BASIC_ENDPOINT
+: endpoint to use for the basic auth provider. Default: `localhost:9146`.
+
+--auth-bearer-endpoint | $STORAGE_AUTH_BEARER_ENDPOINT
+: endpoint to use for the bearer auth provider. Default: `localhost:9148`.
+
 --storage-registry-driver | $STORAGE_STORAGE_REGISTRY_DRIVER
 : driver of the storage registry. Default: `static`.

---storage-home-provider | $STORAGE_STORAGE_HOME_PROVIDER
+--storage-home-provider | $STORAGE_REGISTRY_HOME_PROVIDER
 : mount point of the storage provider for user homes in the global namespace. Default: `/home`.

---frontend-url | $STORAGE_FRONTEND_URL
+--public-url | $STORAGE_FRONTEND_PUBLIC_URL
 : URL to use for the storage service. Default: `https://localhost:9200`.

---datagateway-url | $STORAGE_DATAGATEWAY_URL
+--datagateway-url | $STORAGE_DATAGATEWAY_PUBLIC_URL
 : URL to use for the storage datagateway. Default: `https://localhost:9200/data`.

---users-url | $STORAGE_USERS_URL
-: URL to use for the storage service. Default: `localhost:9144`.
-
---auth-basic-url | $STORAGE_AUTH_BASIC_URL
-: URL to use for the storage service. Default: `localhost:9146`.
-
---auth-bearer-url | $STORAGE_AUTH_BEARER_URL
-: URL to use for the storage service. Default: `localhost:9148`.
-
---sharing-url | $STORAGE_SHARING_URL
-: URL to use for the storage service. Default: `localhost:9150`.
-
---storage-root-url | $STORAGE_STORAGE_ROOT_URL
-: URL to use for the storage service. Default: `localhost:9152`.
+--userprovider-endpoint | $STORAGE_USERPROVIDER_ENDPOINT
+: endpoint to use for the userprovider. Default: `localhost:9144`.

---storage-root-mount-path | $STORAGE_STORAGE_ROOT_MOUNT_PATH
-: mount path. Default: `/`.
+--sharing-endpoint | $STORAGE_SHARING_ENDPOINT
+: endpoint to use for the storage service. Default: `localhost:9150`.

---storage-root-mount-id | $STORAGE_STORAGE_ROOT_MOUNT_ID
-: mount id. Default: `1284d238-aa92-42ce-bdc4-0b0000009152`.
-
---storage-home-url | $STORAGE_STORAGE_HOME_URL
-: URL to use for the storage service. Default: `localhost:9154`.
+--storage-home-endpoint | $STORAGE_STORAGE_HOME_ENDPOINT
+: endpoint to use for the home storage. Default: `localhost:9154`.

 --storage-home-mount-path | $STORAGE_STORAGE_HOME_MOUNT_PATH
 : mount path. Default: `/home`.
@@ -255,450 +288,305 @@ Usage: `storage gateway [command options] [arguments...]`
 --storage-home-mount-id | $STORAGE_STORAGE_HOME_MOUNT_ID
 : mount id. Default: `1284d238-aa92-42ce-bdc4-0b0000009154`.

---storage-eos-url | $STORAGE_STORAGE_EOS_URL
-: URL to use for the storage service. Default: `localhost:9158`.
-
---storage-eos-mount-path | $STORAGE_STORAGE_EOS_MOUNT_PATH
-: mount path. Default: `/eos`.
-
---storage-eos-mount-id | $STORAGE_STORAGE_EOS_MOUNT_ID
-: mount id. Default: `1284d238-aa92-42ce-bdc4-0b0000009158`.
+--storage-users-endpoint | $STORAGE_USERS_ENDPOINT
+: endpoint to use for the users storage. Default: `localhost:9157`.

---storage-oc-url | $STORAGE_STORAGE_OC_URL
-: URL to use for the storage service. Default: `localhost:9162`.
+--storage-users-mount-path | $STORAGE_USERS_MOUNT_PATH
+: mount path. Default: `/users`.

---storage-oc-mount-path | $STORAGE_STORAGE_OC_MOUNT_PATH
-: mount path. Default: `/oc`.
+--storage-users-mount-id | $STORAGE_USERS_MOUNT_ID
+: mount id. Default: `1284d238-aa92-42ce-bdc4-0b0000009157`.

---storage-oc-mount-id | $STORAGE_STORAGE_OC_MOUNT_ID
-: mount id. Default: `1284d238-aa92-42ce-bdc4-0b0000009162`.
-
---public-link-url | $STORAGE_STORAGE_PUBLIC_LINK_URL
-: URL to use for the public links service. Default: `localhost:9178`.
+--public-link-endpoint | $STORAGE_STORAGE_PUBLIC_LINK_ENDPOINT
+: endpoint to use for the public links service. Default: `localhost:9178`.

 --storage-public-link-mount-path | $STORAGE_STORAGE_PUBLIC_LINK_MOUNT_PATH
-: mount path. Default: `/public/`.
-
-### storage storage
-
-Storage service for oCIS
-
-Usage: `storage storage [command options] [arguments...]`
-
---config-file | $STORAGE_CONFIG_FILE
-: Path to config file.
-
---log-level | $STORAGE_LOG_LEVEL
-: Set logging level. Default: `info`.
-
---log-pretty | $STORAGE_LOG_PRETTY
-: Enable pretty logging.
-
---log-color | $STORAGE_LOG_COLOR
-: Enable colored logging.
-
-### storage storage-home
-
-Start storage-home service
-
-Usage: `storage storage-home [command options] [arguments...]`
-
---debug-addr | $STORAGE_STORAGE_HOME_DEBUG_ADDR
-: Address to bind debug server. Default: `<IP_ADDRESS>:9155`.
-
---network | $STORAGE_STORAGE_HOME_NETWORK
-: Network to use for the storage service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
-
---protocol | $STORAGE_STORAGE_HOME_PROTOCOL
-: protocol for storage service, can be 'http' or 'grpc'. Default: `grpc`.
+: mount path. Default: `/public`.

---addr | $STORAGE_STORAGE_HOME_ADDR
-: Address to bind storage service. Default: `<IP_ADDRESS>:9154`.
-
---url | $STORAGE_STORAGE_HOME_URL
-: URL to use for the storage service. Default: `localhost:9154`.
-
---driver | $STORAGE_STORAGE_HOME_DRIVER
-: storage driver for home mount: eg. local, eos, owncloud, ocis or s3. Default: `owncloud`.
-
---mount-path | $STORAGE_STORAGE_HOME_MOUNT_PATH
-: mount path. Default: `/home`.
-
---mount-id | $STORAGE_STORAGE_HOME_MOUNT_ID
-: mount id. Default: `1284d238-aa92-42ce-bdc4-0b0000009162`.
-
---expose-data-server | $STORAGE_STORAGE_HOME_EXPOSE_DATA_SERVER
-: exposes a dedicated data server. Default: `false`.
+### storage health

---data-server-url | $STORAGE_STORAGE_HOME_DATA_SERVER_URL
-: data server url. Default: `http://localhost:9156/data`.
+Check health status

---enable-home | $STORAGE_STORAGE_HOME_ENABLE_HOME
-: enable the creation of home directories. Default: `true`.
+Usage: `storage health [command options] [arguments...]`

---users-url | $STORAGE_USERS_URL
-: URL to use for the storage service. Default: `localhost:9144`.
+--debug-addr | $STORAGE_DEBUG_ADDR
+: Address to debug endpoint. Default: `<IP_ADDRESS>:9109`.

-### storage storage-metadata
+### storage sharing

-Start storage-metadata service
+Start sharing service

-Usage: `storage storage-metadata [command options] [arguments...]`
+Usage: `storage sharing [command options] [arguments...]`

---debug-addr | $STORAGE_STORAGE_METADATA_DEBUG_ADDR
-: Address to bind debug server. Default: `<IP_ADDRESS>:9217`.
+--debug-addr | $STORAGE_SHARING_DEBUG_ADDR
+: Address to bind debug server. Default: `<IP_ADDRESS>:9151`.

---network | $STORAGE_STORAGE_METADATA_NETWORK
+--network | $STORAGE_SHARING_GRPC_NETWORK
 : Network to use for the storage service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.

---provider-addr | $STORAGE_STORAGE_METADATA_PROVIDER_ADDR
-: Address to bind storage service. Default: `<IP_ADDRESS>:9215`.
+--addr | $STORAGE_SHARING_GRPC_ADDR
+: Address to bind storage service. Default: `<IP_ADDRESS>:9150`.

---data-server-url | $STORAGE_STORAGE_METADATA_DATA_SERVER_URL
-: URL of the data-server the storage-provider uses. Default: `http://localhost:9216`.
+--user-driver | $STORAGE_SHARING_USER_DRIVER
+: driver to use for the UserShareProvider. Default: `json`.

---data-server-addr | $STORAGE_STORAGE_METADATA_DATA_SERVER_ADDR
-: Address to bind the metadata data-server to. Default: `<IP_ADDRESS>:9216`.
+--user-json-file | $STORAGE_SHARING_USER_JSON_FILE
+: file used to persist shares for the UserShareProvider. Default: `/var/tmp/ocis/shares.json`.

---storage-provider-driver | $STORAGE_STORAGE_METADATA_PROVIDER_DRIVER
-: storage driver for metadata mount: eg. local, eos, owncloud, ocis or s3. Default: `local`.
+--public-driver | $STORAGE_SHARING_PUBLIC_DRIVER
+: driver to use for the PublicShareProvider. Default: `json`.

---data-provider-driver | $STORAGE_STORAGE_METADATA_DATA_PROVIDER_DRIVER
-: storage driver for data-provider mount: eg. local, eos, owncloud, ocis or s3. Default: `local`.
+### storage storage-users

---storage-root | $STORAGE_STORAGE_METADATA_ROOT
-: the path to the metadata storage root. Default: `/var/tmp/ocis/metadata`.
+Start storage-users service

-### storage storage-oc
+Usage: `storage storage-users [command options] [arguments...]`

-Start storage-oc service
+--debug-addr | $STORAGE_USERS_DEBUG_ADDR
+: Address to bind debug server. Default: `<IP_ADDRESS>:9159`.

-Usage: `storage storage-oc [command options] [arguments...]`
+--grpc-network | $STORAGE_USERS_GRPC_NETWORK
+: Network to use for the users storage, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.

---debug-addr | $STORAGE_STORAGE_OC_DEBUG_ADDR
-: Address to bind debug server. Default: `<IP_ADDRESS>:9163`.
+--grpc-addr | $STORAGE_USERS_GRPC_ADDR
+: GRPC Address to bind users storage. Default: `<IP_ADDRESS>:9157`.

---network | $STORAGE_STORAGE_OC_NETWORK
+--http-network | $STORAGE_USERS_HTTP_NETWORK
 : Network to use for the storage service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.

---protocol | $STORAGE_STORAGE_OC_PROTOCOL
-: protocol for storage service, can be 'http' or 'grpc'. Default: `grpc`.
-
---addr | $STORAGE_STORAGE_OC_ADDR
-: Address to bind storage service. Default: `<IP_ADDRESS>:9162`.
-
---url | $STORAGE_STORAGE_OC_URL
-: URL to use for the storage service. Default: `localhost:9162`.
+--http-addr | $STORAGE_USERS_HTTP_ADDR
+: HTTP Address to bind users storage. Default: `<IP_ADDRESS>:9158`.

---driver | $STORAGE_STORAGE_OC_DRIVER
-: storage driver for oc mount: eg. local, eos, owncloud, ocis or s3. Default: `owncloud`.
+--driver | $STORAGE_USERS_DRIVER
+: storage driver for users mount: eg. local, eos, owncloud, ocis or s3. Default: `ocis`.

---mount-path | $STORAGE_STORAGE_OC_MOUNT_PATH
-: mount path. Default: `/oc`.
+--mount-path | $STORAGE_USERS_MOUNT_PATH
+: mount path. Default: `/users`.

---mount-id | $STORAGE_STORAGE_OC_MOUNT_ID
+--mount-id | $STORAGE_USERS_MOUNT_ID
 : mount id. Default: `1284d238-aa92-42ce-bdc4-0b0000009162`.

---expose-data-server | $STORAGE_STORAGE_OC_EXPOSE_DATA_SERVER
+--expose-data-server | $STORAGE_USERS_EXPOSE_DATA_SERVER
 : exposes a dedicated data server. Default: `false`.

---data-server-url | $STORAGE_STORAGE_OC_DATA_SERVER_URL
-: data server url. Default: `http://localhost:9164/data`.
-
---users-url | $STORAGE_USERS_URL
-: URL to use for the storage service. Default: `localhost:9144`.
-
-### storage storage-oc-data
-
-Start storage-oc-data service
-
-Usage: `storage storage-oc-data [command options] [arguments...]`
-
---debug-addr | $STORAGE_STORAGE_OC_DATA_DEBUG_ADDR
-: Address to bind debug server. Default: `<IP_ADDRESS>:9165`.
-
---network | $STORAGE_STORAGE_OC_DATA_NETWORK
-: Network to use for the storage service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
-
---protocol | $STORAGE_STORAGE_OC_DATA_PROTOCOL
-: protocol for storage service, can be 'http' or 'grpc'. Default: `http`.
-
---addr | $STORAGE_STORAGE_OC_DATA_ADDR
-: Address to bind storage service. Default: `<IP_ADDRESS>:9164`.
+--data-server-url | $STORAGE_USERS_DATA_SERVER_URL
+: data server url. Default: `http://localhost:9158/data`.

---url | $STORAGE_STORAGE_OC_DATA_URL
-: URL to use for the storage service. Default: `localhost:9164`.
-
---driver | $STORAGE_STORAGE_OC_DATA_DRIVER
-: storage driver for oc data mount: eg. local, eos, owncloud, ocis or s3. Default: `owncloud`.
-
---prefix | $STORAGE_STORAGE_OC_DATA_PREFIX
+--http-prefix | $STORAGE_USERS_HTTP_PREFIX
 : prefix for the http endpoint, without leading slash. Default: `data`.

---temp-folder | $STORAGE_STORAGE_OC_DATA_TEMP_FOLDER
-: temp folder. Default: `/var/tmp/`.
+--gateway-endpoint | $STORAGE_GATEWAY_ENDPOINT
+: endpoint to use for the storage gateway service. Default: `localhost:9142`.

---gateway-url | $STORAGE_GATEWAY_URL
-: URL to use for the storage gateway service. Default: `localhost:9142`.
+--users-endpoint | $STORAGE_USERPROVIDER_ENDPOINT
+: endpoint to use for the storage service. Default: `localhost:9144`.

---users-url | $STORAGE_USERS_URL
-: URL to use for the storage service. Default: `localhost:9144`.
-
-### storage storage-public-link
+### storage users

-Start storage-public-link service
+Start users service

-Usage: `storage storage-public-link [command options] [arguments...]`
+Usage: `storage users [command options] [arguments...]`

---debug-addr | $STORAGE_STORAGE_PUBLIC_LINK_DEBUG_ADDR
-: Address to bind debug server. Default: `<IP_ADDRESS>:9179`.
+--debug-addr | $STORAGE_SHARING_DEBUG_ADDR
+: Address to bind debug server. Default: `<IP_ADDRESS>:9145`.

---network | $STORAGE_STORAGE_PUBLIC_LINK_NETWORK
+--network | $STORAGE_USERPROVIDER_NETWORK
 : Network to use for the storage service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.

---protocol | $STORAGE_STORAGE_PUBLIC_LINK_PROTOCOL
-: protocol for storage service, can be 'http' or 'grpc'. Default: `grpc`.
-
---addr | $STORAGE_STORAGE_PUBLIC_LINK_ADDR
-: Address to bind storage service. Default: `<IP_ADDRESS>:9178`.
+--addr | $STORAGE_USERPROVIDER_ADDR
+: Address to bind storage service. Default: `<IP_ADDRESS>:9144`.

---url | $STORAGE_STORAGE_PUBLIC_LINK_URL
-: Address to bind storage service. Default: `localhost:9178`.
+--endpoint | $STORAGE_USERPROVIDER_ENDPOINT
+: URL to use for the storage service. Default: `localhost:9144`.

---mount-path | $STORAGE_STORAGE_PUBLIC_LINK_MOUNT_PATH
-: mount path. Default: `/public/`.
+--driver | $STORAGE_USERPROVIDER_DRIVER
+: user driver: 'demo', 'json', 'ldap', or 'rest'. Default: `ldap`.

---gateway-url | $STORAGE_GATEWAY_URL
-: URL to use for the storage gateway service. Default: `localhost:9142`.
+--json-config | $STORAGE_USERPROVIDER_JSON
+: Path to users.json file.

-### storage auth-bearer
+--rest-client-id | $STORAGE_REST_CLIENT_ID
+: User rest driver Client ID.

-Start authprovider for bearer auth
+--rest-client-secret | $STORAGE_REST_CLIENT_SECRET
+: User rest driver Client Secret.

-Usage: `storage auth-bearer [command options] [arguments...]`
+--rest-redis-address | $STORAGE_REST_REDIS_ADDRESS
+: Address for redis server. Default: `localhost:6379`.

---debug-addr | $STORAGE_AUTH_BEARER_DEBUG_ADDR
-: Address to bind debug server. Default: `<IP_ADDRESS>:9149`.
+--rest-redis-username | $STORAGE_REST_REDIS_USERNAME
+: Username for redis server.

---oidc-issuer | $STORAGE_OIDC_ISSUER
-: OIDC issuer. Default: `https://localhost:9200`.
+--rest-redis-password | $STORAGE_REST_REDIS_PASSWORD
+: Password for redis server.

---oidc-insecure | $STORAGE_OIDC_INSECURE
-: OIDC allow insecure communication. Default: `true`.
+--rest-id-provider | $STORAGE_REST_ID_PROVIDER
+: The OIDC Provider.

---oidc-id-claim | $STORAGE_OIDC_ID_CLAIM
-: OIDC id claim. Default: `preferred_username`.
+--rest-api-base-url | $STORAGE_REST_API_BASE_URL
+: Base API Endpoint.

---oidc-uid-claim | $STORAGE_OIDC_UID_CLAIM
-: OIDC uid claim.
+--rest-oidc-token-endpoint | $STORAGE_REST_OIDC_TOKEN_ENDPOINT
+: Endpoint to generate token to access the API.

---oidc-gid-claim | $STORAGE_OIDC_GID_CLAIM
-: OIDC gid claim.
+--rest-target-api | $STORAGE_REST_TARGET_API
+: The target application.

---network | $STORAGE_AUTH_BEARER_NETWORK
-: Network to use for the storage service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
+### storage storage

---protocol | $STORAGE_AUTH_BEARER_PROTOCOL
-: protocol for storage service, can be 'http' or 'grpc'. Default: `grpc`.
+Storage service for oCIS

---addr | $STORAGE_AUTH_BEARER_ADDR
-: Address to bind storage service. Default: `<IP_ADDRESS>:9148`.
+Usage: `storage storage [command options] [arguments...]`

---url | $STORAGE_AUTH_BEARER_URL
-: URL to use for the storage service. Default: `localhost:9148`.
+--config-file | $STORAGE_CONFIG_FILE
+: Path to config file.

-### storage frontend
+--log-level | $STORAGE_LOG_LEVEL
+: Set logging level. Default: `info`.

-Start frontend service
+--log-pretty | $STORAGE_LOG_PRETTY
+: Enable pretty logging.

-Usage: `storage frontend [command options] [arguments...]`
+--log-color | $STORAGE_LOG_COLOR
+: Enable colored logging.

---debug-addr | $STORAGE_FRONTEND_DEBUG_ADDR
-: Address to bind debug server. Default: `<IP_ADDRESS>:9141`.
+### storage storage-metadata

---transfer-secret | $STORAGE_TRANSFER_SECRET
-: Transfer secret for datagateway. Default: `replace-me-with-a-transfer-secret`.
+Start storage-metadata service

---webdav-namespace | $WEBDAV_NAMESPACE
-: Namespace prefix for the /webdav endpoint. Default: `/home/`.
+Usage: `storage storage-metadata [command options] [arguments...]`

---dav-files-namespace | $DAV_FILES_NAMESPACE
-: Namespace prefix for the webdav /dav/files endpoint. Default: `/oc/`.
+--debug-addr | $STORAGE_METADATA_DEBUG_ADDR
+: Address to bind debug server. Default: `<IP_ADDRESS>:9217`.

---network | $STORAGE_FRONTEND_NETWORK
+--grpc-network | $STORAGE_METADATA_GRPC_NETWORK
 : Network to use for the storage service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.

---protocol | $STORAGE_FRONTEND_PROTOCOL
-: protocol for storage service, can be 'http' or 'grpc'. Default: `http`.
-
---addr | $STORAGE_FRONTEND_ADDR
-: Address to bind storage service. Default: `<IP_ADDRESS>:9140`.
-
---url | $STORAGE_FRONTEND_URL
-: URL to use for the storage service. Default: `https://localhost:9200`.
-
---datagateway-prefix | $STORAGE_FRONTEND_DATAGATEWAY_PREFIX
-: datagateway prefix. Default: `data`.
-
---ocdav-prefix | $STORAGE_FRONTEND_OCDAV_PREFIX
-: owncloud webdav endpoint prefix.
-
---ocs-prefix | $STORAGE_FRONTEND_OCS_PREFIX
-: open collaboration services endpoint prefix. Default: `ocs`.
-
---gateway-url | $STORAGE_GATEWAY_URL
-: URL to use for the storage gateway service. Default: `localhost:9142`.
-
---upload-disable-tus | $STORAGE_FRONTEND_UPLOAD_DISABLE_TUS
-: Disables TUS upload mechanism. Default: `false`.
-
---upload-http-method-override | $STORAGE_FRONTEND_UPLOAD_HTTP_METHOD_OVERRIDE
-: Specify an HTTP method (ex: POST) that clients should to use when uploading instead of PATCH.
-
-### storage auth-basic
-
-Start authprovider for basic auth
-
-Usage: `storage auth-basic [command options] [arguments...]`
-
---debug-addr | $STORAGE_AUTH_BASIC_DEBUG_ADDR
-: Address to bind debug server. Default: `<IP_ADDRESS>:9147`.
-
---auth-driver | $STORAGE_AUTH_DRIVER
-: auth driver: 'demo', 'json' or 'ldap'. Default: `ldap`.
-
---auth-json | $STORAGE_AUTH_JSON
-: Path to users.json file.
-
---network | $STORAGE_AUTH_BASIC_NETWORK
-: Network to use for the storage auth-basic service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
+--grpc-addr | $STORAGE_METADATA_GRPC_PROVIDER_ADDR
+: Address to bind storage service. Default: `<IP_ADDRESS>:9215`.

---protocol | $STORAGE_AUTH_BASIC_PROTOCOL
-: protocol for storage service, can be 'http' or 'grpc'. Default: `grpc`.
+--data-server-url | $STORAGE_METADATA_DATA_SERVER_URL
+: URL of the data-provider the storage-provider uses. Default: `http://localhost:9216`.

---addr | $STORAGE_AUTH_BASIC_ADDR
-: Address to bind storage service. Default: `<IP_ADDRESS>:9146`.
+--http-network | $STORAGE_METADATA_HTTP_NETWORK
+: Network to use for the storage service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.

---url | $STORAGE_AUTH_BASIC_URL
-: URL to use for the storage service. Default: `localhost:9146`.
+--http-addr | $STORAGE_METADATA_HTTP_ADDR
+: Address to bind storage service. Default: `<IP_ADDRESS>:9216`.

-### storage health
+--driver | $STORAGE_METADATA_DRIVER
+: storage driver for metadata mount: eg. local, eos, owncloud, ocis or s3. Default: `ocis`.

-Check health status
+--gateway-endpoint | $STORAGE_GATEWAY_ENDPOINT
+: endpoint to use for the gateway service. Default: `localhost:9142`.

-Usage: `storage health [command options] [arguments...]`
+--userprovider-endpoint | $STORAGE_USERPROVIDER_ENDPOINT
+: endpoint to use for the userprovider service. Default: `localhost:9144`.

---debug-addr | $STORAGE_DEBUG_ADDR
-: Address to debug endpoint. Default: `<IP_ADDRESS>:9109`.
-
-### storage storage-eos-data
+--storage-root | $STORAGE_METADATA_ROOT
+: the path to the metadata storage root. Default: `/var/tmp/ocis/metadata`.

-Start storage-eos-data service
+## Config for the different Storage Drivers

-Usage: `storage storage-eos-data [command options] [arguments...]`
+You can set different storage drivers for the Storage Providers. Please check the storage provider configuration.

---debug-addr | $STORAGE_STORAGE_OC_DATA_DEBUG_ADDR
-: Address to bind debug server. Default: `<IP_ADDRESS>:9161`.
+Example: Set the home Storage Provider to `ocis`

---network | $STORAGE_STORAGE_EOS_DATA_NETWORK
-: Network to use for the storage service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
+`STORAGE_STORAGE_HOME_DRIVER=ocis`
+`STORAGE_STORAGE_HOME_DATA_DRIVER=ocis`

---protocol | $STORAGE_STORAGE_EOS_DATA_PROTOCOL
-: protocol for storage service, can be 'http' or 'grpc'. Default: `http`.
+### Local Driver

---addr | $STORAGE_STORAGE_EOS_DATA_ADDR
-: Address to bind storage service. Default: `<IP_ADDRESS>:9160`.
+--storage-local-root | $STORAGE_STORAGE_LOCAL_ROOT
+: the path to the local storage root. Default: `/var/tmp/ocis/local`.

---url | $STORAGE_STORAGE_EOS_DATA_URL
-: URL to use for the storage service. Default: `localhost:9160`.
+### Eos Driver

---driver | $STORAGE_STORAGE_EOS_DATA_DRIVER
-: storage driver for eos data mount: eg. local, eos, owncloud, ocis or s3. Default: `eos`.
+--storage-eos-namespace | $STORAGE_STORAGE_EOS_NAMESPACE
+: Namespace for metadata operations. Default: `/eos/dockertest/reva`.

---prefix | $STORAGE_STORAGE_EOS_DATA_PREFIX
-: prefix for the http endpoint, without leading slash. Default: `data`.
+--storage-eos-shadow-namespace | $STORAGE_STORAGE_EOS_SHADOW_NAMESPACE
+: Shadow namespace where share references are stored.

---temp-folder | $STORAGE_STORAGE_EOS_DATA_TEMP_FOLDER
-: temp folder. Default: `/var/tmp/`.
+--storage-eos-share-folder | $STORAGE_STORAGE_EOS_SHARE_FOLDER
+: name of the share folder. Default: `/Shares`.

---gateway-url | $STORAGE_GATEWAY_URL
-: URL to use for the storage gateway service. Default: `localhost:9142`.
+--storage-eos-binary | $STORAGE_STORAGE_EOS_BINARY
+: Location of the eos binary. Default: `/usr/bin/eos`.

---users-url | $STORAGE_USERS_URL
-: URL to use for the storage service. Default: `localhost:9144`.
+--storage-eos-xrdcopy-binary | $STORAGE_STORAGE_EOS_XRDCOPY_BINARY
+: Location of the xrdcopy binary. Default: `/usr/bin/xrdcopy`.

-### storage storage-home-data
+--storage-eos-master-url | $STORAGE_STORAGE_EOS_MASTER_URL
+: URL of the Master EOS MGM. Default: `root://eos-mgm1.eoscluster.cern.ch:1094`.

-Start storage-home-data service
+--storage-eos-slave-url | $STORAGE_STORAGE_EOS_SLAVE_URL
+: URL of the Slave EOS MGM. Default: `root://eos-mgm1.eoscluster.cern.ch:1094`.

-Usage: `storage storage-home-data [command options] [arguments...]`
+--storage-eos-cache-directory | $STORAGE_STORAGE_EOS_CACHE_DIRECTORY
+: Location on the local fs where to store reads. Default: `os.TempDir()`.

---debug-addr | $STORAGE_STORAGE_HOME_DATA_DEBUG_ADDR
-: Address to bind debug server. Default: `<IP_ADDRESS>:9157`.
+--storage-eos-enable-logging | $STORAGE_STORAGE_EOS_ENABLE_LOGGING
+: Enables logging of the commands executed.

---network | $STORAGE_STORAGE_HOME_DATA_NETWORK
-: Network to use for the storage service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
+--storage-eos-show-hidden-sysfiles | $STORAGE_STORAGE_EOS_SHOW_HIDDEN_SYSFILES
+: show internal EOS files like .sys.v# and .sys.a# files..

---protocol | $STORAGE_STORAGE_HOME_DATA_PROTOCOL
-: protocol for storage service, can be 'http' or 'grpc'. Default: `http`.
+--storage-eos-force-singleuser-mode | $STORAGE_STORAGE_EOS_FORCE_SINGLEUSER_MODE
+: force connections to EOS to use SingleUsername.

---addr | $STORAGE_STORAGE_HOME_DATA_ADDR
-: Address to bind storage service. Default: `<IP_ADDRESS>:9156`.
+--storage-eos-use-keytab | $STORAGE_STORAGE_EOS_USE_KEYTAB
+: authenticate requests by using an EOS keytab.

---url | $STORAGE_STORAGE_HOME_DATA_URL
-: URL to use for the storage service. Default: `localhost:9156`.
+--storage-eos-enable-home | $STORAGE_STORAGE_EOS_ENABLE_HOME
+: enable the creation of home directories.

---driver | $STORAGE_STORAGE_HOME_DATA_DRIVER
-: storage driver for home data mount: eg. local, eos, owncloud, ocis or s3. Default: `owncloud`.
+--storage-eos-sec-protocol | $STORAGE_STORAGE_EOS_SEC_PROTOCOL
+: the xrootd security protocol to use between the server and EOS.

---prefix | $STORAGE_STORAGE_HOME_DATA_PREFIX
-: prefix for the http endpoint, without leading slash. Default: `data`.
+--storage-eos-keytab | $STORAGE_STORAGE_EOS_KEYTAB
+: the location of the keytab to use to authenticate to EOS.

---temp-folder | $STORAGE_STORAGE_HOME_DATA_TEMP_FOLDER
-: temp folder. Default: `/var/tmp/`.
+--storage-eos-single-username | $STORAGE_STORAGE_EOS_SINGLE_USERNAME
+: the username to use when SingleUserMode is enabled.

---enable-home | $STORAGE_STORAGE_HOME_ENABLE_HOME
-: enable the creation of home directories. Default: `true`.
+--storage-eos-layout | $STORAGE_STORAGE_EOS_LAYOUT
+: `"layout of the users home dir path on disk, in addition to {{.Username}}, {{.UsernameLower}} and {{.Provider}} also supports prefixing dirs: "{{.UsernamePrefixCount.2}}/{{.UsernameLower}}" will turn "Einstein" into "Ei/Einstein" `. Default: `{{substr 0 1 .Username}}/{{.Username}}`.

---gateway-url | $STORAGE_GATEWAY_URL
+--storage-eos-gatewaysvc | $STORAGE_STORAGE_EOS_GATEWAYSVC
 : URL to use for the storage gateway service. Default: `localhost:9142`.

---users-url | $STORAGE_USERS_URL
-: URL to use for the storage service. Default: `localhost:9144`.
-
-### storage storage-root
+### owCloud Driver

-Start storage-root service
+--storage-owncloud-datadir | $STORAGE_STORAGE_OWNCLOUD_DATADIR
+: the path to the owncloud data directory. Default: `/var/tmp/ocis/owncloud`.

-Usage: `storage storage-root [command options] [arguments...]`
+--storage-owncloud-uploadinfo-dir | $STORAGE_STORAGE_OWNCLOUD_UPLOADINFO_DIR
+: the path to the tus upload info directory. Default: `/var/tmp/ocis/uploadinfo`.

---debug-addr | $STORAGE_STORAGE_ROOT_DEBUG_ADDR
-: Address to bind debug server. Default: `<IP_ADDRESS>:9153`.
-
---network | $STORAGE_STORAGE_ROOT_NETWORK
-: Network to use for the storage service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
+--storage-owncloud-share-folder | $STORAGE_STORAGE_OWNCLOUD_SHARE_FOLDER
+: name of the shares folder. Default: `/Shares`.

---protocol | $STORAGE_STORAGE_ROOT_PROTOCOL
-: protocol for storage service, can be 'http' or 'grpc'. Default: `grpc`.
+--storage-owncloud-scan | $STORAGE_STORAGE_OWNCLOUD_SCAN
+: scan files on startup to add fileids. Default: `true`.

---addr | $STORAGE_STORAGE_ROOT_ADDR
-: Address to bind storage service. Default: `<IP_ADDRESS>:9152`.
+--storage-owncloud-redis | $STORAGE_STORAGE_OWNCLOUD_REDIS_ADDR
+: the address of the redis server. Default: `:6379`.

---url | $STORAGE_STORAGE_ROOT_URL
-: URL to use for the storage service. Default: `localhost:9152`.
+--storage-owncloud-enable-home | $STORAGE_STORAGE_OWNCLOUD_ENABLE_HOME
+: enable the creation of home storages. Default: `false`.

---driver | $STORAGE_STORAGE_ROOT_DRIVER
-: storage driver for root mount: eg. local, eos, owncloud, ocis or s3. Default: `local`.
+--storage-owncloud-layout | $STORAGE_STORAGE_OWNCLOUD_LAYOUT
+: `"layout of the users home dir path on disk, in addition to {{.Username}}, {{.Mail}}, {{.Id.OpaqueId}}, {{.Id.Idp}} also supports prefixing dirs: "{{substr 0 1 .Username}}/{{.Username}}" will turn "Einstein" into "Ei/Einstein" `. Default: `{{.Id.OpaqueId}}`.

---mount-path | $STORAGE_STORAGE_ROOT_MOUNT_PATH
-: mount path. Default: `/`.
+### Ocis Driver

---mount-id | $STORAGE_STORAGE_ROOT_MOUNT_ID
-: mount id. Default: `123e4567-e89b-12d3-a456-426655440001`.
+--storage-ocis-root | $STORAGE_STORAGE_OCIS_ROOT
+: the path to the local storage root. Default: `/var/tmp/ocis/storage/users`.

---expose-data-server | $STORAGE_STORAGE_ROOT_EXPOSE_DATA_SERVER
-: exposes a dedicated data server.
+--storage-ocis-enable-home | $STORAGE_STORAGE_OCIS_ENABLE_HOME
+: enable the creation of home storages. Default: `false`.

---data-server-url | $STORAGE_STORAGE_ROOT_DATA_SERVER_URL
-: data server url.
+--storage-ocis-layout | $STORAGE_STORAGE_OCIS_LAYOUT
+: `"layout of the users home dir path on disk, in addition to {{.Username}}, {{.Mail}}, {{.Id.OpaqueId}}, {{.Id.Idp}} also supports prefixing dirs: "{{substr 0 1 .Username}}/{{.Username}}" will turn "Einstein" into "Ei/Einstein" `. Default: `{{.Id.OpaqueId}}`.

New config

title: "Configuration"
date: "2020-10-12T17:33:44+0200"
weight: 20
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs/extensions/storage
geekdocFilePath: configuration.md
{{< toc >}}
Configuration
oCIS Single Binary is not responsible for configuring extensions. Instead, each extension could either be configured by environment variables, cli flags or config files.
Each extension has its dedicated documentation page (e.g. https://owncloud.github.io/extensions/ocis_proxy/configuration) which lists all possible configurations. Config files and environment variables are picked up if you use the ./bin/ocis server command within the oCIS single binary. Command line flags must be set explicitly on the extensions subcommands.
Configuration using config files
Out of the box extensions will attempt to read configuration details from:
/etc/ocis
$HOME/.ocis
./config

For this configuration to be picked up, have a look at your extension root command and look for which default config name it has assigned. i.e: ocis-proxy reads proxy.json | yaml | toml ....
So far we support the file formats JSON and YAML, if you want to get a full example configuration just take a look at our repository, there you can always see the latest configuration format. These example configurations include all available options and the default values. The configuration file will be automatically loaded if it's placed at /etc/ocis/ocis.yml, ${HOME}/.ocis/ocis.yml or $(pwd)/config/ocis.yml.
Envrionment variables
If you prefer to configure the service with environment variables you can see the available variables below.
Commandline flags
If you prefer to configure the service with commandline flags you can see the available variables below. Command line flags are only working when calling the subcommand directly.
Root Command
Storage service for oCIS
Usage: storage [global options] command [command options] [arguments...]
--config-file | $STORAGE_CONFIG_FILE
: Path to config file.
--log-level | $STORAGE_LOG_LEVEL
: Set logging level. Default: info.
--log-pretty | $STORAGE_LOG_PRETTY
: Enable pretty logging.
--log-color | $STORAGE_LOG_COLOR
: Enable colored logging.
Sub Commands
storage storage-home
Start storage-home service
Usage: storage storage-home [command options] [arguments...]
--debug-addr | $STORAGE_HOME_DEBUG_ADDR
: Address to bind debug server. Default: <IP_ADDRESS>:9156.
--grpc-network | $STORAGE_HOME_GRPC_NETWORK
: Network to use for the storage service, can be 'tcp', 'udp' or 'unix'. Default: tcp.
--grpc-addr | $STORAGE_HOME_GRPC_ADDR
: Address to bind storage service. Default: <IP_ADDRESS>:9154.
--http-network | $STORAGE_HOME_HTTP_NETWORK
: Network to use for the storage service, can be 'tcp', 'udp' or 'unix'. Default: tcp.
--http-addr | $STORAGE_HOME_HTTP_ADDR
: Address to bind storage service. Default: <IP_ADDRESS>:9155.
--driver | $STORAGE_HOME_DRIVER
: storage driver for home mount: eg. local, eos, owncloud, ocis or s3. Default: ocis.
--mount-path | $STORAGE_HOME_MOUNT_PATH
: mount path. Default: /home.
--mount-id | $STORAGE_HOME_MOUNT_ID
: mount id. Default: 1284d238-aa92-42ce-bdc4-0b0000009162.
--expose-data-server | $STORAGE_HOME_EXPOSE_DATA_SERVER
: exposes a dedicated data server. Default: false.
--data-server-url | $STORAGE_HOME_DATA_SERVER_URL
: data server url. Default: http://localhost:9155/data.
--http-prefix | $STORAGE_HOME_HTTP_PREFIX
: prefix for the http endpoint, without leading slash. Default: data.
--enable-home | $STORAGE_HOME_ENABLE_HOME
: enable the creation of home directories. Default: true.
--gateway-endpoint | $STORAGE_GATEWAY_ENDPOINT
: endpoint to use for the storage gateway service. Default: localhost:9142.
--users-endpoint | $STORAGE_USERPROVIDER_ENDPOINT
: endpoint to use for the storage service. Default: localhost:9144.
storage storage-public-link
Start storage-public-link service
Usage: storage storage-public-link [command options] [arguments...]
--debug-addr | $STORAGE_PUBLIC_LINK_DEBUG_ADDR
: Address to bind debug server. Default: <IP_ADDRESS>:9179.
--network | $STORAGE_PUBLIC_LINK_GRPC_NETWORK
: Network to use for the storage service, can be 'tcp', 'udp' or 'unix'. Default: tcp.
--addr | $STORAGE_PUBLIC_LINK_GRPC_ADDR
: Address to bind storage service. Default: <IP_ADDRESS>:9178.
--mount-path | $STORAGE_PUBLIC_LINK_MOUNT_PATH
: mount path. Default: /public.
--gateway-endpoint | $STORAGE_GATEWAY_ENDPOINT
: endpoint to use for the storage gateway service. Default: localhost:9142.
storage auth-basic
Start authprovider for basic auth
Usage: storage auth-basic [command options] [arguments...]
--debug-addr | $STORAGE_AUTH_BASIC_DEBUG_ADDR
: Address to bind debug server. Default: <IP_ADDRESS>:9147.
--auth-driver | $STORAGE_AUTH_DRIVER
: auth driver: 'demo', 'json' or 'ldap'. Default: ldap.
--auth-json | $STORAGE_AUTH_JSON
: Path to users.json file.
--network | $STORAGE_AUTH_BASIC_GRPC_NETWORK
: Network to use for the storage auth-basic service, can be 'tcp', 'udp' or 'unix'. Default: tcp.
--addr | $STORAGE_AUTH_BASIC_GRPC_ADDR
: Address to bind storage service. Default: <IP_ADDRESS>:9146.
storage auth-bearer
Start authprovider for bearer auth
Usage: storage auth-bearer [command options] [arguments...]
--debug-addr | $STORAGE_AUTH_BEARER_DEBUG_ADDR
: Address to bind debug server. Default: <IP_ADDRESS>:9149.
--oidc-issuer | $STORAGE_OIDC_ISSUER
: OIDC issuer. Default: https://localhost:9200.
--oidc-insecure | $STORAGE_OIDC_INSECURE
: OIDC allow insecure communication. Default: true.
--oidc-id-claim | $STORAGE_OIDC_ID_CLAIM
: OIDC id claim. Default: preferred_username.
--oidc-uid-claim | $STORAGE_OIDC_UID_CLAIM
: OIDC uid claim.
--oidc-gid-claim | $STORAGE_OIDC_GID_CLAIM
: OIDC gid claim.
--network | $STORAGE_AUTH_BEARER_GRPC_NETWORK
: Network to use for the storage service, can be 'tcp', 'udp' or 'unix'. Default: tcp.
--addr | $STORAGE_AUTH_BEARER_GRPC_ADDR
: Address to bind storage service. Default: <IP_ADDRESS>:9148.
storage frontend
Start frontend service
Usage: storage frontend [command options] [arguments...]
--debug-addr | $STORAGE_FRONTEND_DEBUG_ADDR
: Address to bind debug server. Default: <IP_ADDRESS>:9141.
--transfer-secret | $STORAGE_TRANSFER_SECRET
: Transfer secret for datagateway. Default: replace-me-with-a-transfer-secret.
--webdav-namespace | $WEBDAV_NAMESPACE
: Namespace prefix for the /webdav endpoint. Default: /home/.
--dav-files-namespace | $DAV_FILES_NAMESPACE
: Namespace prefix for the webdav /dav/files endpoint. Default: /users/.
--network | $STORAGE_FRONTEND_HTTP_NETWORK
: Network to use for the storage service, can be 'tcp', 'udp' or 'unix'. Default: tcp.
--addr | $STORAGE_FRONTEND_HTTP_ADDR
: Address to bind storage service. Default: <IP_ADDRESS>:9140.
--public-url | $STORAGE_FRONTEND_PUBLIC_URL
: URL to use for the storage service. Default: https://localhost:9200.
--datagateway-prefix | $STORAGE_FRONTEND_DATAGATEWAY_PREFIX
: datagateway prefix. Default: data.
--ocdav-prefix | $STORAGE_FRONTEND_OCDAV_PREFIX
: owncloud webdav endpoint prefix.
--ocs-prefix | $STORAGE_FRONTEND_OCS_PREFIX
: open collaboration services endpoint prefix. Default: ocs.
--gateway-url | $STORAGE_GATEWAY_ENDPOINT
: URL to use for the storage gateway service. Default: localhost:9142.
--upload-disable-tus | $STORAGE_FRONTEND_UPLOAD_DISABLE_TUS
: Disables TUS upload mechanism. Default: false.
--upload-http-method-override | $STORAGE_FRONTEND_UPLOAD_HTTP_METHOD_OVERRIDE
: Specify an HTTP method (ex: POST) that clients should to use when uploading instead of PATCH.
storage gateway
Start gateway
Usage: storage gateway [command options] [arguments...]
--debug-addr | $STORAGE_GATEWAY_DEBUG_ADDR
: Address to bind debug server. Default: <IP_ADDRESS>:9143.
--transfer-secret | $STORAGE_TRANSFER_SECRET
: Transfer secret for datagateway. Default: replace-me-with-a-transfer-secret.
--network | $STORAGE_GATEWAY_GRPC_NETWORK
: Network to use for the storage service, can be 'tcp', 'udp' or 'unix'. Default: tcp.
--addr | $STORAGE_GATEWAY_GRPC_ADDR
: Address to bind storage service. Default: <IP_ADDRESS>:9142.
--endpoint | $STORAGE_GATEWAY_ENDPOINT
: endpoint to use for the storage service. Default: localhost:9142.
--commit-share-to-storage-grant | $STORAGE_GATEWAY_COMMIT_SHARE_TO_STORAGE_GRANT
: Commit shares to the share manager. Default: true.
--commit-share-to-storage-ref | $STORAGE_GATEWAY_COMMIT_SHARE_TO_STORAGE_REF
: Commit shares to the storage. Default: true.
--share-folder | $STORAGE_GATEWAY_SHARE_FOLDER
: mount shares in this folder of the home storage provider. Default: Shares.
--disable-home-creation-on-login | $STORAGE_GATEWAY_DISABLE_HOME_CREATION_ON_LOGIN
: Disable creation of home folder on login.
--auth-basic-endpoint | $STORAGE_AUTH_BASIC_ENDPOINT
: endpoint to use for the basic auth provider. Default: localhost:9146.
--auth-bearer-endpoint | $STORAGE_AUTH_BEARER_ENDPOINT
: endpoint to use for the bearer auth provider. Default: localhost:9148.
--storage-registry-driver | $STORAGE_STORAGE_REGISTRY_DRIVER
: driver of the storage registry. Default: static.
--storage-home-provider | $STORAGE_REGISTRY_HOME_PROVIDER
: mount point of the storage provider for user homes in the global namespace. Default: /home.
--public-url | $STORAGE_FRONTEND_PUBLIC_URL
: URL to use for the storage service. Default: https://localhost:9200.
--datagateway-url | $STORAGE_DATAGATEWAY_PUBLIC_URL
: URL to use for the storage datagateway. Default: https://localhost:9200/data.
--userprovider-endpoint | $STORAGE_USERPROVIDER_ENDPOINT
: endpoint to use for the userprovider. Default: localhost:9144.
--sharing-endpoint | $STORAGE_SHARING_ENDPOINT
: endpoint to use for the storage service. Default: localhost:9150.
--storage-home-endpoint | $STORAGE_STORAGE_HOME_ENDPOINT
: endpoint to use for the home storage. Default: localhost:9154.
--storage-home-mount-path | $STORAGE_STORAGE_HOME_MOUNT_PATH
: mount path. Default: /home.
--storage-home-mount-id | $STORAGE_STORAGE_HOME_MOUNT_ID
: mount id. Default: 1284d238-aa92-42ce-bdc4-0b0000009154.
--storage-users-endpoint | $STORAGE_USERS_ENDPOINT
: endpoint to use for the users storage. Default: localhost:9157.
--storage-users-mount-path | $STORAGE_USERS_MOUNT_PATH
: mount path. Default: /users.
--storage-users-mount-id | $STORAGE_USERS_MOUNT_ID
: mount id. Default: 1284d238-aa92-42ce-bdc4-0b0000009157.
--public-link-endpoint | $STORAGE_STORAGE_PUBLIC_LINK_ENDPOINT
: endpoint to use for the public links service. Default: localhost:9178.
--storage-public-link-mount-path | $STORAGE_STORAGE_PUBLIC_LINK_MOUNT_PATH
: mount path. Default: /public.
storage health
Check health status
Usage: storage health [command options] [arguments...]
--debug-addr | $STORAGE_DEBUG_ADDR
: Address to debug endpoint. Default: <IP_ADDRESS>:9109.
storage sharing
Start sharing service
Usage: storage sharing [command options] [arguments...]
--debug-addr | $STORAGE_SHARING_DEBUG_ADDR
: Address to bind debug server. Default: <IP_ADDRESS>:9151.
--network | $STORAGE_SHARING_GRPC_NETWORK
: Network to use for the storage service, can be 'tcp', 'udp' or 'unix'. Default: tcp.
--addr | $STORAGE_SHARING_GRPC_ADDR
: Address to bind storage service. Default: <IP_ADDRESS>:9150.
--user-driver | $STORAGE_SHARING_USER_DRIVER
: driver to use for the UserShareProvider. Default: json.
--user-json-file | $STORAGE_SHARING_USER_JSON_FILE
: file used to persist shares for the UserShareProvider. Default: /var/tmp/ocis/shares.json.
--public-driver | $STORAGE_SHARING_PUBLIC_DRIVER
: driver to use for the PublicShareProvider. Default: json.
storage storage-users
Start storage-users service
Usage: storage storage-users [command options] [arguments...]
--debug-addr | $STORAGE_USERS_DEBUG_ADDR
: Address to bind debug server. Default: <IP_ADDRESS>:9159.
--grpc-network | $STORAGE_USERS_GRPC_NETWORK
: Network to use for the users storage, can be 'tcp', 'udp' or 'unix'. Default: tcp.
--grpc-addr | $STORAGE_USERS_GRPC_ADDR
: GRPC Address to bind users storage. Default: <IP_ADDRESS>:9157.
--http-network | $STORAGE_USERS_HTTP_NETWORK
: Network to use for the storage service, can be 'tcp', 'udp' or 'unix'. Default: tcp.
--http-addr | $STORAGE_USERS_HTTP_ADDR
: HTTP Address to bind users storage. Default: <IP_ADDRESS>:9158.
--driver | $STORAGE_USERS_DRIVER
: storage driver for users mount: eg. local, eos, owncloud, ocis or s3. Default: ocis.
--mount-path | $STORAGE_USERS_MOUNT_PATH
: mount path. Default: /users.
--mount-id | $STORAGE_USERS_MOUNT_ID
: mount id. Default: 1284d238-aa92-42ce-bdc4-0b0000009162.
--expose-data-server | $STORAGE_USERS_EXPOSE_DATA_SERVER
: exposes a dedicated data server. Default: false.
--data-server-url | $STORAGE_USERS_DATA_SERVER_URL
: data server url. Default: http://localhost:9158/data.
--http-prefix | $STORAGE_USERS_HTTP_PREFIX
: prefix for the http endpoint, without leading slash. Default: data.
--gateway-endpoint | $STORAGE_GATEWAY_ENDPOINT
: endpoint to use for the storage gateway service. Default: localhost:9142.
--users-endpoint | $STORAGE_USERPROVIDER_ENDPOINT
: endpoint to use for the storage service. Default: localhost:9144.
storage users
Start users service
Usage: storage users [command options] [arguments...]
--debug-addr | $STORAGE_SHARING_DEBUG_ADDR
: Address to bind debug server. Default: <IP_ADDRESS>:9145.
--network | $STORAGE_USERPROVIDER_NETWORK
: Network to use for the storage service, can be 'tcp', 'udp' or 'unix'. Default: tcp.
--addr | $STORAGE_USERPROVIDER_ADDR
: Address to bind storage service. Default: <IP_ADDRESS>:9144.
--endpoint | $STORAGE_USERPROVIDER_ENDPOINT
: URL to use for the storage service. Default: localhost:9144.
--driver | $STORAGE_USERPROVIDER_DRIVER
: user driver: 'demo', 'json', 'ldap', or 'rest'. Default: ldap.
--json-config | $STORAGE_USERPROVIDER_JSON
: Path to users.json file.
--rest-client-id | $STORAGE_REST_CLIENT_ID
: User rest driver Client ID.
--rest-client-secret | $STORAGE_REST_CLIENT_SECRET
: User rest driver Client Secret.
--rest-redis-address | $STORAGE_REST_REDIS_ADDRESS
: Address for redis server. Default: localhost:6379.
--rest-redis-username | $STORAGE_REST_REDIS_USERNAME
: Username for redis server.
--rest-redis-password | $STORAGE_REST_REDIS_PASSWORD
: Password for redis server.
--rest-id-provider | $STORAGE_REST_ID_PROVIDER
: The OIDC Provider.
--rest-api-base-url | $STORAGE_REST_API_BASE_URL
: Base API Endpoint.
--rest-oidc-token-endpoint | $STORAGE_REST_OIDC_TOKEN_ENDPOINT
: Endpoint to generate token to access the API.
--rest-target-api | $STORAGE_REST_TARGET_API
: The target application.
storage storage
Storage service for oCIS
Usage: storage storage [command options] [arguments...]
--config-file | $STORAGE_CONFIG_FILE
: Path to config file.
--log-level | $STORAGE_LOG_LEVEL
: Set logging level. Default: info.
--log-pretty | $STORAGE_LOG_PRETTY
: Enable pretty logging.
--log-color | $STORAGE_LOG_COLOR
: Enable colored logging.
storage storage-metadata
Start storage-metadata service
Usage: storage storage-metadata [command options] [arguments...]
--debug-addr | $STORAGE_METADATA_DEBUG_ADDR
: Address to bind debug server. Default: <IP_ADDRESS>:9217.
--grpc-network | $STORAGE_METADATA_GRPC_NETWORK
: Network to use for the storage service, can be 'tcp', 'udp' or 'unix'. Default: tcp.
--grpc-addr | $STORAGE_METADATA_GRPC_PROVIDER_ADDR
: Address to bind storage service. Default: <IP_ADDRESS>:9215.
--data-server-url | $STORAGE_METADATA_DATA_SERVER_URL
: URL of the data-provider the storage-provider uses. Default: http://localhost:9216.
--http-network | $STORAGE_METADATA_HTTP_NETWORK
: Network to use for the storage service, can be 'tcp', 'udp' or 'unix'. Default: tcp.
--http-addr | $STORAGE_METADATA_HTTP_ADDR
: Address to bind storage service. Default: <IP_ADDRESS>:9216.
--driver | $STORAGE_METADATA_DRIVER
: storage driver for metadata mount: eg. local, eos, owncloud, ocis or s3. Default: ocis.
--gateway-endpoint | $STORAGE_GATEWAY_ENDPOINT
: endpoint to use for the gateway service. Default: localhost:9142.
--userprovider-endpoint | $STORAGE_USERPROVIDER_ENDPOINT
: endpoint to use for the userprovider service. Default: localhost:9144.
--storage-root | $STORAGE_METADATA_ROOT
: the path to the metadata storage root. Default: /var/tmp/ocis/metadata.
Config for the different Storage Drivers
You can set different storage drivers for the Storage Providers. Please check the storage provider configuration.
Example: Set the home Storage Provider to ocis
STORAGE_STORAGE_HOME_DRIVER=ocis
STORAGE_STORAGE_HOME_DATA_DRIVER=ocis
Local Driver
--storage-local-root | $STORAGE_STORAGE_LOCAL_ROOT
: the path to the local storage root. Default: /var/tmp/ocis/local.
Eos Driver
--storage-eos-namespace | $STORAGE_STORAGE_EOS_NAMESPACE
: Namespace for metadata operations. Default: /eos/dockertest/reva.
--storage-eos-shadow-namespace | $STORAGE_STORAGE_EOS_SHADOW_NAMESPACE
: Shadow namespace where share references are stored.
--storage-eos-share-folder | $STORAGE_STORAGE_EOS_SHARE_FOLDER
: name of the share folder. Default: /Shares.
--storage-eos-binary | $STORAGE_STORAGE_EOS_BINARY
: Location of the eos binary. Default: /usr/bin/eos.
--storage-eos-xrdcopy-binary | $STORAGE_STORAGE_EOS_XRDCOPY_BINARY
: Location of the xrdcopy binary. Default: /usr/bin/xrdcopy.
--storage-eos-master-url | $STORAGE_STORAGE_EOS_MASTER_URL
: URL of the Master EOS MGM. Default: root://eos-mgm1.eoscluster.cern.ch:1094.
--storage-eos-slave-url | $STORAGE_STORAGE_EOS_SLAVE_URL
: URL of the Slave EOS MGM. Default: root://eos-mgm1.eoscluster.cern.ch:1094.
--storage-eos-cache-directory | $STORAGE_STORAGE_EOS_CACHE_DIRECTORY
: Location on the local fs where to store reads. Default: os.TempDir().
--storage-eos-enable-logging | $STORAGE_STORAGE_EOS_ENABLE_LOGGING
: Enables logging of the commands executed.
--storage-eos-show-hidden-sysfiles | $STORAGE_STORAGE_EOS_SHOW_HIDDEN_SYSFILES
: show internal EOS files like .sys.v# and .sys.a# files..
--storage-eos-force-singleuser-mode | $STORAGE_STORAGE_EOS_FORCE_SINGLEUSER_MODE
: force connections to EOS to use SingleUsername.
--storage-eos-use-keytab | $STORAGE_STORAGE_EOS_USE_KEYTAB
: authenticate requests by using an EOS keytab.
--storage-eos-enable-home | $STORAGE_STORAGE_EOS_ENABLE_HOME
: enable the creation of home directories.
--storage-eos-sec-protocol | $STORAGE_STORAGE_EOS_SEC_PROTOCOL
: the xrootd security protocol to use between the server and EOS.
--storage-eos-keytab | $STORAGE_STORAGE_EOS_KEYTAB
: the location of the keytab to use to authenticate to EOS.
--storage-eos-single-username | $STORAGE_STORAGE_EOS_SINGLE_USERNAME
: the username to use when SingleUserMode is enabled.
--storage-eos-layout | $STORAGE_STORAGE_EOS_LAYOUT
: "layout of the users home dir path on disk, in addition to {{.Username}}, {{.UsernameLower}} and {{.Provider}} also supports prefixing dirs: "{{.UsernamePrefixCount.2}}/{{.UsernameLower}}" will turn "Einstein" into "Ei/Einstein" . Default: {{substr 0 1 .Username}}/{{.Username}}.
--storage-eos-gatewaysvc | $STORAGE_STORAGE_EOS_GATEWAYSVC
: URL to use for the storage gateway service. Default: localhost:9142.
owCloud Driver
--storage-owncloud-datadir | $STORAGE_STORAGE_OWNCLOUD_DATADIR
: the path to the owncloud data directory. Default: /var/tmp/ocis/owncloud.
--storage-owncloud-uploadinfo-dir | $STORAGE_STORAGE_OWNCLOUD_UPLOADINFO_DIR
: the path to the tus upload info directory. Default: /var/tmp/ocis/uploadinfo.
--storage-owncloud-share-folder | $STORAGE_STORAGE_OWNCLOUD_SHARE_FOLDER
: name of the shares folder. Default: /Shares.
--storage-owncloud-scan | $STORAGE_STORAGE_OWNCLOUD_SCAN
: scan files on startup to add fileids. Default: true.
--storage-owncloud-redis | $STORAGE_STORAGE_OWNCLOUD_REDIS_ADDR
: the address of the redis server. Default: :6379.
--storage-owncloud-enable-home | $STORAGE_STORAGE_OWNCLOUD_ENABLE_HOME
: enable the creation of home storages. Default: false.
--storage-owncloud-layout | $STORAGE_STORAGE_OWNCLOUD_LAYOUT
: "layout of the users home dir path on disk, in addition to {{.Username}}, {{.Mail}}, {{.Id.OpaqueId}}, {{.Id.Idp}} also supports prefixing dirs: "{{substr 0 1 .Username}}/{{.Username}}" will turn "Einstein" into "Ei/Einstein" . Default: {{.Id.OpaqueId}}.
Ocis Driver
--storage-ocis-root | $STORAGE_STORAGE_OCIS_ROOT
: the path to the local storage root. Default: /var/tmp/ocis/storage/users.
--storage-ocis-enable-home | $STORAGE_STORAGE_OCIS_ENABLE_HOME
: enable the creation of home storages. Default: false.
--storage-ocis-layout | $STORAGE_STORAGE_OCIS_LAYOUT
: "layout of the users home dir path on disk, in addition to {{.Username}}, {{.Mail}}, {{.Id.OpaqueId}}, {{.Id.Idp}} also supports prefixing dirs: "{{substr 0 1 .Username}}/{{.Username}}" will turn "Einstein" into "Ei/Einstein" . Default: {{.Id.OpaqueId}}.

Cool, thanks!

Makes more sense seeing the final outcome of that change.

is there a problem with drone?

Pipeline is broken because of syntax error.
https://github.com/owncloud/ocis/pull/657/files#diff-74dc60589e0610c07e491b31f8693b2bR1339

Where did you find this? I was trying to find out why drone wouldn't start...

I was just guessing, checking your .drone.star diff

I see. Good guess, thanks!  :1st_place_medal:

But a github admin could find evidence in the hook logs.

Something seems to be wrong with the dataprovider setup
Error="Patch http://ocis-server:9164/data/85beb903-40e0-41ce-8976-c2d536055cf5: dial tcp <IP_ADDRESS>:9164: connect: connection refused```

port 9164 was the oc storage. Doesn't exist anymore. hmm 🤔

Hm, still not fixed... Will continue tomorrow...

The environment variables of the owncloudriver have not been changed...
          'STORAGE_HOME_DRIVER': '%s' % (storage),
           'STORAGE_OCIS_ROOT': '/srv/app/tmp/ocis/storage/users',
           'STORAGE_LOCAL_ROOT': '/srv/app/tmp/ocis/reva/root',
-          'STORAGE_OWNCLOUD_DATADIR': '/srv/app/tmp/ocis/owncloud/data',
-          'STORAGE_OWNCLOUD_REDIS_ADDR': 'redis:6379',
+          'STORAGE_STORAGE_OWNCLOUD_DATADIR': '/srv/app/tmp/ocis/owncloud/data',
+          'STORAGE_STORAGE_OWNCLOUD_REDIS_ADDR': 'redis:6379',
           'STORAGE_OIDC_ISSUER': 'https://ocis-server:9200',
           'STORAGE_LDAP_IDP': 'https://ocis-server:9200',
           'PROXY_OIDC_ISSUER': 'https://ocis-server:9200',
@@ -1327,8 +1327,8 @@ def ocisServer(storage):
         'STORAGE_HOME_DRIVER': '%s' % (storage),
         'STORAGE_OCIS_ROOT': '/srv/app/tmp/ocis/storage/users',
         'STORAGE_LOCAL_ROOT': '/srv/app/tmp/ocis/local/root',
-        'STORAGE_OWNCLOUD_DATADIR': '/srv/app/tmp/ocis/owncloud/data',
-        'STORAGE_OWNCLOUD_REDIS_ADDR': 'redis:6379',
+        'STORAGE_STORAGE_OWNCLOUD_DATADIR': '/srv/app/tmp/ocis/owncloud/data',
+        'STORAGE_STORAGE_OWNCLOUD_REDIS_ADDR': 'redis:6379',
         'STORAGE_LDAP_IDP': 'https://ocis-server:9200',
         'STORAGE_OIDC_ISSUER': 'https://ocis-server:9200',
         'PROXY_OIDC_ISSUER': 'https://ocis-server:9200',

I think documentation about storage-home, storage-users, and the drivers (eos, ocis, owncloud) and the connection or distinction between storages and drivers would be nice. Also about the data servers...
I'm getting a bit confused...

ocis/ocis/tests/acceptance/features/apiOcisSpecific/apiAuthWebDav-webDavPROPFINDAuth.feature:14
Scenario: send PROPFIND requests to another user's webDav endpoints as normal user
No responds with 403 instead of 207. I guess that is fixing the issue: https://github.com/owncloud/ocis-reva/issues/9
@micbar, should I change the scenario to check for 403?

should I change the scenario to check for 403?

The core scenario is https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthWebDav/webDavPROPFINDAuth.feature#L37
That currently passes in oC10, and is expecting HTTP status 404.
So for now, change the local scenario to expect 403. Then there needs to be discussion about which HTTP status is correct for this case - 404 or 403. Only then will we know if it is oC10 or OCIS that is "broken".

https://drone.owncloud.com/owncloud/ocis/606/14/7
  Scenario: Get the mtime of a file inside a folder shared by public link using new webDAV version (run on OCIS)                                 # /drone/src/ocis/tests/acceptance/features/apiOcisSpecific/apiSharePublicLink1-createPublicLinkShare.feature:10
    Given user "Alice" has created folder "testFolder"                                                                                # FeatureContext::userHasCreatedFolder()
    And user "Alice" has created a public link share with settings                                                                               # FeatureContext::userHasCreatedAPublicLinkShareWithSettings()
      | path        | /testFolder               |
      | permissions | read,update,create,delete |
    When the public uploads file "file.txt" to the last shared folder with mtime "Thu, 08 Aug 2019 04:18:13 GMT" using the new public WebDAV API # PublicWebDavContext::thePublicUploadsFileToLastSharedFolderWithMtimeUsingTheWebdavApi()
    Then as "Alice" file "testFolder/file.txt" should exist                                                                                # FeatureContext::asFileOrFolderShouldExist()
      WebDav::listFolderAndReturnResponseXml Received empty response where XML was expected (Exception)

I am also trying to see why this happens in https://github.com/cs3org/reva/pull/1236
That PR is to update the core test commit id in cs3org/reva and using the latest core test code I start getting problems with Received empty response. I think that the newer test code is now checking more things about responses, and so maybe some things start to fail when previously the "problem" was not noticed. I will look more tomorrow. I suspect that when I find the issue in cs3org/reva then it will help here.

Ah seems like that is related. That would be great.
I'm also off for today...

2020-10-13T15:39:46Z ERR datasvc: file not found error="error: not found: 6e19c61d-2fd0-43c0-95c7-ce74f384949e/Alice" pkg=rhttp service=storage traceid=e307e8639e84ad79b2c0742ab31462c4
I guess this error is happening because the Id passed to LoadAccount is 6e19c61d-2fd0-43c0-95c7-ce74f384949e/Alice but should be 6e19c61d-2fd0-43c0-95c7-ce74f384949e without /Alice.
Now the question is where do we need changes?

Nevermind the last comment, it should be correct.

For some reason the public link storage stops running sometimes.
process [storage-public-link] exited with code: 0 this is outputted during the startup of the ocis binary

@micbar @phil-davis, can the acceptance tests also output the timestamps when they run? This would help immensely for debugging.

Manually tesing public links locally showed me this:
2020-10-14T18:54:13+02:00 ERR home/corby/Development/go/pkg/mod/github.com/cs3org/reva@v1.3.1-0.20201009145025-fb08084ce5d4/internal/grpc/services/storageregistry/storageregistry.go:127 > error finding storage provider error="error: not found: storage provider not found for ref id:<storage_id:\"1284d238-aa92-42ce-bdc4-0b0000009162\" opaque_id:\"7fdcaad3-09c0-4f1f-b5de-110a76dbad82\" > " pkg=rgrpc service=storage traceid=6e391d4c17439f67d2e6e25324778b70
2020-10-14T18:54:13+02:00 INF unary code=OK end="14/Oct/2020:18:54:13 +0200" from=tcp://[::1]:39558 pkg=rgrpc service=storage start="14/Oct/2020:18:54:13 +0200" time_ns=166780 traceid=6e391d4c17439f67d2e6e25324778b70 uri=/cs3.storage.registry.v1beta1.RegistryAPI/GetStorageProvider user-agent=grpc-go/1.26.0
2020-10-14T18:54:13+02:00 ERR home/corby/Development/go/pkg/mod/github.com/cs3org/reva@v1.3.1-0.20201009145025-fb08084ce5d4/internal/grpc/services/gateway/storageprovider.go:1283 > error finding storage provider error="gateway: grpc failed with code CODE_INTERNAL" pkg=rgrpc service=storage traceid=6e391d4c17439f67d2e6e25324778b70
2020-10-14T18:54:13+02:00 ERR home/corby/Development/go/pkg/mod/github.com/cs3org/reva@v1.3.1-0.20201009145025-fb08084ce5d4/internal/grpc/services/gateway/storageprovider.go:676 > error error stating ref:id:<storage_id:"1284d238-aa92-42ce-bdc4-0b0000009162" opaque_id:"7fdcaad3-09c0-4f1f-b5de-110a76dbad82" >  error="gateway: grpc failed with code CODE_INTERNAL" pkg=rgrpc service=storage traceid=6e391d4c17439f67d2e6e25324778b70
2020-10-14T18:54:13+02:00 INF unary code=OK end="14/Oct/2020:18:54:13 +0200" from=tcp://[::1]:39036 pkg=rgrpc service=storage start="14/Oct/2020:18:54:13 +0200" time_ns=648073 traceid=6e391d4c17439f67d2e6e25324778b70 uri=/cs3.gateway.v1beta1.GatewayAPI/GetPath user-agent=grpc-go/1.26.0
goroutine 236 [running]:
runtime/debug.Stack(0xc00012be80, 0xc0006e17a0, 0xbfd9e92d5ec8ea1e)
	/usr/lib/go/src/runtime/debug/stack.go:24 +0x9f
runtime/debug.PrintStack()
	/usr/lib/go/src/runtime/debug/stack.go:16 +0x25
github.com/cs3org/reva/internal/grpc/interceptors/recovery.recoveryFunc(0x3b6a620, 0xc001c02810, 0x31f5c20, 0xc001874860, 0x1, 0x4dd5ad8)
	/home/corby/Development/go/pkg/mod/github.com/cs3org/reva@v1.3.1-0.20201009145025-fb08084ce5d4/internal/grpc/interceptors/recovery/recovery.go:48 +0x26
github.com/grpc-ecosystem/go-grpc-middleware/recovery.recoverFrom(0x3b6a620, 0xc001c02810, 0x31f5c20, 0xc001874860, 0x3868b70, 0xc0008b4be0, 0x1372226)
	/home/corby/Development/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.2.2/recovery/interceptors.go:61 +0x5a
github.com/grpc-ecosystem/go-grpc-middleware/recovery.UnaryServerInterceptor.func1.1(0xc0008b5456, 0x3b6a620, 0xc001c02810, 0xc00137c090, 0xc0008b54d8)
	/home/corby/Development/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.2.2/recovery/interceptors.go:29 +0x6d
panic(0x31f5c20, 0xc001874860)
	/usr/lib/go/src/runtime/panic.go:969 +0x175
github.com/cs3org/reva/internal/grpc/services/gateway.(*svc).split(0xc00137a000, 0x3b6a620, 0xc001c02b10, 0x4dd5ad8, 0x1, 0x2, 0x0)
	/home/corby/Development/go/pkg/mod/github.com/cs3org/reva@v1.3.1-0.20201009145025-fb08084ce5d4/internal/grpc/services/gateway/storageprovider.go:1986 +0x4ee
github.com/cs3org/reva/internal/grpc/services/gateway.(*svc).isSharedFolder(...)
	/home/corby/Development/go/pkg/mod/github.com/cs3org/reva@v1.3.1-0.20201009145025-fb08084ce5d4/internal/grpc/services/gateway/storageprovider.go:1961
github.com/cs3org/reva/internal/grpc/services/gateway.(*svc).Stat(0xc00137a000, 0x3b6a620, 0xc001c02b10, 0xc0007e24b0, 0xc00137a000, 0x31f05e0, 0x3ac4ea8)
	/home/corby/Development/go/pkg/mod/github.com/cs3org/reva@v1.3.1-0.20201009145025-fb08084ce5d4/internal/grpc/services/gateway/storageprovider.go:1315 +0x658
github.com/cs3org/go-cs3apis/cs3/gateway/v1beta1._GatewayAPI_Stat_Handler.func1(0x3b6a620, 0xc001c02b10, 0x361e720, 0xc0007e24b0, 0x36b86a0, 0xc00012bf00, 0x3b6a620, 0xc001c02b10)
	/home/corby/Development/go/pkg/mod/github.com/cs3org/go-cs3apis@v0.0.0-20200929101248-821df597ec8d/cs3/gateway/v1beta1/gateway_api.pb.go:2673 +0x8b
github.com/cs3org/reva/internal/grpc/interceptors/auth.NewUnary.func1(0x3b6a620, 0xc001c02810, 0x361e720, 0xc0007e24b0, 0xc000693700, 0xc000693720, 0x0, 0x0, 0x0, 0x0)
	/home/corby/Development/go/pkg/mod/github.com/cs3org/reva@v1.3.1-0.20201009145025-fb08084ce5d4/internal/grpc/interceptors/auth/auth.go:96 +0x31a
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1(0x3b6a620, 0xc001c02810, 0x361e720, 0xc0007e24b0, 0x13bd618, 0xc001c023f0, 0x8, 0x5164400)
	/home/corby/Development/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.2.2/chain.go:25 +0x63
github.com/grpc-ecosystem/go-grpc-middleware/recovery.UnaryServerInterceptor.func1(0x3b6a620, 0xc001c02810, 0x361e720, 0xc0007e24b0, 0xc000693700, 0xc000693740, 0x0, 0x0, 0x0, 0x0)
	/home/corby/Development/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.2.2/recovery/interceptors.go:33 +0xcb
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1(0x3b6a620, 0xc001c02810, 0x361e720, 0xc0007e24b0, 0xc000184400, 0x7f9053521e98, 0x0, 0x7f905352bc20)
	/home/corby/Development/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.2.2/chain.go:25 +0x63
github.com/cs3org/reva/internal/grpc/interceptors/log.NewUnary.func1(0x3b6a620, 0xc001c02810, 0x361e720, 0xc0007e24b0, 0xc000693700, 0xc000693760, 0x0, 0xc000693800, 0x1, 0x1)
	/home/corby/Development/go/pkg/mod/github.com/cs3org/reva@v1.3.1-0.20201009145025-fb08084ce5d4/internal/grpc/interceptors/log/log.go:39 +0x9e
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1(0x3b6a620, 0xc001c02810, 0x361e720, 0xc0007e24b0, 0x2, 0x3b6a620, 0xc001c02810, 0xc001c027e0)
	/home/corby/Development/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.2.2/chain.go:25 +0x63
github.com/cs3org/reva/internal/grpc/interceptors/token.NewUnary.func1(0x3b6a620, 0xc001c027b0, 0x361e720, 0xc0007e24b0, 0xc000693700, 0xc000693780, 0x371fc40, 0xc001705680, 0x3b6a620, 0xc001c027b0)
	/home/corby/Development/go/pkg/mod/github.com/cs3org/reva@v1.3.1-0.20201009145025-fb08084ce5d4/internal/grpc/interceptors/token/token.go:44 +0xb4
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1(0x3b6a620, 0xc001c027b0, 0x361e720, 0xc0007e24b0, 0xc001c027b0, 0xc0012f4a00, 0x4f, 0x1f4)
	/home/corby/Development/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.2.2/chain.go:25 +0x63
github.com/cs3org/reva/internal/grpc/interceptors/appctx.NewUnary.func1(0x3b6a620, 0xc001c02750, 0x361e720, 0xc0007e24b0, 0xc000693700, 0xc0006937a0, 0x299ecda, 0x353fce0, 0xc0006937c0, 0xc000693700)
	/home/corby/Development/go/pkg/mod/github.com/cs3org/reva@v1.3.1-0.20201009145025-fb08084ce5d4/internal/grpc/interceptors/appctx/appctx.go:36 +0x576
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1(0x3b6a620, 0xc001c02750, 0x361e720, 0xc0007e24b0, 0xc0007e0200, 0xc0006936c0, 0xc000609b08, 0x1311d78)
	/home/corby/Development/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.2.2/chain.go:25 +0x63
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1(0x3b6a620, 0xc001c02750, 0x361e720, 0xc0007e24b0, 0xc000693700, 0xc000693720, 0xc00008fb78, 0x13bd1a6, 0x35a7ae0, 0xc001c02750)
	/home/corby/Development/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.2.2/chain.go:34 +0xd7
github.com/cs3org/go-cs3apis/cs3/gateway/v1beta1._GatewayAPI_Stat_Handler(0x37972a0, 0xc00137a000, 0x3b6a620, 0xc001c02750, 0xc0017055c0, 0xc0013337a0, 0x3b6a620, 0xc001c02750, 0xc00014ebb8, 0x5)
	/home/corby/Development/go/pkg/mod/github.com/cs3org/go-cs3apis@v0.0.0-20200929101248-821df597ec8d/cs3/gateway/v1beta1/gateway_api.pb.go:2675 +0x150
google.golang.org/grpc.(*Server).processUnaryRPC(0xc001376180, 0x3b85b40, 0xc000001c80, 0xc0007e0200, 0xc001333950, 0x4d79f48, 0x0, 0x0, 0x0)
	/home/corby/Development/go/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:1024 +0x522
google.golang.org/grpc.(*Server).handleStream(0xc001376180, 0x3b85b40, 0xc000001c80, 0xc0007e0200, 0x0)
	/home/corby/Development/go/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:1313 +0xd34
google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc0003fc090, 0xc001376180, 0x3b85b40, 0xc000001c80, 0xc0007e0200)
	/home/corby/Development/go/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:722 +0xa5
created by google.golang.org/grpc.(*Server).serveStreams.func1
	/home/corby/Development/go/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:720 +0xa5
2020-10-14T18:54:13+02:00 ERR split: len(parts) < 2: path:/ parts:[] pkg=rgrpc service=storage traceid=6e391d4c17439f67d2e6e25324778b70
2020-10-14T18:54:13+02:00 ERR unary code=Internal end="14/Oct/2020:18:54:13 +0200" from=tcp://[::1]:39036 pkg=rgrpc service=storage start="14/Oct/2020:18:54:13 +0200" time_ns=328485 traceid=6e391d4c17439f67d2e6e25324778b70 uri=/cs3.gateway.v1beta1.GatewayAPI/Stat user-agent=grpc-go/1.26.0
2020-10-14T18:54:13+02:00 ERR home/corby/Development/go/pkg/mod/github.com/cs3org/reva@v1.3.1-0.20201009145025-fb08084ce5d4/internal/grpc/services/publicstorageprovider/publicstorageprovider.go:384 > gateway: error calling Stat for ref:path:"/public/aUExLCHEjzPSAGr"  error="rpc error: code = Internal desc = split: len(parts) < 2: path:/ parts:[]" pkg=rgrpc service=storage traceid=6e391d4c17439f67d2e6e25324778b70

So that is something to continue for tomorrow.

The public link storage is called with an id reference but the storage only handles path references.

This code is the beginning of where it goes wrong.
pathRes, err := s.gateway.GetPath(ctx, &provider.GetPathRequest{
		ResourceId: publicShareResponse.GetShare().GetResourceId(),
})

The resource id looks like this:
<storage_id:"1284d238-aa92-42ce-bdc4-0b0000009162" opaque_id:"7fdcaad3-09c0-4f1f-b5de-110a76dbad82" >

It contains a storage_id and a opaque_id but no path. Because of that the storage registry tries to find the storage by the storage id. The storage for the public links isn't registered by an id though but by a path.
I need to look into the GetPath method tomorrow.

Now that is interesting. Somehow changing the mount id of the home storage made the tests get stuck... at least they are not red... :hankey:

Some small clarification about the mount ids and especially this comment
// This is the mount id of the storage provider using the same storage driver
// as /home but withoud home enabled.

would be nice.
Why is the mount id in the storage-home flagset different than the one in the gateway flagset?

I think I just found a never ending loop in the ocis storage.
That could explain why the tests get stuck at some point.
2020-10-15T14:49:27+02:00 DBG ReadNode() found parent node={"Exists":false,"ID":"cb71fc26-decb-443f-a70d-dc2441f4ba42","Name":"myChecksumFile.txt","ParentID":"8d2de3d2-756d-4395-993e-700d076818f5"} parentID=8d2de3d2-756d-4395-993e-700d076818f5 pkg=rgrpc root.ID=9c37fe9f-b42d-43c3-9a29-498cbac9e7aa service=storage traceid=651e06548d24bf31cf3c0c967f93bfbe
2020-10-15T14:49:27+02:00 DBG ReadNode() node={"Exists":false,"ID":"cb71fc26-decb-443f-a70d-dc2441f4ba42","Name":"myChecksumFile.txt","ParentID":"8d2de3d2-756d-4395-993e-700d076818f5"} pkg=rgrpc root.ID=9c37fe9f-b42d-43c3-9a29-498cbac9e7aa service=storage traceid=651e06548d24bf31cf3c0c967f93bfbe
2020-10-15T14:49:27+02:00 DBG ReadNode() found parent node={"Exists":false,"ID":"cb71fc26-decb-443f-a70d-dc2441f4ba42","Name":"myChecksumFile.txt","ParentID":"8d2de3d2-756d-4395-993e-700d076818f5"} parentID=root pkg=rgrpc root.ID=9c37fe9f-b42d-43c3-9a29-498cbac9e7aa service=storage traceid=651e06548d24bf31cf3c0c967f93bfbe
2020-10-15T14:49:27+02:00 DBG ReadNode() node={"Exists":false,"ID":"cb71fc26-decb-443f-a70d-dc2441f4ba42","Name":"myChecksumFile.txt","ParentID":"8d2de3d2-756d-4395-993e-700d076818f5"} pkg=rgrpc root.ID=9c37fe9f-b42d-43c3-9a29-498cbac9e7aa service=storage traceid=651e06548d24bf31cf3c0c967f93bfbe
2020-10-15T14:49:27+02:00 DBG ReadNode() found parent node={"Exists":false,"ID":"cb71fc26-decb-443f-a70d-dc2441f4ba42","Name":"myChecksumFile.txt","ParentID":"8d2de3d2-756d-4395-993e-700d076818f5"} parentID= pkg=rgrpc root.ID=9c37fe9f-b42d-43c3-9a29-498cbac9e7aa service=storage traceid=651e06548d24bf31cf3c0c967f93bfbe

uhh

Also, I think we need to change some config for the acceptance tests with the ocis storage.
The OCIS_REVA_DATA_ROOT variable needs to be changed.

Can't we just remove the block of code in lookup.go:117-119
if node, err = lu.RootNode(ctx); err != nil {
		return
}

lu.RootNodealways returns nil for the error.

So I did some more debugging and found out that:
The rootID is not "root" but instead a uuid a40c77f6-9be6-4b64-bee1-9ae809977170. Which is expected when /home is enabled.
But when we look through the logs from ReadNode() we can see in the second line that the found parentID is root which means the files parent is the root node. That also means that the loops condition will never be true since "a40c77f6-9be6-4b64-bee1-9ae809977170" != "root" and the root node doesn't have a parent node.
2020-10-16T12:50:01+02:00 DBG ReadNode() node={"Exists":false,"ID":"f4138423-70f8-4ed1-8b26-6fc9e4db2b35","Name":"textfile0.txt","ParentID":"254eda5c-fbcc-4fb8-bd5c-687171f63e50"} pkg=rgrpc root.ID=a40c77f6-9be6-4b64-bee1-9ae809977170 service=storage traceid=c340b6ba95eb0be2431ab4d44a036205
2020-10-16T12:50:01+02:00 DBG ReadNode() found parent node={"Exists":false,"ID":"f4138423-70f8-4ed1-8b26-6fc9e4db2b35","Name":"textfile0.txt","ParentID":"254eda5c-fbcc-4fb8-bd5c-687171f63e50"} parentID=root pkg=rgrpc root.ID=a40c77f6-9be6-4b64-bee1-9ae809977170 service=storage traceid=c340b6ba95eb0be2431ab4d44a036205
2020-10-16T12:50:01+02:00 DBG ReadNode() node={"Exists":false,"ID":"f4138423-70f8-4ed1-8b26-6fc9e4db2b35","Name":"textfile0.txt","ParentID":"254eda5c-fbcc-4fb8-bd5c-687171f63e50"} pkg=rgrpc root.ID=a40c77f6-9be6-4b64-bee1-9ae809977170 service=storage traceid=c340b6ba95eb0be2431ab4d44a036205
2020-10-16T12:50:01+02:00 DBG ReadNode() found parent node={"Exists":false,"ID":"f4138423-70f8-4ed1-8b26-6fc9e4db2b35","Name":"textfile0.txt","ParentID":"254eda5c-fbcc-4fb8-bd5c-687171f63e50"} parentID= pkg=rgrpc root.ID=a40c77f6-9be6-4b64-bee1-9ae809977170 service=storage traceid=c340b6ba95eb0be2431ab4d44a036205

Ok, I'm not getting anywhere...
I will leave it at that for now and hope that @butonic can help us on monday to make the tests pass.
We also shouldn't forget to remove the workaround I added here: https://github.com/owncloud/ocis/pull/657/files#diff-e8d2021e168980de8a688c0345fcf4c86e40e59ac60d90f942e698dd1684ad69R247-R248

I think the scenarios
apiShareReshareToShares3/reShareUpdate.feature:42
apiShareReshareToShares3/reShareUpdate.feature:43
apiShareReshareToShares3/reShareUpdate.feature:76
apiShareReshareToShares3/reShareUpdate.feature:77
apiShareReshareToShares3/reShareUpdate.feature:93
apiShareReshareToShares3/reShareUpdate.feature:94

fail because when updating a share the data in the shares.json gets updated but the share reference not.
E.g.
Brian shares /TMP with Carol.
When Carol accepts the share in her directory there will be a reference created like /Shares/TMP/ -> /<Brian's Home>/TMP/. The reference holds Carol's grants.
Now when the share gets updated the reference stays the same with the original grants.
@butonic, does that make sense?

Wait a moment... why does it work with the owncloud storage?

The last remaining failing tests should be fixed by those PRs:
https://github.com/cs3org/reva/pull/1258
https://github.com/cs3org/reva/pull/1259

Oh ffs... now apiWebdavEtagPropagation1/moveFileFolder.feature:20 is switching between passing and failing...

Oh I hope it'll work.

🥳


Feature request: Highlight nodes with no dependencies or x number of dependencies

Summary
Using this tool to help port libraries from one framework to another.
Highlighting nodes with zero dependencies shows which files to port first.
Details
Could be enabled by toggle or text box with input in the Dependencies configuration tab.
Highlight could be in green - or any color.
Standard questions
Would you consider contributing a PR? Yes

Nevermind, realised in Graph Configuration sidebar select layout Hierarchical, spacing Directed
The bottom layer has no import locally


Connection tracking doesn't work when IPv6 is disabled

See https://github.com/st3v/weave-scope-release/issues/3

Thanks for looking into this!

@2opremio For testing, once https://github.com/kinvolk/stage1-builder/issues/23 is done, we could use RKT_HYPERVISOR_EXTRA_KERNEL_PARAMS=ipv6.disable=1 in semaphore.sh when rkt is called.

@2opremio what's left to do here?

We are now tolerant towards ipV6 being disabled in the /proc walker but not in the eBPF connection tracker.


Age backend: ssh password requested twice

Summary

Steps To Reproduce

./gopass init --storage fs --crypto age
select a private key for encrypting secrets  age - ssh-rsa (this key is password protected
create a new account for titi.com ./gopass create you need to provide the (ssh key paswword one time)
display the password ./gopass show titi

The Issue in the last step is that you need to enter your ssh password 2 times
Expected behavior
Enter the ssh password only one time
Environment

OS:Ubuntu 18.04.5 LTS
gopass Version:

gopass 1.10.0+a6031515 (a6031515 2020-08-24 11:09:18) go1.15 linux amd64
<root>     -  age 0.0.1 -   fs 0.1.0
Available Crypto Backends: age, gpgcli, plain
Available Storage Backends: fs, gitfs, ondisk

Installation method: source

Additional context

Yes, this is currently a limitation. Thanks for reporting this.
The reason for that is that both the age keyring as well as the secret are encrypted.
We'll eventually want to fix this, but this is rather low priority.
However please continue to test the new backend and report bugs.

This seems to have been fixed aready, probably with #2060
Closing.

Yes, this should definitely be fixed by now. Thanks for cleaning up this issue.


service: communicate with other instances

Such as the removed peer service removed in commit 774a2b9dfd257cfea83488599571718217937fc5.

Prototype implemented in https://github.com/gate-computer/python-service/blob/master/prototype/peer.py.


mesh default cell data

STYLE: Move type declaration into itkMeshCellDataTest function
BUG: Create CellData when requested in Mesh::GetCellData

\azp run ITK.macOS

This is a spurious failure.


Support Kick and Rumble streams

Since Destiny has started streaming on these platforms it would be nice to include them in the app.
I updated the compileSdkVersion and the firebase dependency override because it wouldn't compile otherwise, I don't know if it's a problem of only my configuration

Thanks for implementing the new embeds! I'll merge this now and do some cleanup related to new Flutter versions. Once that's done I will push a new version to the Play Store.


Release 0.9.0

Improvements landed in the develop branch since 0.8.0:

timezone aware datetime object support
read-only access to saved state of the Task in the database
several bug fixes

Released! https://pypi.python.org/pypi/tasklib/0.9.0