l500-depth.cpp:363) UVC metadata payloads are not available for stream . Please refer to installation chapter for details.

Required Info

Camera Model
L515

Firmware Version
<IP_ADDRESS>

Operating System & Version
Ubuntu 18.04.6 LTS

Kernel Version (Linux Only)
4.9.201-tegra

Platform
NVIDIA Jetson AGX Xavier with connecttect board Rogue Carrier and ARMv8 Processor

SDK Version
2

Language
c++/ROS

Segment
ROS

Issue Description
I am facing an issue with launching L515 either on realsense-viewer or ROS. I looked on the former issues there were no specific solution. The problem is primarily with streaming Depth-image.
As I ran

Ubuntu 14/16/18/20 with LTS kernel
./scripts/patch-realsense-ubuntu-lts.sh
in the source

I got
Package required libusb-1.0-0-dev:  - found
Package required libssl-dev:  - found
Unsupported kernel version 4.9.201-tegra . The Bionic patches are maintained for Bionic Beaver LTS kernels 4.1[5/8], 5.[0/3/4]
Is this the reason why I get the error? Anyway, what is the fix?
Thanks for support

Hi @hichamhendy  Jetson boards have their own dedicated patch script called ./scripts/patch-realsense-ubuntu-L4T.sh
https://github.com/IntelRealSense/librealsense/blob/master/doc/installation_jetson.md#building-from-source-using-native-backend
You can also build librealsense from source code with the RSUSB backend installation method, which is not dependent on Linux versions or kernel versions and does not require patching.  Instructions for doing so on Jetson can be foubd at https://github.com/IntelRealSense/librealsense/issues/6964#issuecomment-707501049

regarding ./scripts/patch-realsense-ubuntu-L4T.sh
I get
Jetson Board (proc/device-tree/model): jetson-xavier
Jetson L4T version: 32.5
L4T 32.5 to use patches revision 4.4.1
Retrieve the corresponding L4T git tag the kernel source tree
Directory ../linux-4.9-source-tree is present, skipping initialization...
/media/drakula/ssd/Libraries/linux-4.9-source-tree /media/drakula/ssd/Libraries/librealsense
fatal: unable to connect to nv-tegra.nvidia.com:
nv-tegra.nvidia.com[0: <IP_ADDRESS>]: errno=Connection timed out
nv-tegra.nvidia.com[1: <IP_ADDRESS>]: errno=Connection timed out
The matching L4T source tree tag is
/media/drakula/ssd/Libraries/librealsense
Create the sandbox - NVidia L4T source tree(s)
Downloading default kernel/kernel-4.9 source...
fatal: repository '/media/drakula/ssd/Libraries/librealsense/Tegra/sources/kernel/kernel-4.9' does not exist
/media/drakula/ssd/Libraries/librealsense/Tegra/sources/kernel/kernel-4.9 source sync failed!
Downloading default kernel/nvgpu source...
fatal: repository '/media/drakula/ssd/Libraries/librealsense/Tegra/sources/kernel/nvgpu' does not exist
/media/drakula/ssd/Libraries/librealsense/Tegra/sources/kernel/nvgpu source sync failed!
Downloading default kernel/nvidia source...
fatal: repository '/media/drakula/ssd/Libraries/librealsense/Tegra/sources/kernel/nvidia' does not exist
/media/drakula/ssd/Libraries/librealsense/Tegra/sources/kernel/nvidia source sync failed!
Downloading default hardware/nvidia/soc/t18x source...
fatal: repository '/media/drakula/ssd/Libraries/librealsense/Tegra/sources/hardware/nvidia/soc/t18x' does not exist
/media/drakula/ssd/Libraries/librealsense/Tegra/sources/hardware/nvidia/soc/t18x source sync failed!
Downloading default hardware/nvidia/platform/tegra/common source...
fatal: repository '/media/drakula/ssd/Libraries/librealsense/Tegra/sources/hardware/nvidia/platform/tegra/common' does not exist
/media/drakula/ssd/Libraries/librealsense/Tegra/sources/hardware/nvidia/platform/tegra/common source sync failed!
Downloading default hardware/nvidia/platform/t18x/common source...
fatal: repository '/media/drakula/ssd/Libraries/librealsense/Tegra/sources/hardware/nvidia/platform/t18x/common' does not exist
/media/drakula/ssd/Libraries/librealsense/Tegra/sources/hardware/nvidia/platform/t18x/common source sync failed!
Downloading default hardware/nvidia/platform/t18x/quill source...
fatal: repository '/media/drakula/ssd/Libraries/librealsense/Tegra/sources/hardware/nvidia/platform/t18x/quill' does not exist
/media/drakula/ssd/Libraries/librealsense/Tegra/sources/hardware/nvidia/platform/t18x/quill source sync failed!
Downloading default hardware/nvidia/soc/t210 source...
fatal: repository '/media/drakula/ssd/Libraries/librealsense/Tegra/sources/hardware/nvidia/soc/t210' does not exist
/media/drakula/ssd/Libraries/librealsense/Tegra/sources/hardware/nvidia/soc/t210 source sync failed!
Downloading default hardware/nvidia/platform/t210/common source...
fatal: repository '/media/drakula/ssd/Libraries/librealsense/Tegra/sources/hardware/nvidia/platform/t210/common' does not exist
/media/drakula/ssd/Libraries/librealsense/Tegra/sources/hardware/nvidia/platform/t210/common source sync failed!
Downloading default hardware/nvidia/platform/t210/jetson source...
fatal: repository '/media/drakula/ssd/Libraries/librealsense/Tegra/sources/hardware/nvidia/platform/t210/jetson' does not exist
/media/drakula/ssd/Libraries/librealsense/Tegra/sources/hardware/nvidia/platform/t210/jetson source sync failed!
Downloading default hardware/nvidia/platform/t210/porg source...
fatal: repository '/media/drakula/ssd/Libraries/librealsense/Tegra/sources/hardware/nvidia/platform/t210/porg' does not exist
/media/drakula/ssd/Libraries/librealsense/Tegra/sources/hardware/nvidia/platform/t210/porg source sync failed!
Downloading default hardware/nvidia/soc/t19x source...
fatal: repository '/media/drakula/ssd/Libraries/librealsense/Tegra/sources/hardware/nvidia/soc/t19x' does not exist
/media/drakula/ssd/Libraries/librealsense/Tegra/sources/hardware/nvidia/soc/t19x source sync failed!
Downloading default hardware/nvidia/platform/t19x/common source...
fatal: repository '/media/drakula/ssd/Libraries/librealsense/Tegra/sources/hardware/nvidia/platform/t19x/common' does not exist
/media/drakula/ssd/Libraries/librealsense/Tegra/sources/hardware/nvidia/platform/t19x/common source sync failed!
Downloading default hardware/nvidia/platform/t19x/galen/kernel-dts source...
fatal: repository '/media/drakula/ssd/Libraries/librealsense/Tegra/sources/hardware/nvidia/platform/t19x/galen/kernel-dts' does not exist
/media/drakula/ssd/Libraries/librealsense/Tegra/sources/hardware/nvidia/platform/t19x/galen/kernel-dts source sync failed!
Downloading default hardware/nvidia/platform/t19x/jakku/kernel-dts source...
fatal: repository '/media/drakula/ssd/Libraries/librealsense/Tegra/sources/hardware/nvidia/platform/t19x/jakku/kernel-dts' does not exist
/media/drakula/ssd/Libraries/librealsense/Tegra/sources/hardware/nvidia/platform/t19x/jakku/kernel-dts source sync failed!
Downloading default hardware/nvidia/soc/tegra source...
fatal: repository '/media/drakula/ssd/Libraries/librealsense/Tegra/sources/hardware/nvidia/soc/tegra' does not exist
/media/drakula/ssd/Libraries/librealsense/Tegra/sources/hardware/nvidia/soc/tegra source sync failed!
./Tegra/sources/kernel/kernel-4.9
./scripts/patch-realsense-ubuntu-L4T.sh: line 104: pushd: ./Tegra/sources/kernel/kernel-4.9: No such file or directory

Understood
I think the problem now is that cmake can't finde CUDA Compiler
cmake .. -DBUILD_EXAMPLES=true -DCMAKE_BUILD_TYPE=release -DFORCE_RSUSB_BACKEND=false -DBUILD_WITH_CUDA=true
-- Checking internet connection...
-- Internet connection identified
-- Info: REALSENSE_VERSION_STRING=2.50.0
-- Setting Unix configurations
-- Info: Building with CUDA requires CMake v3.8+
-- The CUDA compiler identification is unknown
CMake Error at CMake/cuda_config.cmake:3 (project):
No CMAKE_CUDA_COMPILER could be found.
Tell CMake where to find the compiler by setting either the environment
variable "CUDACXX" or the CMake cache entry CMAKE_CUDA_COMPILER to the full
path to the compiler, or to the compiler name if it is in the PATH.
Call Stack (most recent call first):
CMake/global_config.cmake:73 (include)
CMakeLists.txt:48 (global_set_flags)
-- Configuring incomplete, errors occurred!
See also "/media/drakula/ssd/Libraries/librealsense-2.50.0/build/CMakeFiles/CMakeOutput.log".
See also "/media/drakula/ssd/Libraries/librealsense-2.50.0/build/CMakeFiles/CMakeError.log".

Problem solved
just followed instructions here

That's great to hear that https://github.com/IntelRealSense/librealsense/issues/6964#issuecomment-667569588 worked for you, @hichamhendy - thanks very much for the update   :)


can someone please let me know what's failing my application to run on Heroku

I have created an issue with all the detailed logs
I can see there are some fail logs for kestrel but don't understand what's causing them. Why they fail only in case of yeoman created project & not in case of dotnet new -t web
Please see this issue & let me know thanks a lot.
Also link to app just in case: ivdstudios.herokuapp.com
Runs perfectly fine on local server
⇨ dotnet run                                                                              ✔ master
Project ivdstudios (.NETCoreApp,Version=v1.1) will be compiled because project is not safe for incremental compilation. Use --build-profile flag for more information.
Compiling ivdstudios for .NETCoreApp,Version=v1.1
Bundling with configuration from /Users/abhimanyuaryan/Unity3D/DotNetCore/ivdstudios/bundleconfig.json
Processing wwwroot/css/site.min.css
Processing wwwroot/js/site.min.js
Processing wwwroot/js/semantic.validation.min.js

Compilation succeeded.
    0 Warning(s)
    0 Error(s)

Time elapsed 00:00:02.5516152
 (The compilation time can be improved. Run "dotnet build --build-profile" for more information)

Hosting environment: Production
Content root path: /Users/abhimanyuaryan/Unity3D/DotNetCore/ivdstudios
Now listening on: http://localhost:5000
Application started. Press Ctrl+C to shut down.
info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
      Request starting HTTP/1.1 GET http://localhost:5000/
info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[1]
      Executing action method ivdstudios.Controllers.HomeController.Index (ivdstudios) with arguments ((null)) - ModelState is Valid
info: Microsoft.AspNetCore.Mvc.ViewFeatures.Internal.ViewResultExecutor[1]
      Executing ViewResult, running view at path /Views/Home/Index.cshtml.
info: Microsoft.Extensions.DependencyInjection.DataProtectionServices[0]
      User profile is available. Using '/Users/abhimanyuaryan/.aspnet/DataProtection-Keys' as key repository; keys will not be encrypted at rest.
info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[2]
      Executed action ivdstudios.Controllers.HomeController.Index (ivdstudios) in 1878.4716ms
info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
      Request starting HTTP/1.1 GET http://localhost:5000/css/site.min.css?v=eGLdvn2tBaqnV-bHovKaeY6XQXNFe2SHC6zPXyhyajI
info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
      Request starting HTTP/1.1 GET http://localhost:5000/js/site.min.js?v=E_42uAMVa8TinYN8jq3FpRD6ZhY1iq4oX08MOz5oby4
info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]
      Request finished in 2108.1357ms 200 text/html; charset=utf-8
info: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[6]
      The file /js/site.min.js was not modified
info: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[6]
      The file /css/site.min.css was not modified
info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]

You may get more eyes from people that have more context on ASP.NET Core on this repo: https://github.com/aspnet/home

@AbhimanyuAryan , hopefully you are getting support on the aspnet repo. I'm going to close this here.


libpanel: init at 1.0.alpha1

Description of changes

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

We have this in https://github.com/NixOS/nixpkgs/pull/182618, including devdocs.

Closing for #182618.


Added todo warn Warn new commits after reviews

It warns if there has been new commits after the last review
if so it warns and then outputs two links to one of the commit
link and other link for all the changes that the PR has.

The tests are coming soon probably early tomorrow. Just wanted to get it in so you guys could review it. Definitely request changes on naming things differently.
Sample output below.

Codecov Report

Merging #56 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #56   +/-   ##
=======================================
  Coverage   96.54%   96.54%
=======================================
  Files          13       13
  Lines         434      434
=======================================
  Hits          419      419
  Misses         15       15

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e21cf5a...12c3372. Read the comment docs.

Also can you add some tests in test/unit/pr_checker.test.js? Thanks!

I will probably close this PR and make a new one with test and all the suggestions and fixes.

I think a combination of what @tniessen proposed and something like
[WARN] Changes were pushed since the last review:
[WARN] src: improve module loader readability
[WARN] nit: fix error
[WARN] See https://github.com/nodejs/node/pull/16536/commits for more info
I'm not a big fan of adding [commit] or something similar to each line because once you're familiar with the tool it just means you have to look that far further right to start reading. But maybe others have different thoughts on that.

[WARN] There has been new commits pushed to Pull Request since the last review

See https://github.com/joyeecheung/node-core-utils/pull/56#discussion_r148713805. You don't need to use that suggestion but it should conform to English grammar.

[WARN] [New Commit] src: improve module loader readability

Why is New Commit enclosed in brackets while New commit url is not? Why is Commit capitalized in New Commit but not in New commit url?

I just tried to ninja edit but I'll post it separately, in case people miss it:

Maybe if it's just one commit, it links to it and if it's multiple it links to the aggregate commits page.

added new pr


Move LECO communication to new serializer (PyMoDAQ 5)

Follow up of #431
Also: Adapt the LECO example (maybe)

actually thinking about it, I have the feeling the serializer could be located in pymodaq_utils as we are registering Serializable objects during their declaration. We would then have:

pymodaq_utils: containing the factory , base, and implementations for base types (float, int, list, ndarray, bytes...)
pymodaq_data: register all the data types and axis
pymodaq_gui: register ParameterWithPath

Like this we could extend this serialization feature out of the data management package (pymodaq_data)

Even better 😁

ok it's all done! @BenediktBurger you can make the pymodaq pyleco modification importing the factory from pymodaq_utils.serialize.factory

I just did a PR #467 to make sure DataActuator are properly serialized but also to make sure the actual version pass its tests due to change of imports for serializer stuff. At the end of this work, we will drop the lecacy TCP/IP stuff


Simpler way to avoid the UserWarning: Converting sparse IndexedSlices

I have a rnn network structure that looks like following
    cells = rnn.MultiRNNCell(
        [self._one_rnn_cell(l + 1) for l in range(self.layers)],
        state_is_tuple=True
    ) if self.layers > 1 else self._one_rnn_cell(1)

    out, _ = tf.nn.dynamic_rnn(cells, self.inputs,
                               dtype=tf.float32, scope="DyRNN")
    out = tf.transpose(out, [1, 0, 2])
    num_time_steps = int(out.get_shape()[0])
    last_state = tf.gather(out, num_time_steps - 1, name="last_lstm_state")

Here while I am running the code, I am getting the following warning.
UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory. "Converting sparse IndexedSlices to a dense Tensor of unknown shape. "

Now I understand why this error is coming. I tried out several ideas and the most common one is the following
How to deal with UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape
But the problem is this requires too many variable like
max_length time_steps seq_length n_dim partitions
this make the code very unreadable. I wanted to know if there is a simpler way I can avoid the problem.
Also if the sequence length remain same across all the batches, can I assume max_length == time_steps == seq_length ?
Please help, documentation is very less.
Note: I added this question in stackoverflow here but haven't received any replies yet.

@MayukhSobo
Please provide details about what platform you are using (operating system, architecture). Also include your TensorFlow version.
Please provide the colab link or simple standalone code to reproduce the issue in our environment. It helps us in localizing the issue faster.Request you to fill the Github new issue template. Thanks!

@ravikyram
Please find the following information about the systems.
System information

OS Platform and Distribution: Ubuntu 18.04 LTS
Tensorflow gpu was installed using conda 4.8.2
Numpy version:     1.18.1  py37h4f9e942_0
Numpy base:         1.18.1  py37hde5b4d6_1
Python version: 3.7.5
print(tf.GIT_VERSION, tf.VERSION) gives output as v1.15.0-rc3-22-g590d6ee 1.15.0

Here is the code snippet of the model. Due to some legal obligation, I can not share the complete code with you..
def _one_rnn_cell(self, layer):
        """
        This create one RNN cell
        :param layer: Mention the layer number
        :return: LSTM cell with or without dropout
        """

        lstm_cell = rnn.LSTMCell(self.lstm_size,
                                 state_is_tuple=True,
                                 name=f'LSTMCell_layer_{layer}')
        kp = self.keep_prob
        if not self.training:
            print("Setting keep_prob = 1 for testing!!")
            kp = 1.0

        lstm_cell = rnn.DropoutWrapper(lstm_cell,
                                       output_keep_prob=kp)
        return lstm_cell
    def _get_all_placeholders(self):
        # self.learning_rate = tf.placeholder(tf.float32, None,
        #                                     name="learning_rate")
        self.inputs = tf.placeholder(tf.float32,
                                     [None, self.time_steps, self.input_size],
                                     name="inputs")
        self.targets = tf.placeholder(tf.float32, [None, self.input_size],
                                      name="targets")

    def _build_graph(self):
        # First get all the placeholders
        self._get_all_placeholders()

        # Second, create multiple LSTM cells for all the layers
        cells = rnn.MultiRNNCell(
            [self._one_rnn_cell(l + 1) for l in range(self.layers)],
            state_is_tuple=True
        ) if self.layers > 1 else self._one_rnn_cell(1)

        # Wrap the all the cells into an RNN
        # We shall use dynamic_rnn so that
        """
          1. All the cells for each time_steps in the graph
            is not created while creating the RNN

          2. We can pass more/less than the sequence length.
        """

        # self.inputs = tf.transpose(self.inputs, perm=[1, 0, 2])
        out, states = tf.nn.dynamic_rnn(cells, self.inputs,
                                        dtype=tf.float32, scope="dynamic_rnn",
                                        time_major=False)

        # output[:, ts-1, :]
        # It returns the output of all the time_steps. However we need the
        # output only from the last time step.

        # We could have also wriiten self.time_steps - 1
        # but that would not have been generic.

        # Actual output => (batch_size, num_steps, lstm_size)
        # This converts => (num_steps, batch_size, lstm_size)
        # JUST LIKE THAT!!

        # all_timesteps = tf.reshape(out, [-1, self.lstm_size])
        # row_inds = tf.range(0, 16) * 30 + (30 - 1)
        # partitions = tf.reduce_sum(tf.one_hot(row_inds, tf.shape(all_timesteps)[0], dtype='int32'), 0)
        # last_timesteps = tf.dynamic_partition(all_timesteps, partitions, 2)  # (batch_size, n_dim)
        # last_state = last_timesteps[1]

        out = tf.transpose(out, [1, 0, 2])
        num_time_steps = int(out.get_shape()[0])
        last_state = tf.gather(out, num_time_steps - 1, name="last_lstm_state")

        # last_state = states.h

        # last_state = tf.unstack(outputs, axis=0)
        # last_state = last_state[-1]
        # last_state = tf.transpose(last_state, perm=[0, 1, 2])
        # Now that we've got the last_state, we can add it to a linear layer for Regression.

        with tf.name_scope('Linear'):
            W = LstmRNN._variable_on_device(
                name='l_w',
                shape=(self.lstm_size, self.input_size),
                initializer=tf.truncated_normal_initializer(),
                device=self.device
            )
            bias = LstmRNN._variable_on_device(
                name='l_b',
                shape=(self.input_size,),
                initializer=tf.constant_initializer(0.1),
                device=self.device
            )
            self.pred = tf.add(tf.matmul(last_state, W, name='Wx'),
                               bias, name='output')

The problem (UserWarning) apparently is coming from the following lines
        out = tf.transpose(out, [1, 0, 2])
        num_time_steps = int(out.get_shape()[0])
        last_state = tf.gather(out, num_time_steps - 1, name="last_lstm_state")

I can use tf.dynamic_partition but that makes the code a lot more unreadable and not easy to follow and more than that I need to pass the batch_size before hand which is also a little inconvenient.
I am currently not getting any warning if I write

last_state = states.h

But I am not sure what would happen if the sequence length is not equal to max_length. Is is a good idea to use states.h all the time?

This question is better asked on StackOverflow since it is not a bug or feature request. There is also a larger community that reads questions there.
Thanks!


Use PSI for running OpenShift 3.x tests

OpenShift 3.11 tests that were earlier working on Travis need to run on PSI infrastructure. This is mainly required to test Service Catalog.
/area testing
/priority high
Work pending for Sprint 198:
The required scrips are pending review in PR https://github.com/openshift/odo/pull/4406. Once merged the corrsponding Jenkins job will be enabled to complete the setup for first run.
Acceptance criteria:

[x] Trigger Jenkins job to run tests in Minishift
[x] E2E and integration tests are executed
[ ] Tests are running correctly. This means that they are not flaking. If the test suite passes, then the repeated runs with the same code base pass as well.

If we manage to address #4242 elegantly (that is, odo code becoming more clear and maintainable w.r.t supporting multiple service backends - Svc Cat and Operator Hub), we can simply run the Service Catalog tests on minikube as proposed in #4287. All we will then have to do is bring up Svc Cat on minikube .
IMO, that's what we should target. I don't see any other reason why we would want to test things on OCP 3.x. @kadel @girishramnani @mik-dass is there anything else we test on 3.x?

$ minishift openshift component add service-catalog
$ minishift openshift component add automation-broker
$ minishift openshift component add template-service-broker

This is ALL you need to enable Service Catalog on a Minishift VM.

$ minishift openshift component add service-catalog
$ minishift openshift component add automation-broker
$ minishift openshift component add template-service-broker

This is ALL you need to enable Service Catalog on a Minishift VM.

Create image:
openstack server create --flavor m1.large --image Fedora-Cloud-Base-32 --nic net-id=provider_net_shared_3 --security-group default --security-group all-open --key-name releng-pub-key odo-fedora-minishift
ssh to the image using the releng-key:
ssh -i ~/psi/PSI/lib/common/releng-key fedora@<ip_address>
Had problems to ssh:
$ ssh -i ~/psi/PSI/lib/common/releng-key<EMAIL_ADDRESS>@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: UNPROTECTED PRIVATE KEY FILE!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0644 for<EMAIL_ADDRESS>are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.
Load key<EMAIL_ADDRESS>bad permissions
<EMAIL_ADDRESS>password:

Resolved by changing permissions to the local private key file to 600
$ ssh -i ~/psi/PSI/lib/common/releng-key<EMAIL_ADDRESS>sudo -i
dnf install grubby
grubby --update-kernel=ALL --args="systemd.unified_cgroup_hierarchy=0"
systemctl reboot
ssh -i ~/psi/PSI/lib/common/releng-key fedora@<ip_address>
sudo -id

Install docker:
Follow https://docs.docker.com/engine/install/fedora/
sudo dnf -y install dnf-plugins-core
sudo dnf config-manager \
    --add-repo \
    https://download.docker.com/linux/fedora/docker-ce.repo

sudo dnf install docker-ce docker-ce-cli containerd.io

sudo systemctl start docker

[root@odo-fedora-minishift ~]# docker version
Client: Docker Engine - Community
 Version:           20.10.2
 API version:       1.41
 Go version:        go1.13.15
 Git commit:        2291f61
 Built:             Mon Dec 28 16:18:28 2020
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.2
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       8891c58
  Built:            Mon Dec 28 16:15:41 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.3
  GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
 runc:
  Version:          1.0.0-rc92
  GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Setup Minishift by following:
https://docs.okd.io/3.11/minishift/getting-started/setting-up-virtualization-environment.html#kvm-driver-fedora
s```
udo dnf install libvirt qemu-kvm
sudo usermod -a -G libvirt $(whoami)
newgrp libvirt

Install minishift:
Download it:
`curl -Lo minishift.tgz https://github.com/minishift/minishift/releases/download/v1.34.3/minishift-1.34.3-linux-amd64.tgz`

Extract it:
`tar -xvzf minishift.tgz`

`sudo mv minishift-1.34.3-linux-amd64/minishift /usr/local/bin`

Create image:
openstack server create --flavor m1.large --image Fedora-Cloud-Base-32 --nic net-id=provider_net_shared_3 --security-group default --security-group all-open --key-name releng-pub-key odo-fedora-minishift
ssh to the image using the releng-key:
ssh -i ~/psi/PSI/lib/common/releng-key fedora@<ip_address>
Had problems to ssh:
$ ssh -i ~/psi/PSI/lib/common/releng-key<EMAIL_ADDRESS>@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: UNPROTECTED PRIVATE KEY FILE!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0644 for<EMAIL_ADDRESS>are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.
Load key<EMAIL_ADDRESS>bad permissions
<EMAIL_ADDRESS>password:

Resolved by changing permissions to the local private key file to 600
$ ssh -i ~/psi/PSI/lib/common/releng-key<EMAIL_ADDRESS>sudo -i
dnf install grubby
grubby --update-kernel=ALL --args="systemd.unified_cgroup_hierarchy=0"
systemctl reboot
ssh -i ~/psi/PSI/lib/common/releng-key fedora@<ip_address>
sudo -id

Install docker:
Follow https://docs.docker.com/engine/install/fedora/
sudo dnf -y install dnf-plugins-core
sudo dnf config-manager \
    --add-repo \
    https://download.docker.com/linux/fedora/docker-ce.repo

sudo dnf install docker-ce docker-ce-cli containerd.io

sudo systemctl start docker

[root@odo-fedora-minishift ~]# docker version
Client: Docker Engine - Community
 Version:           20.10.2
 API version:       1.41
 Go version:        go1.13.15
 Git commit:        2291f61
 Built:             Mon Dec 28 16:18:28 2020
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.2
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       8891c58
  Built:            Mon Dec 28 16:15:41 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.3
  GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
 runc:
  Version:          1.0.0-rc92
  GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Setup Minishift by following:
https://docs.okd.io/3.11/minishift/getting-started/setting-up-virtualization-environment.html#kvm-driver-fedora
s```
udo dnf install libvirt qemu-kvm
sudo usermod -a -G libvirt $(whoami)
newgrp libvirt

Install minishift:
Download it:
`curl -Lo minishift.tgz https://github.com/minishift/minishift/releases/download/v1.34.3/minishift-1.34.3-linux-amd64.tgz`

Extract it:
`tar -xvzf minishift.tgz`

`sudo mv minishift-1.34.3-linux-amd64/minishift /usr/local/bin`

Error starting minishift:
$ minishift start
-- Starting profile 'minishift'
-- Check if deprecated options are used ... OK
-- Checking if https://github.com is reachable ... OK
-- Checking if requested OpenShift version 'v3.11.0' is valid ...
   Hit github rate limit: GET https://api.github.com/repos/openshift/origin/releases/tags/v3.11.0: 403 API rate limit exceeded for <IP_ADDRESS>. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.) [rate reset in 58m18s]
FAIL

Error starting minishift:
$ minishift start
-- Starting profile 'minishift'
-- Check if deprecated options are used ... OK
-- Checking if https://github.com is reachable ... OK
-- Checking if requested OpenShift version 'v3.11.0' is valid ...
   Hit github rate limit: GET https://api.github.com/repos/openshift/origin/releases/tags/v3.11.0: 403 API rate limit exceeded for <IP_ADDRESS>. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.) [rate reset in 58m18s]
FAIL

minishift start
-- Starting profile 'minishift'
-- Check if deprecated options are used ... OK
-- Checking if https://github.com is reachable ... OK
-- Checking if requested OpenShift version 'v3.11.0' is valid ... OK
-- Checking if requested OpenShift version 'v3.11.0' is supported ... OK
-- Checking if requested hypervisor 'kvm' is supported on this platform ... OK
-- Checking if KVM driver is installed ...
   Driver is available at /usr/local/bin/docker-machine-driver-kvm ...
   Checking driver binary is executable ... OK
-- Checking if Libvirt is installed ... OK
-- Checking if Libvirt default network is present ... FAIL
   See the 'Setting Up the Virtualization Environment' topic (https://docs.okd.io/latest/minishift/getting-started/setting-up-virtualization-environment.html) for more information

Checking on it

minishift start
-- Starting profile 'minishift'
-- Check if deprecated options are used ... OK
-- Checking if https://github.com is reachable ... OK
-- Checking if requested OpenShift version 'v3.11.0' is valid ... OK
-- Checking if requested OpenShift version 'v3.11.0' is supported ... OK
-- Checking if requested hypervisor 'kvm' is supported on this platform ... OK
-- Checking if KVM driver is installed ...
   Driver is available at /usr/local/bin/docker-machine-driver-kvm ...
   Checking driver binary is executable ... OK
-- Checking if Libvirt is installed ... OK
-- Checking if Libvirt default network is present ... FAIL
   See the 'Setting Up the Virtualization Environment' topic (https://docs.okd.io/latest/minishift/getting-started/setting-up-virtualization-environment.html) for more information

Checking on it

To setup virtualization env:
sudo dnf install @virtualization
sudo dnf group install --with-optional virtualization
sudo systemctl start libvirtd
sudo systemctl enable libvirtd      # To start the service on boot

To setup virtualization env:
sudo dnf install @virtualization
sudo dnf group install --with-optional virtualization
sudo systemctl start libvirtd
sudo systemctl enable libvirtd      # To start the service on boot

minishift start
-- Starting profile 'minishift'
-- Check if deprecated options are used ... OK
-- Checking if https://github.com is reachable ... OK
-- Checking if requested OpenShift version 'v3.11.0' is valid ... OK
-- Checking if requested OpenShift version 'v3.11.0' is supported ... OK
-- Checking if requested hypervisor 'kvm' is supported on this platform ... OK
-- Checking if KVM driver is installed ...
   Driver is available at /usr/local/bin/docker-machine-driver-kvm ...
   Checking driver binary is executable ... OK
-- Checking if Libvirt is installed ... OK
-- Checking if Libvirt default network is present ... OK
-- Checking if Libvirt default network is active ... OK
-- Checking the ISO URL ... OK
-- Downloading OpenShift binary 'oc' version 'v3.11.0'
 53.89 MiB / 53.89 MiB [================================================================================================================================] 100.00% 0s-- Downloading OpenShift v3.11.0 checksums ... OK
-- Checking if provided oc flags are supported ... OK
-- Starting the OpenShift cluster using 'kvm' hypervisor ...
-- Minishift VM will be configured with ...
   Memory:    4 GB
   vCPUs :    2
   Disk size: 20 GB

   Downloading ISO 'https://github.com/minishift/minishift-centos-iso/releases/download/v1.17.0/minishift-centos7.iso'
 375.00 MiB / 375.00 MiB [==============================================================================================================================] 100.00% 0s
-- Starting Minishift VM ................................................................. FAIL E0126 14:31:30.949960   18432 start.go:499] Error starting the VM: Error creating the VM. Error creating machine: Error detecting OS: Too many retries waiting for SSH to be available.  Last error: Maximum number of retries (60) exceeded. Retrying.
Error starting the VM: Error creating the VM. Error creating machine: Error detecting OS: Too many retries waiting for SSH to be available.  Last error: Maximum number of retries (60) exceeded

minishift start
-- Starting profile 'minishift'
-- Check if deprecated options are used ... OK
-- Checking if https://github.com is reachable ... OK
-- Checking if requested OpenShift version 'v3.11.0' is valid ... OK
-- Checking if requested OpenShift version 'v3.11.0' is supported ... OK
-- Checking if requested hypervisor 'kvm' is supported on this platform ... OK
-- Checking if KVM driver is installed ...
   Driver is available at /usr/local/bin/docker-machine-driver-kvm ...
   Checking driver binary is executable ... OK
-- Checking if Libvirt is installed ... OK
-- Checking if Libvirt default network is present ... OK
-- Checking if Libvirt default network is active ... OK
-- Checking the ISO URL ... OK
-- Downloading OpenShift binary 'oc' version 'v3.11.0'
 53.89 MiB / 53.89 MiB [================================================================================================================================] 100.00% 0s-- Downloading OpenShift v3.11.0 checksums ... OK
-- Checking if provided oc flags are supported ... OK
-- Starting the OpenShift cluster using 'kvm' hypervisor ...
-- Minishift VM will be configured with ...
   Memory:    4 GB
   vCPUs :    2
   Disk size: 20 GB

   Downloading ISO 'https://github.com/minishift/minishift-centos-iso/releases/download/v1.17.0/minishift-centos7.iso'
 375.00 MiB / 375.00 MiB [==============================================================================================================================] 100.00% 0s
-- Starting Minishift VM ................................................................. FAIL E0126 14:31:30.949960   18432 start.go:499] Error starting the VM: Error creating the VM. Error creating machine: Error detecting OS: Too many retries waiting for SSH to be available.  Last error: Maximum number of retries (60) exceeded. Retrying.
Error starting the VM: Error creating the VM. Error creating machine: Error detecting OS: Too many retries waiting for SSH to be available.  Last error: Maximum number of retries (60) exceeded

sudo virsh net-start default
sudo virsh net-autostart default
minishift delete
rm -RF .minishift
minishift start
-- Starting profile 'minishift'
-- Check if deprecated options are used ... OK
-- Checking if https://github.com is reachable ... OK
-- Checking if requested OpenShift version 'v3.11.0' is valid ... OK
-- Checking if requested OpenShift version 'v3.11.0' is supported ... OK
-- Checking if requested hypervisor 'kvm' is supported on this platform ... OK
-- Checking if KVM driver is installed ...
Driver is available at /usr/local/bin/docker-machine-driver-kvm ...
Checking driver binary is executable ... OK
-- Checking if Libvirt is installed ... OK
-- Checking if Libvirt default network is present ... OK
-- Checking if Libvirt default network is active ... OK
-- Checking the ISO URL ... OK
-- Downloading OpenShift binary 'oc' version 'v3.11.0'
53.89 MiB / 53.89 MiB [================================================================================================================================] 100.00% 0s-- Downloading OpenShift v3.11.0 checksums ... OK
-- Checking if provided oc flags are supported ... OK
-- Starting the OpenShift cluster using 'kvm' hypervisor ...
-- Minishift VM will be configured with ...
Memory:    4 GB
vCPUs :    2
Disk size: 20 GB
Downloading ISO 'https://github.com/minishift/minishift-centos-iso/releases/download/v1.17.0/minishift-centos7.iso'
375.00 MiB / 375.00 MiB [==============================================================================================================================] 100.00% 0s
-- Starting Minishift VM ..................... OK
-- Checking for IP address ... OK
-- Checking for nameservers ... OK
-- Checking if external host is reachable from the Minishift VM ...
Pinging <IP_ADDRESS> ... OK
-- Checking HTTP connectivity from the VM ...
Retrieving http://minishift.io/index.html ... OK
-- Checking if persistent storage volume is mounted ... OK
-- Checking available disk space ... 1% used OK
-- Writing current configuration for static assignment of IP address ... WARN
Importing 'openshift/origin-control-plane:v3.11.0'  CACHE MISS
Importing 'openshift/origin-docker-registry:v3.11.0'  CACHE MISS
Importing 'openshift/origin-haproxy-router:v3.11.0'  CACHE MISS
-- OpenShift cluster will be configured with ...
Version: v3.11.0
-- Pulling the OpenShift Container Image .Error pulling the OpenShift container image: ssh command error:
command : docker pull openshift/origin-control-plane:v3.11.0
err     : exit status 1
output  : Trying to pull repository docker.io/openshift/origin-control-plane ...
toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit

sudo virsh net-start default
sudo virsh net-autostart default
minishift delete
rm -RF .minishift
minishift start
-- Starting profile 'minishift'
-- Check if deprecated options are used ... OK
-- Checking if https://github.com is reachable ... OK
-- Checking if requested OpenShift version 'v3.11.0' is valid ... OK
-- Checking if requested OpenShift version 'v3.11.0' is supported ... OK
-- Checking if requested hypervisor 'kvm' is supported on this platform ... OK
-- Checking if KVM driver is installed ...
Driver is available at /usr/local/bin/docker-machine-driver-kvm ...
Checking driver binary is executable ... OK
-- Checking if Libvirt is installed ... OK
-- Checking if Libvirt default network is present ... OK
-- Checking if Libvirt default network is active ... OK
-- Checking the ISO URL ... OK
-- Downloading OpenShift binary 'oc' version 'v3.11.0'
53.89 MiB / 53.89 MiB [================================================================================================================================] 100.00% 0s-- Downloading OpenShift v3.11.0 checksums ... OK
-- Checking if provided oc flags are supported ... OK
-- Starting the OpenShift cluster using 'kvm' hypervisor ...
-- Minishift VM will be configured with ...
Memory:    4 GB
vCPUs :    2
Disk size: 20 GB
Downloading ISO 'https://github.com/minishift/minishift-centos-iso/releases/download/v1.17.0/minishift-centos7.iso'
375.00 MiB / 375.00 MiB [==============================================================================================================================] 100.00% 0s
-- Starting Minishift VM ..................... OK
-- Checking for IP address ... OK
-- Checking for nameservers ... OK
-- Checking if external host is reachable from the Minishift VM ...
Pinging <IP_ADDRESS> ... OK
-- Checking HTTP connectivity from the VM ...
Retrieving http://minishift.io/index.html ... OK
-- Checking if persistent storage volume is mounted ... OK
-- Checking available disk space ... 1% used OK
-- Writing current configuration for static assignment of IP address ... WARN
Importing 'openshift/origin-control-plane:v3.11.0'  CACHE MISS
Importing 'openshift/origin-docker-registry:v3.11.0'  CACHE MISS
Importing 'openshift/origin-haproxy-router:v3.11.0'  CACHE MISS
-- OpenShift cluster will be configured with ...
Version: v3.11.0
-- Pulling the OpenShift Container Image .Error pulling the OpenShift container image: ssh command error:
command : docker pull openshift/origin-control-plane:v3.11.0
err     : exit status 1
output  : Trying to pull repository docker.io/openshift/origin-control-plane ...
toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit

As per https://www.docker.com/increase-rate-limits:
_On November 20, 2020, rate limits anonymous and free authenticated use of Docker Hub went into effect. Anonymous and Free Docker Hub users are limited to 100 and 200 container image pull requests per six hours.
If you are affected by these changes you will receive this error message:
ERROR: toomanyrequests: Too Many Requests.
OR
You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limits.
To increase your pull rate limits you can upgrade your account to a Docker Pro or Team subscription.
The rate limits of 100 container image requests per six hours for anonymous usage, and 200 container image requests per six hours for free Docker accounts are now in effect. Image requests exceeding these limits will be denied until the six hour window elapses._

As per https://www.docker.com/increase-rate-limits:
_On November 20, 2020, rate limits anonymous and free authenticated use of Docker Hub went into effect. Anonymous and Free Docker Hub users are limited to 100 and 200 container image pull requests per six hours.
If you are affected by these changes you will receive this error message:
ERROR: toomanyrequests: Too Many Requests.
OR
You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limits.
To increase your pull rate limits you can upgrade your account to a Docker Pro or Team subscription.
The rate limits of 100 container image requests per six hours for anonymous usage, and 200 container image requests per six hours for free Docker accounts are now in effect. Image requests exceeding these limits will be denied until the six hour window elapses._

Trying a bit later:
minishift start
-- Starting profile 'minishift'
-- Check if deprecated options are used ... OK
-- Checking if https://github.com is reachable ... OK
-- Checking if requested OpenShift version 'v3.11.0' is valid ... OK
-- Checking if requested OpenShift version 'v3.11.0' is supported ... OK
-- Checking if requested hypervisor 'kvm' is supported on this platform ... OK
-- Checking if KVM driver is installed ...
   Driver is available at /usr/local/bin/docker-machine-driver-kvm ...
   Checking driver binary is executable ... OK
-- Checking if Libvirt is installed ... OK
-- Checking if Libvirt default network is present ... OK
-- Checking if Libvirt default network is active ... OK
-- Checking the ISO URL ... OK
-- Downloading OpenShift binary 'oc' version 'v3.11.0'
 53.89 MiB / 53.89 MiB [================================================================================================================================] 100.00% 0s-- Downloading OpenShift v3.11.0 checksums ... OK
-- Checking if provided oc flags are supported ... OK
-- Starting the OpenShift cluster using 'kvm' hypervisor ...
-- Minishift VM will be configured with ...
   Memory:    4 GB
   vCPUs :    2
   Disk size: 20 GB

   Downloading ISO 'https://github.com/minishift/minishift-centos-iso/releases/download/v1.17.0/minishift-centos7.iso'
 375.00 MiB / 375.00 MiB [==============================================================================================================================] 100.00% 0s
-- Starting Minishift VM ..................... OK
-- Checking for IP address ... OK
-- Checking for nameservers ... OK
-- Checking if external host is reachable from the Minishift VM ...
   Pinging <IP_ADDRESS> ... OK
-- Checking HTTP connectivity from the VM ...
   Retrieving http://minishift.io/index.html ... OK
-- Checking if persistent storage volume is mounted ... OK
-- Checking available disk space ... 1% used OK
-- Writing current configuration for static assignment of IP address ... WARN
   Importing 'openshift/origin-control-plane:v3.11.0' . CACHE MISS
   Importing 'openshift/origin-docker-registry:v3.11.0'  CACHE MISS
   Importing 'openshift/origin-haproxy-router:v3.11.0'  CACHE MISS
-- OpenShift cluster will be configured with ...
   Version: v3.11.0
-- Pulling the OpenShift Container Image ............ OK
-- Copying oc binary from the OpenShift container image to VM ... OK
-- Starting OpenShift cluster .......................................................................................Error during 'cluster up' execution: Error starting the cluster. ssh command error:
command : /var/lib/minishift/bin/oc cluster up --routing-suffix <IP_ADDRESS>.nip.io --base-dir /var/lib/minishift/base --image 'openshift/origin-${component}:v3.11.0' --public-hostname <IP_ADDRESS>
err     : exit status 1
output  : Getting a Docker client ...
Checking if image openshift/origin-control-plane:v3.11.0 is available ...
Pulling image openshift/origin-cli:v3.11.0
E0126 17:21:52.567614    2177 helper.go:173] Reading docker config from /home/docker/.docker/config.json failed: open /home/docker/.docker/config.json: no such file or directory, will attempt to pull image docker.io/openshift/origin-cli:v3.11.0 anonymously
Image pull complete
Pulling image openshift/origin-node:v3.11.0
E0126 17:21:53.027758    2177 helper.go:173] Reading docker config from /home/docker/.docker/config.json failed: open /home/docker/.docker/config.json: no such file or directory, will attempt to pull image docker.io/openshift/origin-node:v3.11.0 anonymously
Pulled 5/6 layers, 93% complete
Pulled 6/6 layers, 100% complete
Extracting
Image pull complete
Checking type of volume mount ...
Determining server IP ...
Using public hostname IP <IP_ADDRESS> as the host IP
Checking if OpenShift is already running ...
Checking for supported Docker version (=>1.22) ...
Checking if insecured registry is configured properly in Docker ...
Checking if required ports are available ...
Checking if OpenShift client is configured properly ...
Checking if image openshift/origin-control-plane:v3.11.0 is available ...
Starting OpenShift using openshift/origin-control-plane:v3.11.0 ...
I0126 17:22:04.912775    2177 config.go:40] Running "create-master-config"
I0126 17:22:07.981100    2177 config.go:46] Running "create-node-config"
I0126 17:22:08.970032    2177 flags.go:30] Running "create-kubelet-flags"
I0126 17:22:09.631298    2177 run_kubelet.go:49] Running "start-kubelet"
I0126 17:22:09.964747    2177 run_self_hosted.go:181] Waiting for the kube-apiserver to be ready ...
I0126 17:22:39.993724    2177 interface.go:26] Installing "kube-proxy" ...
I0126 17:22:39.993830    2177 interface.go:26] Installing "kube-dns" ...
I0126 17:22:39.993848    2177 interface.go:26] Installing "openshift-service-cert-signer-operator" ...
I0126 17:22:39.993858    2177 interface.go:26] Installing "openshift-apiserver" ...
I0126 17:22:39.993940    2177 apply_template.go:81] Installing "openshift-apiserver"
I0126 17:22:39.997096    2177 apply_template.go:81] Installing "kube-dns"
I0126 17:22:39.997104    2177 apply_template.go:81] Installing "kube-proxy"
I0126 17:22:39.997673    2177 apply_template.go:81] Installing "openshift-service-cert-signer-operator"
I0126 17:22:46.318392    2177 interface.go:41] Finished installing "kube-proxy" "kube-dns" "openshift-service-cert-signer-operator" "openshift-apiserver"
I0126 17:24:51.362191    2177 run_self_hosted.go:242] openshift-apiserver available
I0126 17:24:51.363028    2177 interface.go:26] Installing "openshift-controller-manager" ...
I0126 17:24:51.363101    2177 apply_template.go:81] Installing "openshift-controller-manager"
I0126 17:24:56.663651    2177 interface.go:41] Finished installing "openshift-controller-manager"
Adding default OAuthClient redirect URIs ...
Adding web-console ...
Adding registry ...
Adding router ...
Adding persistent-volumes ...
Adding centos-imagestreams ...
Adding sample-templates ...
I0126 17:24:56.695192    2177 interface.go:26] Installing "openshift-web-console-operator" ...
I0126 17:24:56.695214    2177 interface.go:26] Installing "openshift-image-registry" ...
I0126 17:24:56.695224    2177 interface.go:26] Installing "openshift-router" ...
I0126 17:24:56.695231    2177 interface.go:26] Installing "persistent-volumes" ...
I0126 17:24:56.695240    2177 interface.go:26] Installing "centos-imagestreams" ...
I0126 17:24:56.695249    2177 interface.go:26] Installing "sample-templates" ...
I0126 17:24:56.695361    2177 interface.go:26] Installing "sample-templates/rails quickstart" ...
I0126 17:24:56.695370    2177 interface.go:26] Installing "sample-templates/mariadb" ...
I0126 17:24:56.695377    2177 interface.go:26] Installing "sample-templates/django quickstart" ...
I0126 17:24:56.695384    2177 interface.go:26] Installing "sample-templates/postgresql" ...
I0126 17:24:56.695391    2177 interface.go:26] Installing "sample-templates/cakephp quickstart" ...
I0126 17:24:56.695398    2177 interface.go:26] Installing "sample-templates/dancer quickstart" ...
I0126 17:24:56.695404    2177 interface.go:26] Installing "sample-templates/nodejs quickstart" ...
I0126 17:24:56.695413    2177 interface.go:26] Installing "sample-templates/jenkins pipeline ephemeral" ...
I0126 17:24:56.695419    2177 interface.go:26] Installing "sample-templates/sample pipeline" ...
I0126 17:24:56.695427    2177 interface.go:26] Installing "sample-templates/mongodb" ...
I0126 17:24:56.695433    2177 interface.go:26] Installing "sample-templates/mysql" ...
I0126 17:24:56.695483    2177 apply_list.go:67] Installing "sample-templates/mysql"
I0126 17:24:56.695945    2177 apply_template.go:81] Installing "openshift-web-console-operator"
I0126 17:24:56.697488    2177 apply_list.go:67] Installing "centos-imagestreams"
I0126 17:24:56.697707    2177 apply_list.go:67] Installing "sample-templates/rails quickstart"
I0126 17:24:56.697895    2177 apply_list.go:67] Installing "sample-templates/mariadb"
I0126 17:24:56.697999    2177 apply_list.go:67] Installing "sample-templates/django quickstart"
I0126 17:24:56.698095    2177 apply_list.go:67] Installing "sample-templates/postgresql"
I0126 17:24:56.698280    2177 apply_list.go:67] Installing "sample-templates/cakephp quickstart"
I0126 17:24:56.698382    2177 apply_list.go:67] Installing "sample-templates/dancer quickstart"
I0126 17:24:56.698477    2177 apply_list.go:67] Installing "sample-templates/nodejs quickstart"
I0126 17:24:56.698642    2177 apply_list.go:67] Installing "sample-templates/jenkins pipeline ephemeral"
I0126 17:24:56.698773    2177 apply_list.go:67] Installing "sample-templates/sample pipeline"
I0126 17:24:56.698881    2177 apply_list.go:67] Installing "sample-templates/mongodb"
I0126 17:25:20.261735    2177 interface.go:41] Finished installing "sample-templates/rails quickstart" "sample-templates/mariadb" "sample-templates/django quickstart" "sample-templates/postgresql" "sample-templates/cakephp quickstart" "sample-templates/dancer quickstart" "sample-templates/nodejs quickstart" "sample-templates/jenkins pipeline ephemeral" "sample-templates/sample pipeline" "sample-templates/mongodb" "sample-templates/mysql"
E0126 17:30:20.431540    2177 interface.go:34] Failed to install "openshift-web-console-operator": timed out waiting for the condition
I0126 17:30:20.432707    2177 interface.go:41] Finished installing "openshift-web-console-operator" "openshift-image-registry" "openshift-router" "persistent-volumes" "centos-imagestreams" "sample-templates"
Error: timed out waiting for the condition

[fedora@odo-fedora-minishift ~]$ minishift status
Minishift:  Running
Profile:    minishift
OpenShift:  Running (openshift v3.11.0+1cd89d4-542)
DiskUsage:  13% of 19G (Mounted On: /mnt/sda1)
CacheUsage: 513.6 MB (used by oc binary, ISO or cached images)

Trying a bit later:
minishift start
-- Starting profile 'minishift'
-- Check if deprecated options are used ... OK
-- Checking if https://github.com is reachable ... OK
-- Checking if requested OpenShift version 'v3.11.0' is valid ... OK
-- Checking if requested OpenShift version 'v3.11.0' is supported ... OK
-- Checking if requested hypervisor 'kvm' is supported on this platform ... OK
-- Checking if KVM driver is installed ...
   Driver is available at /usr/local/bin/docker-machine-driver-kvm ...
   Checking driver binary is executable ... OK
-- Checking if Libvirt is installed ... OK
-- Checking if Libvirt default network is present ... OK
-- Checking if Libvirt default network is active ... OK
-- Checking the ISO URL ... OK
-- Downloading OpenShift binary 'oc' version 'v3.11.0'
 53.89 MiB / 53.89 MiB [================================================================================================================================] 100.00% 0s-- Downloading OpenShift v3.11.0 checksums ... OK
-- Checking if provided oc flags are supported ... OK
-- Starting the OpenShift cluster using 'kvm' hypervisor ...
-- Minishift VM will be configured with ...
   Memory:    4 GB
   vCPUs :    2
   Disk size: 20 GB

   Downloading ISO 'https://github.com/minishift/minishift-centos-iso/releases/download/v1.17.0/minishift-centos7.iso'
 375.00 MiB / 375.00 MiB [==============================================================================================================================] 100.00% 0s
-- Starting Minishift VM ..................... OK
-- Checking for IP address ... OK
-- Checking for nameservers ... OK
-- Checking if external host is reachable from the Minishift VM ...
   Pinging <IP_ADDRESS> ... OK
-- Checking HTTP connectivity from the VM ...
   Retrieving http://minishift.io/index.html ... OK
-- Checking if persistent storage volume is mounted ... OK
-- Checking available disk space ... 1% used OK
-- Writing current configuration for static assignment of IP address ... WARN
   Importing 'openshift/origin-control-plane:v3.11.0' . CACHE MISS
   Importing 'openshift/origin-docker-registry:v3.11.0'  CACHE MISS
   Importing 'openshift/origin-haproxy-router:v3.11.0'  CACHE MISS
-- OpenShift cluster will be configured with ...
   Version: v3.11.0
-- Pulling the OpenShift Container Image ............ OK
-- Copying oc binary from the OpenShift container image to VM ... OK
-- Starting OpenShift cluster .......................................................................................Error during 'cluster up' execution: Error starting the cluster. ssh command error:
command : /var/lib/minishift/bin/oc cluster up --routing-suffix <IP_ADDRESS>.nip.io --base-dir /var/lib/minishift/base --image 'openshift/origin-${component}:v3.11.0' --public-hostname <IP_ADDRESS>
err     : exit status 1
output  : Getting a Docker client ...
Checking if image openshift/origin-control-plane:v3.11.0 is available ...
Pulling image openshift/origin-cli:v3.11.0
E0126 17:21:52.567614    2177 helper.go:173] Reading docker config from /home/docker/.docker/config.json failed: open /home/docker/.docker/config.json: no such file or directory, will attempt to pull image docker.io/openshift/origin-cli:v3.11.0 anonymously
Image pull complete
Pulling image openshift/origin-node:v3.11.0
E0126 17:21:53.027758    2177 helper.go:173] Reading docker config from /home/docker/.docker/config.json failed: open /home/docker/.docker/config.json: no such file or directory, will attempt to pull image docker.io/openshift/origin-node:v3.11.0 anonymously
Pulled 5/6 layers, 93% complete
Pulled 6/6 layers, 100% complete
Extracting
Image pull complete
Checking type of volume mount ...
Determining server IP ...
Using public hostname IP <IP_ADDRESS> as the host IP
Checking if OpenShift is already running ...
Checking for supported Docker version (=>1.22) ...
Checking if insecured registry is configured properly in Docker ...
Checking if required ports are available ...
Checking if OpenShift client is configured properly ...
Checking if image openshift/origin-control-plane:v3.11.0 is available ...
Starting OpenShift using openshift/origin-control-plane:v3.11.0 ...
I0126 17:22:04.912775    2177 config.go:40] Running "create-master-config"
I0126 17:22:07.981100    2177 config.go:46] Running "create-node-config"
I0126 17:22:08.970032    2177 flags.go:30] Running "create-kubelet-flags"
I0126 17:22:09.631298    2177 run_kubelet.go:49] Running "start-kubelet"
I0126 17:22:09.964747    2177 run_self_hosted.go:181] Waiting for the kube-apiserver to be ready ...
I0126 17:22:39.993724    2177 interface.go:26] Installing "kube-proxy" ...
I0126 17:22:39.993830    2177 interface.go:26] Installing "kube-dns" ...
I0126 17:22:39.993848    2177 interface.go:26] Installing "openshift-service-cert-signer-operator" ...
I0126 17:22:39.993858    2177 interface.go:26] Installing "openshift-apiserver" ...
I0126 17:22:39.993940    2177 apply_template.go:81] Installing "openshift-apiserver"
I0126 17:22:39.997096    2177 apply_template.go:81] Installing "kube-dns"
I0126 17:22:39.997104    2177 apply_template.go:81] Installing "kube-proxy"
I0126 17:22:39.997673    2177 apply_template.go:81] Installing "openshift-service-cert-signer-operator"
I0126 17:22:46.318392    2177 interface.go:41] Finished installing "kube-proxy" "kube-dns" "openshift-service-cert-signer-operator" "openshift-apiserver"
I0126 17:24:51.362191    2177 run_self_hosted.go:242] openshift-apiserver available
I0126 17:24:51.363028    2177 interface.go:26] Installing "openshift-controller-manager" ...
I0126 17:24:51.363101    2177 apply_template.go:81] Installing "openshift-controller-manager"
I0126 17:24:56.663651    2177 interface.go:41] Finished installing "openshift-controller-manager"
Adding default OAuthClient redirect URIs ...
Adding web-console ...
Adding registry ...
Adding router ...
Adding persistent-volumes ...
Adding centos-imagestreams ...
Adding sample-templates ...
I0126 17:24:56.695192    2177 interface.go:26] Installing "openshift-web-console-operator" ...
I0126 17:24:56.695214    2177 interface.go:26] Installing "openshift-image-registry" ...
I0126 17:24:56.695224    2177 interface.go:26] Installing "openshift-router" ...
I0126 17:24:56.695231    2177 interface.go:26] Installing "persistent-volumes" ...
I0126 17:24:56.695240    2177 interface.go:26] Installing "centos-imagestreams" ...
I0126 17:24:56.695249    2177 interface.go:26] Installing "sample-templates" ...
I0126 17:24:56.695361    2177 interface.go:26] Installing "sample-templates/rails quickstart" ...
I0126 17:24:56.695370    2177 interface.go:26] Installing "sample-templates/mariadb" ...
I0126 17:24:56.695377    2177 interface.go:26] Installing "sample-templates/django quickstart" ...
I0126 17:24:56.695384    2177 interface.go:26] Installing "sample-templates/postgresql" ...
I0126 17:24:56.695391    2177 interface.go:26] Installing "sample-templates/cakephp quickstart" ...
I0126 17:24:56.695398    2177 interface.go:26] Installing "sample-templates/dancer quickstart" ...
I0126 17:24:56.695404    2177 interface.go:26] Installing "sample-templates/nodejs quickstart" ...
I0126 17:24:56.695413    2177 interface.go:26] Installing "sample-templates/jenkins pipeline ephemeral" ...
I0126 17:24:56.695419    2177 interface.go:26] Installing "sample-templates/sample pipeline" ...
I0126 17:24:56.695427    2177 interface.go:26] Installing "sample-templates/mongodb" ...
I0126 17:24:56.695433    2177 interface.go:26] Installing "sample-templates/mysql" ...
I0126 17:24:56.695483    2177 apply_list.go:67] Installing "sample-templates/mysql"
I0126 17:24:56.695945    2177 apply_template.go:81] Installing "openshift-web-console-operator"
I0126 17:24:56.697488    2177 apply_list.go:67] Installing "centos-imagestreams"
I0126 17:24:56.697707    2177 apply_list.go:67] Installing "sample-templates/rails quickstart"
I0126 17:24:56.697895    2177 apply_list.go:67] Installing "sample-templates/mariadb"
I0126 17:24:56.697999    2177 apply_list.go:67] Installing "sample-templates/django quickstart"
I0126 17:24:56.698095    2177 apply_list.go:67] Installing "sample-templates/postgresql"
I0126 17:24:56.698280    2177 apply_list.go:67] Installing "sample-templates/cakephp quickstart"
I0126 17:24:56.698382    2177 apply_list.go:67] Installing "sample-templates/dancer quickstart"
I0126 17:24:56.698477    2177 apply_list.go:67] Installing "sample-templates/nodejs quickstart"
I0126 17:24:56.698642    2177 apply_list.go:67] Installing "sample-templates/jenkins pipeline ephemeral"
I0126 17:24:56.698773    2177 apply_list.go:67] Installing "sample-templates/sample pipeline"
I0126 17:24:56.698881    2177 apply_list.go:67] Installing "sample-templates/mongodb"
I0126 17:25:20.261735    2177 interface.go:41] Finished installing "sample-templates/rails quickstart" "sample-templates/mariadb" "sample-templates/django quickstart" "sample-templates/postgresql" "sample-templates/cakephp quickstart" "sample-templates/dancer quickstart" "sample-templates/nodejs quickstart" "sample-templates/jenkins pipeline ephemeral" "sample-templates/sample pipeline" "sample-templates/mongodb" "sample-templates/mysql"
E0126 17:30:20.431540    2177 interface.go:34] Failed to install "openshift-web-console-operator": timed out waiting for the condition
I0126 17:30:20.432707    2177 interface.go:41] Finished installing "openshift-web-console-operator" "openshift-image-registry" "openshift-router" "persistent-volumes" "centos-imagestreams" "sample-templates"
Error: timed out waiting for the condition

[fedora@odo-fedora-minishift ~]$ minishift status
Minishift:  Running
Profile:    minishift
OpenShift:  Running (openshift v3.11.0+1cd89d4-542)
DiskUsage:  13% of 19G (Mounted On: /mnt/sda1)
CacheUsage: 513.6 MB (used by oc binary, ISO or cached images)

after a while it gets operational
minishift openshift component add service-catalog
Adding service-catalog ...
I0126 20:55:21.667316   29370 interface.go:26] Installing "openshift-service-catalog" ...
I0126 20:55:22.498959   29370 apply_template.go:81] Installing "service-catalog"
I0126 20:55:50.611690   29370 interface.go:41] Finished installing "openshift-service-catalog"

minishift openshift component add automation-service-broker
Adding automation-service-broker ...
I0126 20:57:55.948860   31149 interface.go:26] Installing "automation-service-broker" ...
I0126 20:57:55.990482   31149 apply_template.go:81] Installing "automation-service-broker"
I0126 20:59:06.991486   31149 interface.go:41] Finished installing "automation-service-broker"

minishift openshift component add template-service-broker
Adding template-service-broker ...
I0126 20:59:16.848960     466 interface.go:26] Installing "openshift-template-service-broker" ...
I0126 20:59:16.851596     466 apply_template.go:81] Installing "template-service-broker-apiserver"
I0126 20:59:48.550132     466 apply_template.go:81] Installing "tsb-registration"
I0126 20:59:52.352624     466 interface.go:41] Finished installing "openshift-template-service-broker"

after a while it gets operational
minishift openshift component add service-catalog
Adding service-catalog ...
I0126 20:55:21.667316   29370 interface.go:26] Installing "openshift-service-catalog" ...
I0126 20:55:22.498959   29370 apply_template.go:81] Installing "service-catalog"
I0126 20:55:50.611690   29370 interface.go:41] Finished installing "openshift-service-catalog"

minishift openshift component add automation-service-broker
Adding automation-service-broker ...
I0126 20:57:55.948860   31149 interface.go:26] Installing "automation-service-broker" ...
I0126 20:57:55.990482   31149 apply_template.go:81] Installing "automation-service-broker"
I0126 20:59:06.991486   31149 interface.go:41] Finished installing "automation-service-broker"

minishift openshift component add template-service-broker
Adding template-service-broker ...
I0126 20:59:16.848960     466 interface.go:26] Installing "openshift-template-service-broker" ...
I0126 20:59:16.851596     466 apply_template.go:81] Installing "template-service-broker-apiserver"
I0126 20:59:48.550132     466 apply_template.go:81] Installing "tsb-registration"
I0126 20:59:52.352624     466 interface.go:41] Finished installing "openshift-template-service-broker"

oc login -u developer -p developer --insecure-skip-tls-verify $(minishift ip):8443
Login successful.

You don't have any projects. You can try to create a new project, by running

    oc new-project <projectname>

Welcome! See 'oc help' to get started.

oc login -u developer -p developer --insecure-skip-tls-verify $(minishift ip):8443
Login successful.

You don't have any projects. You can try to create a new project, by running

    oc new-project <projectname>

Welcome! See 'oc help' to get started.

All tests for test-cmd-service. Analyzing to resolve failures
Summarizing 12 Failures:

[Fail] odo service command tests [BeforeEach] When working from outside a component dir should be able to list services, as well as json list in a given app and project combination
/root/openshift/odo/tests/helper/helper_run.go:34

[Fail] odo service command tests [BeforeEach] When describing services should succeed when we're describing service that could have integer value for default field
/root/openshift/odo/tests/helper/helper_run.go:34

[Fail] odo service command tests [BeforeEach] check catalog service search functionality check that a service does not exist
/root/openshift/odo/tests/helper/helper_run.go:34

[Fail] odo service command tests [BeforeEach] create service with Env non-interactively should be able to create postgresql with env multiple times
/root/openshift/odo/tests/helper/helper_run.go:34

[Fail] odo service command tests [BeforeEach] When working from outside a component dir should be able to create, list and delete a service using a given value for --context
/root/openshift/odo/tests/helper/helper_run.go:34

[Fail] odo service command tests [BeforeEach] When the application is deleted should delete the service(s) in the application as well
/root/openshift/odo/tests/helper/helper_run.go:34

[Fail] odo service command tests [BeforeEach] check search functionality should pass with searching for part of a service name
/root/openshift/odo/tests/helper/helper_run.go:34

[Fail] odo service command tests [BeforeEach] When working from outside a component dir should be able to create, list and delete services without a context and using --app and --project flags instaed
/root/openshift/odo/tests/helper/helper_run.go:34

[Fail] odo service command tests [BeforeEach] when running help for service command should display the help
/root/openshift/odo/tests/helper/helper_run.go:34

[Fail] odo service command tests [BeforeEach] checking machine readable output for service catalog should succeed listing catalog components
/root/openshift/odo/tests/helper/helper_run.go:34

[Fail] odo service command tests [BeforeEach] checking machine readable output for service catalog should succeed listing catalog components
/root/openshift/odo/tests/helper/helper_run.go:34

[Fail] odo service command tests [BeforeEach] create service with Env non-interactively should be able to create postgresql with env
/root/openshift/odo/tests/helper/helper_run.go:34

Ran 12 of 13 Specs in 0.934 seconds
FAIL! -- 0 Passed | 12 Failed | 0 Pending | 1 Skipped

All tests for test-cmd-service. Analyzing to resolve failures
Summarizing 12 Failures:

[Fail] odo service command tests [BeforeEach] When working from outside a component dir should be able to list services, as well as json list in a given app and project combination
/root/openshift/odo/tests/helper/helper_run.go:34

[Fail] odo service command tests [BeforeEach] When describing services should succeed when we're describing service that could have integer value for default field
/root/openshift/odo/tests/helper/helper_run.go:34

[Fail] odo service command tests [BeforeEach] check catalog service search functionality check that a service does not exist
/root/openshift/odo/tests/helper/helper_run.go:34

[Fail] odo service command tests [BeforeEach] create service with Env non-interactively should be able to create postgresql with env multiple times
/root/openshift/odo/tests/helper/helper_run.go:34

[Fail] odo service command tests [BeforeEach] When working from outside a component dir should be able to create, list and delete a service using a given value for --context
/root/openshift/odo/tests/helper/helper_run.go:34

[Fail] odo service command tests [BeforeEach] When the application is deleted should delete the service(s) in the application as well
/root/openshift/odo/tests/helper/helper_run.go:34

[Fail] odo service command tests [BeforeEach] check search functionality should pass with searching for part of a service name
/root/openshift/odo/tests/helper/helper_run.go:34

[Fail] odo service command tests [BeforeEach] When working from outside a component dir should be able to create, list and delete services without a context and using --app and --project flags instaed
/root/openshift/odo/tests/helper/helper_run.go:34

[Fail] odo service command tests [BeforeEach] when running help for service command should display the help
/root/openshift/odo/tests/helper/helper_run.go:34

[Fail] odo service command tests [BeforeEach] checking machine readable output for service catalog should succeed listing catalog components
/root/openshift/odo/tests/helper/helper_run.go:34

[Fail] odo service command tests [BeforeEach] checking machine readable output for service catalog should succeed listing catalog components
/root/openshift/odo/tests/helper/helper_run.go:34

[Fail] odo service command tests [BeforeEach] create service with Env non-interactively should be able to create postgresql with env
/root/openshift/odo/tests/helper/helper_run.go:34

Ran 12 of 13 Specs in 0.934 seconds
FAIL! -- 0 Passed | 12 Failed | 0 Pending | 1 Skipped

Tests are now passing on PSI minishift:
------------------------------
Running tests...
------------------------------

+ make test-cmd-project
ginkgo  -randomizeAllSpecs -slowSpecThreshold=120 -timeout 7200s -nodes=1 -focus="odo project command tests" tests/integration/project/
Running Suite: Project Suite
============================
Random Seed:<PHONE_NUMBER> - Will randomize all specs
Will run 7 of 7 specs

•••••••
JUnit report was created: /home/fedora/openshift/odo/reports/junit_2021-2-2_22-07-31_1.xml

Ran 7 of 7 Specs in 24.943 seconds
SUCCESS! -- 7 Passed | 0 Failed | 0 Pending | 0 Skipped
PASS

Ginkgo ran 1 suite in 28.74755811s
Test Suite Passed
+ make test-cmd-service
ginkgo  -randomizeAllSpecs -slowSpecThreshold=120 -timeout 7200s -nodes=2 -focus="odo service command tests" tests/integration/servicecatalog/
Running Suite: Servicecatalog Suite
===================================
Random Seed:<PHONE_NUMBER> - Will randomize all specs
Will run 13 specs

Running in parallel across 2 nodes

•••S•••••
------------------------------
• [SLOW TEST:170.968 seconds]
odo service command tests
/home/fedora/openshift/odo/tests/integration/servicecatalog/cmd_service_test.go:13
  create service with Env non-interactively
  /home/fedora/openshift/odo/tests/integration/servicecatalog/cmd_service_test.go:81
    should be able to create postgresql with env multiple times
    /home/fedora/openshift/odo/tests/integration/servicecatalog/cmd_service_test.go:100
------------------------------
•••
Ran 12 of 13 Specs in 225.586 seconds
SUCCESS! -- 12 Passed | 0 Failed | 0 Pending | 1 Skipped

Ginkgo ran 1 suite in 3m48.081352406s
Test Suite Passed

@rnapoles-rh I tried replicating the steps you mentioned above but minishift start fails on
# minishift start
-- Starting profile 'minishift'
-- Check if deprecated options are used ... OK
-- Checking if https://github.com is reachable ... OK
-- Checking if requested OpenShift version 'v3.11.0' is valid ...
   Hit github rate limit: GET https://api.github.com/repos/openshift/origin/releases/tags/v3.11.0: 403 API rate limit exceeded
[...]

Might be I am doing something silly and end up with rate limit. Could please summarise e2e process/step in a single comment.

Steps to setup OpenStack Linux VM:
#Install openstack as per: https://docs.openstack.org/mitaka/cli-reference/common/cli_install_openstack_command_line_clients.html

#Download the OpenStack RC file from https://cloud.psi.redhat.com/dashboard/project/api_access/

#Source the rc file devtools-odo-openrc.sh

#Edit the rc file to set user (OS_USERNAME) and password (OS_PASSWORD)

#Create VM:
openstack server create --flavor m1.large --image Fedora-Cloud-Base-32 --nic net-id=provider_net_shared_3 --security-group default --security-group all-open --key-name releng-pub-key odo-fedora-minishift

#ssh to the vm:
ssh -i ~/psi/PSI/lib/common/releng-key<EMAIL_ADDRESS>
sudo -i
dnf install grubby
grubby --update-kernel=ALL --args="systemd.unified_cgroup_hierarchy=0"
systemctl reboot
ssh -i ~/psi/PSI/lib/common/releng-key fedora@<ip_address>
sudo -id

#Install docker:
#Follow https://docs.docker.com/engine/install/fedora/

sudo dnf -y install dnf-plugins-core

sudo dnf config-manager --add-repo https://download.docker.com/linux/fedora/docker-ce.repo

sudo dnf install docker-ce docker-ce-cli containerd.io

sudo systemctl start docker

#Setup virtualization environment by following:
https://docs.okd.io/3.11/minishift/getting-started/setting-up-virtualization-environment.html#kvm-driver-fedora

sudo dnf install libvirt qemu-kvm
sudo usermod -a -G libvirt $(whoami)
sudo newgrp libvirt
curl -L https://github.com/dhiltgen/docker-machine-kvm/releases/download/v0.10.0/docker-machine-driver-kvm-centos7 -o /usr/local/bin/docker-machine-driver-kvm
sudo chmod +x /usr/local/bin/docker-machine-driver-kvm
sudo dnf install @virtualization
sudo dnf group install --with-optional virtualization
sudo systemctl start libvirtd
sudo systemctl enable libvirtd
sudo virsh net-autostart default

#Install minishift:
curl -Lo minishift.tgz https://github.com/minishift/minishift/releases/download/v1.34.3/minishift-1.34.3-linux-amd64.tgz
tar -xvzf minishift.tgz
sudo mv minishift-1.34.3-linux-amd64/minishift /usr/local/bin

sudo dnf install make
sudo dnf install gcc
sudo dnf install git
sudo dnf install wget
sudo wget https://golang.org/dl/go1.13.linux-amd64.tar.gz
sudo tar -C /usr/local -xzf go1.13.linux-amd64.tar.gz

#Login to docker to avoid getting "ERROR: toomanyrequests: Too Many Requests. You have reached your pull rate limit." during minishift start
docker login --username <user_name> --password <user_token>

Steps to run tests (scripts/minishift-all-tests.sh):
#!/usr/bin/env bash

executing() {
   set +x
  echo -e "\n------------------------------\n${1}\n------------------------------\n"
   set -x
}

set -ex

#Export GitHub token to avoid
export MINISHIFT_GITHUB_API_TOKEN="03d362b0f2a7ecf02b30bfbdd40fe720a4d6b8b3"
executing "Setting up environment..."

export PATH="$PATH:/usr/local/go/bin/"
export GOPATH=$HOME/go
git clone https://github.com/openshift/odo.git openshift/odo
cd openshift/odo

mkdir -p $GOPATH/bin
make goget-ginkgo
export PATH="$PATH:$(pwd):$GOPATH/bin"

executing "Building ODO..."
make bin
sudo cp odo /usr/bin

executing "Stopping minishift..."
minishift stop
yes | minishift delete
MINISHIFT_ENABLE_EXPERIMENTAL=y

executing "Starting minishift..."
minishift start
executing "Adding components: service-catalog, automation-service-broker, and template-service-broker ..."
minishift openshift component add service-catalog
minishift openshift component add automation-service-broker
minishift openshift component add template-service-broker
sleep 3m
eval $(minishift oc-env)

executing "Executing tests..."
make test-cmd-project
make test-cmd-service

executing "Removing cloned repo..."
rm -Rf openshift

I am having problems to get minishift started:
Could not set oc CLI context for 'minishift' profile: Error during setting 'minishift' as active profile: The specified path to the kube config '/home/fedora/.minishift/machines/minishift_kubeconfig' does not exist

Created an issue https://github.com/openshift/odo/issues/4410 to track work for Jenkins configuration for running tests on PSI minishift

Left to be done:
[ ] Fine tune the minishift-all-tests.sh script

Splitting scripts: execution, clean up, setup
Reorganizing to prep VM after cleanup and check if VM is stable (testing the script that verifies stability)

Blocked by #4523.

Due to issue #4523 we have to separate minikube and minishift from a single script. Direct dependency with #4288
Requires:

[ ] adjust Jenkins script
[ ] Create json file suing latest releng-key
[ ] Test all pieces together (Jenkins, scripts in openshift/release repo, scripts in openshift/odo repo, and gitlab.cee.redhat.com/openshiftdo/psi-jenkins/ repo)

Getting

For the issue of pulling images from docker.io, can you PTAL if minishift image cache-config command can help? I think it will cache the image on the host so that images don't need to be pulled again when someone does minishift delete and then minishift start.
But I could be wrong.

Exploring minishift image cache-config and the use of local registry mirror

I was able to manually pull all images required by minishift and add them to the cache:
Issues starting minishift:

Went back to standard minishift due to the CDK (3.11) issues.

Found this minishift issue where people were able to resolve this issue with fedora31. I will try on a fedora31 PSI VM

When running when running provision-hosts.sh I get the following:

[x] PSI resource allocation was increased.
[x] Install ansible and required pre-requesites in control machine (Linux VirtualBox)
[ ] Provision 3.11 cluster
[ ] After 3.11 provisioning run tests and ensure all test pass, fix failing tests or report bugs as required

I was able to move forward from the previous errors, the ocp-ansible-machine instance got created, then it fail due to timeout waiting for it, so I ran the provision-hosts.sh script again and then it created the ocp-dns-machine. Failed again due to timeout, ran it again and now I get the following, note I changed the permissions on releng-key and releng-key pub to 600 and still get this:
PLAY RECAP ********************************************************************************************************************************************************************************************
<IP_ADDRESS>                  : ok=2    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
++ awk '{print $2}'
++ grep ansible
++ cat /tmp/ansible-generated-files/hosts-ip
+ ANSIBLE_IP=<IP_ADDRESS>
++ awk '{print $2}'
++ grep ssh_key
++ cat /tmp/ansible-generated-files/hosts-ip
+ SSH_KEY_NAME=
+ SSH_KEY_PATH=/home/rnapoles/PSI/ocp-311-cluster/
+ ssh -o 'StrictHostKeyChecking no' -i /home/rnapoles/PSI/ocp-311-cluster/<EMAIL_ADDRESS>rm -rf /root/deploy-ocp-crew
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0775 for '/home/rnapoles/PSI/ocp-311-cluster/' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.
Load key "/home/rnapoles/PSI/ocp-311-cluster/": bad permissions
<EMAIL_ADDRESS>password:
Connection closed by <IP_ADDRESS> port 22
+ scp -o 'StrictHostKeyChecking no' -i /home/rnapoles/PSI/ocp-311-cluster/ -r /home/rnapoles/PSI/ocp-311-cluster root@<IP_ADDRESS>:/root/deploy-ocp-crew
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0775 for '/home/rnapoles/PSI/ocp-311-cluster/' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.
Load key "/home/rnapoles/PSI/ocp-311-cluster/": bad permissions
<EMAIL_ADDRESS>password:
Permission denied, please try again.
<EMAIL_ADDRESS>password:
Permission denied, please try again.
<EMAIL_ADDRESS>password:
<EMAIL_ADDRESS>Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
lost connection
+ ssh -o 'StrictHostKeyChecking no' -i /home/rnapoles/PSI/ocp-311-cluster/<EMAIL_ADDRESS>rm -rf /tmp/ansible-generated-files
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0775 for '/home/rnapoles/PSI/ocp-311-cluster/' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.
Load key "/home/rnapoles/PSI/ocp-311-cluster/": bad permissions
<EMAIL_ADDRESS>password:
Permission denied, please try again.
<EMAIL_ADDRESS>password:
Permission denied, please try again.
<EMAIL_ADDRESS>password:
<EMAIL_ADDRESS>Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
+ ssh -o 'StrictHostKeyChecking no' -i /home/rnapoles/PSI/ocp-311-cluster/<EMAIL_ADDRESS>mkdir /tmp/ansible-generated-files
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0775 for '/home/rnapoles/PSI/ocp-311-cluster/' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.
Load key "/home/rnapoles/PSI/ocp-311-cluster/": bad permissions
<EMAIL_ADDRESS>password:
ssh_dispatch_run_fatal: Connection to <IP_ADDRESS> port 22: Broken pipe
+ scp -o 'StrictHostKeyChecking no' -i /home/rnapoles/PSI/ocp-311-cluster/ /tmp/ansible-generated-files/main-configuration-inventory root@<IP_ADDRESS>:/tmp/ansible-generated-files/main-configuration-inventory
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0775 for '/home/rnapoles/PSI/ocp-311-cluster/' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.
Load key "/home/rnapoles/PSI/ocp-311-cluster/": bad permissions
<EMAIL_ADDRESS>password:
Permission denied, please try again.
<EMAIL_ADDRESS>password:
Permission denied, please try again.
<EMAIL_ADDRESS>password:
<EMAIL_ADDRESS>Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
lost connection
+ scp -o 'StrictHostKeyChecking no' -i /home/rnapoles/PSI/ocp-311-cluster/ /tmp/ansible-generated-files/ocp-instalation-inventory root@<IP_ADDRESS>:/tmp/ansible-generated-files/ocp-instalation-inventory
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0775 for '/home/rnapoles/PSI/ocp-311-cluster/' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.
Load key "/home/rnapoles/PSI/ocp-311-cluster/": bad permissions
<EMAIL_ADDRESS>password:
Permission denied, please try again.
<EMAIL_ADDRESS>password:
Permission denied, please try again.
<EMAIL_ADDRESS>password:
<EMAIL_ADDRESS>Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
lost connection
+ ssh -o 'StrictHostKeyChecking no' -i /home/rnapoles/PSI/ocp-311-cluster/<EMAIL_ADDRESS>chmod 0600 /root/deploy-ocp-crew/
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0775 for '/home/rnapoles/PSI/ocp-311-cluster/' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.
Load key "/home/rnapoles/PSI/ocp-311-cluster/": bad permissions
<EMAIL_ADDRESS>password:
Permission denied, please try again.
<EMAIL_ADDRESS>password:
Permission denied, please try again.
<EMAIL_ADDRESS>password:
<EMAIL_ADDRESS>Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
+ ssh -o 'StrictHostKeyChecking no' -i /home/rnapoles/PSI/ocp-311-cluster/<EMAIL_ADDRESS>/root/deploy-ocp-crew/deploy-ocp.sh
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0775 for '/home/rnapoles/PSI/ocp-311-cluster/' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.
Load key "/home/rnapoles/PSI/ocp-311-cluster/": bad permissions
<EMAIL_ADDRESS>password:
Permission denied, please try again.
<EMAIL_ADDRESS>password:
Permission denied, please try again.
<EMAIL_ADDRESS>password:
<EMAIL_ADDRESS>Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
[rnapoles@localhost ocp-311-cluster]$ ls -las
total 28
0 drwxrwxr-x. 1 rnapoles rnapoles  292 May 26 07:53 .
0 drwxrwxr-x. 1 rnapoles rnapoles  270 May 10 13:09 ..
4 -rw-rw-r--. 1 rnapoles rnapoles 2257 May 10 13:09 cluster-configuration
0 drwxrwxr-x. 1 rnapoles rnapoles   24 May 10 13:09 common
4 -rwxrwxr-x. 1 rnapoles rnapoles  718 May 10 13:09 configure-ansible-host.sh
4 -rwxrwxr-x. 1 rnapoles rnapoles 1743 May 10 13:09 deploy-ocp.sh
0 drwxrwxr-x. 1 rnapoles rnapoles  250 May 10 13:09 hosts-provision
0 drwxrwxr-x. 1 rnapoles rnapoles   56 May 10 13:09 htpasswd
0 drwxrwxr-x. 1 rnapoles rnapoles  252 May 10 13:09 ocp-setup
4 -rwxrwxr-x. 1 rnapoles rnapoles 2828 May 10 13:09 provision-hosts.sh
4 -rw-rw-r--. 1 rnapoles rnapoles  266 May 10 13:09 provision.py
4 -rw-rw-r--. 1 rnapoles rnapoles 2686 May 10 13:09 README.md
4 lrwxrwxrwx. 1 rnapoles rnapoles   24 May 10 13:09 releng-key -> ../lib/common/releng-key
[rnapoles@localhost ocp-311-cluster]$ ls -las ../lib/common/
total 20
0 drwxrwxr-x. 1 rnapoles rnapoles  118 May 20 13:50 .
0 drwxrwxr-x. 1 rnapoles rnapoles   68 May 10 13:09 ..
4 -rw-------. 1 rnapoles rnapoles  245 May 10 13:09 clouds.yaml
4 -rw-------. 1 rnapoles rnapoles  118 May 10 13:09 helpers.sh
4 -rw-------. 1 rnapoles rnapoles  779 May 10 13:09 htpass
0 drw-------. 1 rnapoles rnapoles   48 May 10 13:09 old-keys
4 -rw-------. 1 rnapoles rnapoles 1679 May 10 13:09 releng-key
4 -rw-------. 1 rnapoles rnapoles  399 May 10 13:09 releng-key.pub
[rnapoles@localhost ocp-311-cluster]$

3.11 provisioning failing with the following after creating all VM instances:
TASK [Gathering Facts] *********************************************************
task path: /root/deploy-ocp-crew/ocp-setup/post-install-actions.yml:3
/usr/lib/python2.7/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.24.3) or chardet (3.0.4) doesn't match a supported version!

following up with Zeeshan
Just started exploring cluster deployment in the ibm cloud. IBM cloud account created. Checking with Karel about budget.

Onboarded to the IBM Cloud. Currently reviewing guidelines, documentation, and troubleshooting permissions.

Created 4.7 cluster in the IBM Cloud. Looking on how to provision a 3.11 cluster (depreciated in IBM Cloud). Also checking  how to onboard a robot account.

Looking on how to provision a 3.11 cluster (depreciated in IBM Cloud).

minishift can work with a remote VM. Might be worth exploring.

Closing this issue as we don't support 3.11 and hence doesn't make sense to have test running on it.


11.0.0 DSiWare Downgrade Section II fails

The inject of the FBI_inject_with_banner.app fails with this message:

The 3DS #1 was handled by the letter of this guide.

Well, that's what GW and/or rxTools does sometimes, fragmenting the CTRNAND... There's no way Decrypt9 can handle fragmentation, but there's a way around. Read this post carefully, and decide if you want to do it. As I wrote in that post, Decrypt9's injection routines are known to work reliably, and there was never a single case were anyone complained about failures. The code for that in Decrypt9 being pretty simple and mainly constisting of safety clamps also helps. Still, the normal advice for people without a NANDmod is to stay away from bigger (or even any) SysNAND modifications.
https://gbatemp.net/threads/release-inject-any-app-into-health-safety-o3ds-n3ds-cfw-only.402236/page-6


Move Headers phases before Sources phases

I don't think it's going to be a problem for this framework, but this is the correct way to order these. The Swift files depend on the framework header, so it should be operated on first.

👍


How to get the map appear like a flat one instead of the way  it is on a curved surface?

Pass the projection="geoMercator" prop to ComposableMap.
<ComposableMap projection="geoMercator" width="1000" projectionConfig={{ rotate: [-10, 0, 0], scale: 150 }} >


Remove version: BlenderFoundation.Blender.LTS version 3.3.9

Old Package
Microsoft Reviewers: Open in CodeFlow

URL: https://download.blender.org/release/Blender3.3/blender-3.3.9-windows-x64.msi
OK
(Automated message - build 888)


Correct issues with integration tests and LocalStack

Prior to this change, the integration test invoked the Lambda knowing that it would fail to execute.  However, the failure indicated the Lambda was installed and that was deemed a sufficient integration test for the time being.  The Lambda failed because the free version of LocalStack does not support the AWS organizations service so calls to that service were attempting to reach out to AWS itself with erroneous account information.
To work around that problem, the LocalStack docker container was modified to additionally run moto_server to provide a mock AWS organizations server.  A docker network also was created for the LocalStack/Moto docker container.
Wherever the Lambda created a client for an AWS service, an endpoint_url argument was added to point to the LocalStack network when the Lambda detected it had been invoked by LocalStack (i.e.,LOCALSTACK_HOSTNAME was non-null).
The LocalStack/moto docker container needed to specify the environment variable for LAMBDA_DOCKER_NETWORK, and both organizations and support was added to the SERVICES environment variable.
The integration test was modified to create an organization using moto and an organization account_id for use in the event passed to the Lambda.  All client calls also had an endpoint_url added to point to the LocalStack network.  The test was also modified to expect a successful Lambda execution and to test that a support case was added.

Loren has sent me two collections of rewrites for this code; those files can be found in Trello.

@lorengordon Forgot to bump the version, so I created another PR.


Remove version: CrystalDewWorld.CrystalDiskInfo.KureiKeiEdition version 8.11.1

[Automated] It returns code over 400 in all urls
Microsoft Reviewers: Open in CodeFlow

Close with reason: Duplicate;


Typings from npm are wrong and version 3.8.1 is already 11 months old

Hello,
when I install the latest version of Immutable from NPM, I get the 3.8.1 version which is somewhat outdated. Is there a chance that you will deploy a new version soon?
Thanks

Yes - a new version is on its way soon


Support for .NET 6 RC1

Is your feature request related to a problem? Please describe.
.NET 6 RC1 was released.  Please add support for this SDK and framework to the actions

Oryx has been updated to support RC1. Thanks @william-msft

To clarify, this enables building .NET 6 RC1 Blazor apps. We're actively working on support for .NET 6 function apps.


On IOS Safari, md-button needs to click twice

dup #1792

Have you tried the solution from #1792?

First, I have updated the version to @angular/material": "^2.0.0-alpha.10
Also, I have update the code in the following file based on the instruction of #1792.
However,  the problem is still exist. It looks like the issue is ONLY on md-button but not on md-raised-button.

I'll give it another look, in that case. Regarding this:

...  It looks like the issue ONLY exists on md-button but not on md-raised-button.

If you comment out this line, does the raised button break again?

Could you post a simplified example @johnqiuwan, I've been trying this out locally against the master branch and it works fine, both on raised buttons and on regular ones:
Click counter: {{ clickCounter }}
<button md-button (click)="clickCounter=clickCounter+1" color="primary">Click</button>
<button md-raised-button (click)="clickCounter=clickCounter+1" color="primary">Click</button>

Hi @crisbeto,

	Register Now

Only on ios, the "click" not been caught on the first click but on the second click. If I use md-raised-button instead of md-button, the problem is gone.
In addition, it has no issue on Android and Web.

Hmm, I hadn't tried with a link, I was only testing with button. Does the .banner-button class have any special styling (:hover, :active etc.)?

I just enable / disable the :hover on the .banner-button class. However, it is not working on both cases on ios. No matter the .banner-button class has a :hover or not.

I see, I'll try it out with links later.

Still works fine @johnqiuwan, I used the exact same HTML. It might be better to revisit this once the fix in master has been released.

@johnqiuwan alpha.11 was released with the fix for this. Can you give it another shot?

It's been a while since the last response and it has been 2 releases since the fix for this was released. I'll close the issue, but it can always be re-opened if it's still a problem.

Sorry for the late reply. I have confirmed that the alpha.11 fixes the bug
Thank you for your great work : )


"Connect Domain" more than once (i.e., multiple domains)

I've noticed in my testing, as well as in the screenshots here, the "Connect Domain" button becomes greyed out once you connect a single domain.  IF my Email Communication Service has 3 domains, for example, I have to pick just 1 of those in the "Connect Domain" process, and then that selected domain is the only one whose MailFrom addresses can be used.
I thought maybe it was just a GUI thing and tested sending an email from another domain/MailFrom address but get this:
The server response was: 5.1.7 The specified sender domain has not been linked.

I can't find a command to get extra domains connected either for PowerShell or CLI
Any ideas?

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: d6801ca2-0239-f782-d7a6-5af175298673
Version Independent ID: c08324f9-bda2-c1a4-cc92-9c7e6e7fc312
Content: How to connect a verified email domain - An Azure Communication Services quick start guide
Content Source: articles/communication-services/quickstarts/email/connect-email-communication-resource.md
Service: azure-communication-services
GitHub Login: @bashan-git
Microsoft Alias: bashan

@JeremyTBradshaw
Thanks for your feedback! We will investigate and update as appropriate.

Your feedback has been shared with the content owner for further review.

Same problem here. Adding a separate ACS instance per domain as advised here: https://learn.microsoft.com/en-us/answers/questions/1305388/can-we-associate-multiple-domains-to-an-azure-comm is not an option of course.
This needs to be fixed ASAP.