How to confirm D4xx cameras are in Slave mode?

Required Info

Camera Model
D415 D435 D455

Firmware Version
just updated by realsense-viewer

Operating System & Version
Ubuntu 18.04 Windows 10

Kernel Version (Linux Only)
4.18  or Raspbian img(https://dev.intelrealsense.com/docs/open-source-ethernet-networking-for-intel-realsense-depth-cameras)

Platform
PC/Raspberry Pi

SDK Version
{ legacy / 2.. }

Language
c/c++

Issue Description
So, I set the external sync mode to 2(slave mode). How do I verify sync mode is set?

According to the document, "When the Genlock mode is selected, the camera will wait indefinitely for a trigger." The problem is, the camera DOES NOT wait indefinitely in slave mode when sync pulse is still NOT kicked in; it runs as usual. So, how can I verify if sync is doing its job?

NO SYNC in this picture, but the camera still runs.

Hi @bobluoluo   Mode '2' (Slave) on the Inter Cam Sync Mode settings is not genlock.  Mode '3' (Full Slave) and above belongs to the External Synchronization (genlock) system.
Modes 1 (Master) and 2 (Slave) originate from an earlier hardware sync system that does not have Mode 3 and above.  Mode 2 (Slave) behaves differently from the genlock system.  In Mode 2, the camera listens on each frame for a sync trigger pulse from a Master camera or from a signal generator, and if it does not detect that pulse within a certain time period then it stops listening and initiates capture independently (unsynced).  Mode 2 does not wait indefinitely.
The hardware sync system that uses Mode 1 and 2 is described in the Intel white-paper document linked to below.  This system is mature and validated by Intel.   The more recent genlock system (Mode 3+) is not considered mature, but you are free to experiment with it.
https://dev.intelrealsense.com/docs/multiple-depth-cameras-configuration
In regard to verifying whether hardware sync is occurring or whether the streams are unsynched, Section 3 of the paper explains that you can observe whether timestamps are drifting apart over time to confirm sync.  If there is drift then hardware sync is active.  If the timestamps do not drift then the system is unsynced.  This paradox is explained in Section 3 in the paragraph with the bolded text Now to the somewhat counter intuitive aspect of time stamps.
Sync can also be validated by using high-speed LED panels, though it is expected that most users will not need to use this high-tech apprpoach.
https://dev.intelrealsense.com/docs/multiple-depth-cameras-configuration#section-g-hw-sync-validation
https://support.intelrealsense.com/hc/en-us/community/posts/360049401673/comments/360012819533

@MartyG-RealSense there still seems to be a lot of confusion about Mode '3' (Full Slave) and the Genlock Modes '4+'
According to the whitepaper and @RealSenseSupport in this issue https://github.com/IntelRealSense/librealsense/issues/8419 Mode '3' is intended to operate exactly the same as Mode '2', except that it is supposed to also sync the color camera as well, hence "Full".
Unfortunately Mode '3' is just a placeholder for now as that feature has not been implemented.
Modes '4+' are the "genlock" modes that allow arbitrary framerates from an external source.
Because you have recently been describing Modes '4+' as "experimental" and Mode '3' is "unfinished" I sort of get lumping them together. But because Modes '4+' have additional limitations I wanted to make sure that I am understanding all of this correctly.
Thanks.

@sam598   I have no further information to add beyond the already stated position that Mode 1 and 2 belong to the mature and validated hardware sync system.  You are free to experiment with Mode 3+ but there is no technical support that I can provide for that.

Hi @bobluoluo  Do you require further assistance with this case, please?  Thanks!

Thanks you all for the helpful information about the sync. Frames under sync mode 4 can be triggered individually by a 30fps sync signal. However it realsense-viewer shows the frame rate is @15fps. What can I do to fix this.

Hi @bobluoluo   If you are using a master and slave camera under the genlock system (and I see in the image above that you are using Mode 4) then the slave camera should use a higher FPS than the master camera.  This is described in the section of the genlock paper linked to below.
https://dev.intelrealsense.com/docs/external-synchronization-of-intel-realsense-depth-cameras#section-2-principles-of-operation
To quote this section:
"In order to have all cameras stream at 30fps, for example, the Master must be set to 30fps, and all Genlock cameras must be natively set to 90fps modes with burst count set to 1.  The master camera will then deliver the trigger at 30fps, and all Genlock cameras will trigger in sync with the master".

I set the trigger to 60 fps, native 60 fps . And it gives me this.

I guess my question is how can I get 30 fps under mode 4?

I can't believe I missed such important text. I must be blind. :-)
So, I tried native mode 60fps + trigger 30fps and 60fps + trigger 60fps. Neither of them worked.
How do I setup a 30fps capture under mode 4 correctly?

I can't believe I missed such important text. I must be blind. :-)
So, I tried native mode 60fps + trigger 30fps and 60fps + trigger 60fps. Neither of them worked.
How do I setup a 30fps capture under mode 4 correctly?

Little update. Previous comment, its experiment was made on a Raspberry PI with RTSP solution.
The good news is the configuration of native 60 fps + pulse 30 fps is working under mode 4 with USB(direct connection) solution.

If you are using the RTSP protocol that is part of Intel' open-source ethernet networking system that uses rs-server then rs-server has some steaming limitations at the time of writing this.  In order to use 60 FPS, a resolution of 480x270 should be used.
https://dev.intelrealsense.com/docs/open-source-ethernet-networking-for-intel-realsense-depth-cameras#section-3-6-software-limitations
A new networking extension called LRS-Net 2.0 will be in an upcoming SDK version that has support for a greater range of resolutions, formats and FPs speeds.
https://github.com/IntelRealSense/librealsense/issues/8372

Do we have a release date? I can't wait. :-)

I do not have a release date, though its developer welcomes people testing it before its release.  Their LRS-Net 2.0 fork of librealsense is in the link below.
https://github.com/IntelRealSenseArchive/lrs-net/tree/apuzhevi_lrsnet

Just encountered a couple of compiler error. Do I need some extra dependencies? What flags should I use (btw, master  compiles just fine.)

My cmake cmoonad is : cmake .. -DBUILD_NETWORK_DEVICE=ON -DFORCE_RSUSB_BACKEND=ON

What method did you use to compile the source code please?
Did you download the source as a zip file using the Code button on its GitHub, extract the folder and then mkdir build && cd build?

yes, I cloned the fork branch and it gives me compiler error.
The "master-branch" under the primary branch https://github.com/IntelRealSense/librealsense has no build issue is what I meant. Sorry about the confusion.

I will tag the LRS-Net developer @apuzhevi into this discussion to seek input on how to compile his LRS-equipped fork of librealsense.  Thanks very much @bobluoluo

yes, I cloned the fork branch using the link you gave me and it outputs compiler error during build. e.g. make . -j4

OKay. I think I found the problem. I made a mistake. The clone command provided by github does not switch to the default branch automatically. It is still pointing at the very old master branch when it was forked.

the code builds successfully on branch apuzhevi_lrsnet

I'm now running the new rs-server. How do I solve this issue?

Hi @bobluoluo   Excellent to hear that you built the fork successfully.
I guess that the issue you are referring to is the side-panel options showing values of zero?  If so, what do the drop-down menus look like when opened (if they are openable)?

no items in the drop downs

Did you set the static IP in the Pi's secure terminal, which is a step in the ethernet white-paper that comes before setting the camera's Network Device number?
https://dev.intelrealsense.com/docs/open-source-ethernet-networking-for-intel-realsense-depth-cameras#section-2-6-setting-static-ip

Sorry again, I forgot to update the code on the client side too. Now I got this.
Now I got this. There might be a bug in the UI I see two of the same IP camera interface in the device list.

Thanks very much for the illustrated reports of your progress.  If you start the streams of one of the listed cameras, does it stream correctly please?

I tried starting a depth sensor with following settings and it failed.

FYI, at the server side(the machine running rs-server), it does show it support this profile.

And couple of error/warning at server side.

If you have two D455 listed, does streaming work if you close the lower one in the side-panel?

yes, the x button works.

Hi @bobluoluo  Do you have new progress to report since your problem with enabling a sensor, please?

I can't turn on the camera on any allowed profile. So I'm writing up some code for testing.
in the meantime, I got some error when running  sudo make install
CMake Error at src/network/cmake_install.cmake:128 (file):
file INSTALL cannot find
"/home/bob/dev/lrs-net/build/src/ethernet/realsense2-netConfig.cmake".
Call Stack (most recent call first):
cmake_install.cmake:132 (include)

I will seek advice from the LRS-Net developer about your current problems.  Thanks very much for your patience.

Hi @bobluoluo   I just wanted to provide a quick update that the developer of RS2-Net is looking at the information that you have kindly posted above.

btw, the "make install" script seems to be broken too.
-- Installing: /usr/local/include/librealsense2-net/rs_net.hpp
-- Installing: /usr/local/lib/cmake/realsense2-net/realsense2-netTargets.cmake
-- Installing: /usr/local/lib/cmake/realsense2-net/realsense2-netTargets-noconfig.cmake
CMake Error at src/network/cmake_install.cmake:128 (file):
file INSTALL cannot find
"/home/bob/dev/lrs-net/build/src/ethernet/realsense2-netConfig.cmake".
Call Stack (most recent call first):
cmake_install.cmake:132 (include)
Makefile:85: recipe for target 'install' failed
make: *** [install] Error 1

Thanks very much @bobluoluo for the update of your feedback!

I actually have two questions related to the new rs-server. 1. What is the approximate release date for the new rs-server/receiver, a week or two? 2. Will it provide some synchronization method over the network with like NTP or PTP or something else. The reason why I ask is I'm evaluating if I can build a reliable Realsense project based on it. Thanks.

There is not currently a release date for LRS-Net 2.0.  Given that SDK 2.43.0 has just been released, it would not be available in an SDK release at least during the next month from the time of writing this.

There is not a mention about synchronization on the advance-released feature list.

https://github.com/IntelRealSense/librealsense/pull/8343

Hi @bobluoluo  Do you require further assistance with this case, please?  Thanks!

yes, please help me. How do I fix this.  it was compiled with make -j1

@bobluoluo  I will contact the LRS-Net developer again.

I downloaded the pre-release 2.44.0. Compiler error is fixed but the rs-server is no good.

usb has no issue. firmware is the latest.

I note that the D455 camera above is listed in the options side panel as D455.  Have you tried it configured as a Network Camera like in your previous tests, please?

I configured a D455 on a Raspberry PI and ran rs-server(V2.44.0). I launched realsense-viewer(v2.44.0) on a server. Like this.

This is the error I got on the server

In my previous previous test on lrs-net 2.0.  Server sees two cameras even though I attached  only one D455 on the Raspberry PI.

And then,  when I enable a camera either stereo or rgb camera, the rs-server crashes with no useful debug info but segfault.

In summary, I tested both lrs-net 2.0 and v2.44.0 on the same hardware. Unfortunately, none of them works without issue.
I also tested lrs-net-dev branch. It has a compiler error.

Another RealSense user encountered the same problem with accessing the network camera in the RealSense Viewer today, in SDK 2.43.0 in that particular case.
https://github.com/IntelRealSense/librealsense/issues/8766

Hi @bobluoluo   Regarding your errors with rs-server in https://github.com/IntelRealSense/librealsense/issues/8529#issuecomment-814275813 - the original version of rs-server in the SDK supports a limited range of resolutions listed in the link below.
https://dev.intelrealsense.com/docs/open-source-ethernet-networking-for-intel-realsense-depth-cameras#section-3-6-software-limitations
424x240 supports 6, 15, 30 and 60 FPS for the RGB stream.  For Z16 depth, 480x270 should be used with FPS speeds 6, 15, 30 or 60.

Also, please do not use genlock with rs-server and instead set Inter Cam Sync Mode to '0' in the Viewer if you are using a mode other than 0.

@MartyG-RealSense  The original limited version has no issue. Please don't get confused. I'm able to run it with or without gen-lock mode enabled on the original code. The problem is in the lrs-net and v2.44.0-pre. Either mode 0 or not, the program crashes in lrs-net, v2.44.0. We are looking forward to using this new lrs-net with more resolution options.

Please let know if there is a plan of fixing the bugs in lrs-net or v2.44.0-pre. Thanks

We have the same problem with the 2.44 viewer; 2.42 works fine. It got broken from 2.43

Hi @bobluoluo and @mirkocomparetti-synesis   This case and the recent problems encountered with 2.43 / 2.44 has already been highlighted to the networking developer.   I cannot provide a time estimate for when a response will be received though.

The other thing I want to emphasis is I found out I can't launch more than one net_device in neither old code and new code.

Hi @bobluoluo  Have you made further progress in this case yet?  Thanks!

nope

Unfortunately I believe that there is nothing more that I can add to this case as it requires the input of the networking developer and I cannot compel them to respond.  I do apologize.

Thank you for the patient assistance.

I thought I'd bring to your attention an item in the release notes of the new 2.45.0 SDK that was released today.
https://github.com/IntelRealSense/librealsense/releases/tag/v2.45.0

[LRS-Net]  Fix connecting to network device with realsense-viewer
There was changes introduced in Feb 15 in model-views.cpp that breaks connections to network devices from realsense-viewer.
This PR fixes this.
https://github.com/IntelRealSense/librealsense/pull/8751

Hello, at least this works.
Still compiling source with python does not generate the full library (i.e. missing pipeline object) on arm...

Hi @mirkocomparetti-synesis   Could you provide more information about your Python problem please, including any error message that you receive?

Hello @MartyG-RealSense sure! https://github.com/IntelRealSense/realsense-ros/issues/1804#issuecomment-828210420 here it is!

Thanks very much for the link.  The RealSense team member RealSenseSupport is handling that case, so please leave them another comment there.  As you are using a Pi, the only suggestion I would add is to use the version of Ubuntu Server 20.04 provided at the dedicated Raspberry Pi link below if you are not using it already.
https://ubuntu.com/download/raspberry-pi

Thank you @MartyG-RealSense
Yes, that is the ubuntu version I'm using.

Cool. It looks like new 2.45.0 SDK  is here. I will test it out next week.

Hi @bobluoluo   Have you had the opportunity to test SDK 2.45.0 yet please?  Thanks!

Hi @bobluoluo  Do you require further assistance with this case, please?  Thanks!

Hi, I was pretty occupied last week. I will try testing the new release as soon as possible. Thanks

No problem @bobluoluo - thanks very much for the update.

Hi @bobluoluo  Do you have an update about this case that you can provide, please?  Thanks!

Case closed due to no further comments received.


refactor(agent): JWT parsing & custom validation

Thanks to @trkohler for the initial half of this issue, your efforts were useful and appreciated. :+1:
Done in the PR:

JWT parsing and validation with tests
merged to the new JWT library

@robot9706 good catch, the issue was completely valid, our crux component used instead of the standard second based timestamp, what is also in the spec.
Corrected the related parts. Please check again @polaroi8d @robot9706


Prisma Schema Type inside a Type not generating a right Payload

Bug description
Hi,
My file.dto:
import type { SiteInfosPayload } from '.prisma/client'

class ClientDto {
    @ApiProperty()
    role: ClientRole

    @ApiProperty()
    @ValidateIf(o => o.client.role === ClientRoleValue.MANAGER || o.client.role === ClientRoleValue.USER)
    @IsNotEmpty({ message: `'sites' doit être rempli quand le userType est ${ClientRoleValue.MANAGER} ou ${ClientRoleValue.USER}` })
    sites: SiteInfosPayload[]

    @ApiProperty()
    group: GroupInfos
}

export class CreateUserDto {
    [...]
    @ApiProperty()
    userType: UserType;

    @ApiProperty()
    @ValidateIf(o => o.userType === UserTypeValue.CLIENT)
    @IsNotEmpty({ message: `'client' doit être rempli quand le userType est ${UserTypeValue.CLIENT}` })
    client: ClientDto;
}

And here is the problem inside my service:
async create(createUserDto: CreateUserDto) {         return this.prisma.user.create({ data: createUserDto });     }
The error:
  error TS2322: Type 'CreateUserDto' is not assignable to type '(Without<UserCreateInput, UserUncheckedCreateInput> & UserUncheckedCreateInput) | (Without<...> & UserCreateInput)'.
  Type 'CreateUserDto' is not assignable to type 'Without<UserUncheckedCreateInput, UserCreateInput> & UserCreateInput'.
    Type 'CreateUserDto' is not assignable to type 'UserCreateInput'.
      Types of property 'client' are incompatible.
        Type 'ClientDto' is not assignable to type '(Without<ClientNullableCreateEnvelopeInput, ClientCreateInput> & ClientCreateInput) | (Without<...> & ClientNullableCreateEnvelopeInput)'.
          Type 'ClientDto' is not assignable to type 'Without<ClientNullableCreateEnvelopeInput, ClientCreateInput> & ClientCreateInput'.
            Type 'ClientDto' is not assignable to type 'ClientCreateInput'.
              Types of property 'sites' are incompatible.
                Type 'SiteInfosPayload[]' is not assignable to type 'Enumerable<SiteInfosCreateInput>'.
                  Type 'SiteInfosPayload[]' is not assignable to type 'SiteInfosCreateInput[]'.
                    Type 'SiteInfosPayload' is missing the following properties from type 'SiteInfosCreateInput': id, name, siret

22         return this.prisma.user.create({ data: createUserDto });
                                            ~~~~

  node_modules/.prisma/client/index.d.ts:3026:5
    3026     data: XOR<UserCreateInput, UserUncheckedCreateInput>
             ~~~~
    The expected type comes from property 'data' which is declared here on type '{ select?: UserSelect<DefaultArgs>; include?: UserInclude<DefaultArgs>; data: (Without<UserCreateInput, UserUncheckedCreateInput> & UserUncheckedCreateInput) | (Without<...> & UserCreateInput); }'

If I'm understanding: the "SiteInfosPayload" is generated by Prisma but it miss my fields 'id', 'name', 'siret' but as you can see it's well declared on the Schema ...
How to reproduce
During run time, typescript throw me the previous error .
Expected behavior
Obviously I want my DTO working with my Schema.
Prisma information
Here is my Schema :
model User {
  id         String      @id @default(auto()) @map("_id") @db.ObjectId
 [...]
  userType   UserType
  client     Client?

  @@map("staffUser")
}

type Client {
  [...]
  sites SiteInfos[]
}

type SiteInfos {
  id    String
  name  String
  siret String
}

Environment & setup
NestJs / TypeScript
Database: MongoDb
nodeJs: v16.20.0
Prisma Version
prisma                  : 4.16.0
@prisma/client          : 4.16.0

FYI, if I remove sites SiteInfos[]  from my schema it's working well again.
But I want it here, and I want to put some decorator on this field on my file.DTO.

I'm just starting this project. I can't tell you, would you want me to try to downgrade ?

You could try that yes, I am trying to narrow down if it is related to a recently fixed issue https://github.com/prisma/prisma/issues/19880

Could I also please ask you to give it a try with<EMAIL_ADDRESS>which contains the fix?

Yes I don't have the error either on this version

Great! Thanks for the help. We will be publishing a patch soon. Let me know if everything worked with your project.

class ClientDto {
    @ApiProperty()
    role: ClientRole

    @ApiProperty()
    @ValidateIf(o => o.client.role === ClientRoleValue.MANAGER || o.client.role === ClientRoleValue.USER)
    @ArrayNotEmpty({ message: `'sites' doit être rempli quand le userType est ${ClientRoleValue.MANAGER} ou ${ClientRoleValue.USER}` })
    sites: SiteInfos[]

    @ApiProperty()
    group: GroupInfos
}

Thanks to your fix, I'm trying to add this to my DTO. But It's not seem to work. I don't know if you could see the problem.
It's more a NestJs / Typescript problem maybe.

Not sure what the error is. If this is also not working on 4.15.0, would you mind creating a conversation for your questions, with more context on the failures. Then I can close this issue and we help you from there?

I'm also experiencing the same issue, only started happening after updating to 4.16.0

Thanks for the confirmations, I'll mark this issue as fixed.

4.16.1 patch release just happened.
Let us know if this version works as expected 🙌🏼

This is still broken

4.16.1 patch release just happened.
Let us know if this version works as expected 🙌🏼

This is still broken in 14.16.1

@mayerlench Could you open a new bug issue and provide more information?
We will need to reproduce your issue to fix it.


scikit-fmm prevents package installation

Failed to build scikit-fmm
ERROR: Could not build wheels for scikit-fmm, which is required to install pyproject.toml-based projects
Seems to be related to scikit-fmm.

scikit-fmm is imported with porespy marching_map, which is used to compute the time of flight feature. We should possibly look into removing the porespy dependency.


Profiler Window Doesn't Close, Prevents Recompilation

What steps will reproduce the problem?
1. Compile and run a project
2. Close it (the window)
3. MicroProfiler remains open, unable to close MicroProfiler window

What is the expected output? What do you see instead?
The MP window shold be able to close?

What version of the product are you using? On what operating system?
VS 2010, Windows 7, 1.0.500.

Please provide any additional information below.
Btw the register DLL call didnt't work, had to copy it to Windows system folder.

Original issue reported on code.google.com by<EMAIL_ADDRESS>on 3 Jun 2012 at 10:59

As of 1.1.591.0 profiler's window can be closed at any time.


Looking for GameUserSettings.ini at wrong path and "ARK server exec not found"

I am trying to set up arkmanager in Ubuntu. I have my Ark server installed into the directory "/home/ark/server/" I edited the config file at /etc/arkmanager/arkmanager.cfg to reflect this. I set "arkserverroot" to "/home/ark/server".
After saving the config, and running arkmanager status I get:
ark@arkserver:~$ arkmanager status
Running command 'status' for instance 'main'
[  WARN  ]      Your ARK server exec could not be found.
 Server running:   No
/usr/local/bin/arkmanager: line 369: /home/ark/ARK/ShooterGame/Saved/Config/LinuxServer/GameUserSettings.ini: No such file or directory
/usr/local/bin/arkmanager: line 369: /home/ark/ARK/ShooterGame/Saved/Config/LinuxServer/GameUserSettings.ini: No such file or directory
 Server listening:   No
 Server build ID:

Unless I am misunderstanding something, it is looking for the ark server in "/home/ark/ARK" which is not what I set.
  If so, then delete them.

I installed the Ark server and SteamCmd under the user "ark" on a fresh Ubuntu Server 18.04.
I ran this command to install arkmanager:
curl -sL http://git.io/vtf5N | sudo bash -s ark
I wasn't asked to do a user install and no config files exist in those directories. I know it is reading from the config file at /etc/arkmanager.cfg because I made a change to correct the SteamCmd exec filename. I had to change "steamcmd.sh" to "steamcmd" and by doing so, it removed an error.

Please check /etc/arkmanager/instances/main.cfg

Please check /etc/arkmanager/instances/main.cfg

Looks like this may have solved it. Thank you! I was not aware of this file and I don't understand how /etc/arkmanager/instances/main.cfg interacts with /etc/arkmanager/arkmanager.cfg. I see similar options in both files for "Ark server flags" and Ark server options". I would assume one of these files overrides the other? I am trying to use my existing game settings in /home/ark/server/ShooterGame/Config/GameUserSettings.ini.

Global settings go into arkmanager.cfg, while instance settings go into the instance config (e.g. main.cfg is for the main instance).


bootstrap_breadcrumb

What is the change?
I have written the documentation for bootstrap breadcrumb.
Related issue?
closes: #67
Checklist:
Before you create this PR, confirm all the requirements listed below by checking the checkboxes [x]:

[x] I have followed the Code of Conduct.
[x] I have  checked there aren't other open Pull Requests for the same update/change.
[x] I have you tested the code before submission.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] My changes generates no new warnings.
[ ] I'm Hacktoberfest'21 contributor.

Screenshots (if any)

@girija0707 I have created the pr for #67 can u please check it.

Good work 👍 @Koustavjr

@Koustavjr hey just take a look this is how you should put the issue no just #number nothing else

Oh I got it now🙂

Good work 👍 @Koustavjr but next time try to write more on your own and elaborate a little more on the topic

OK sure....😀


Error setup Kafka provider

During the installation of the kafka provider, docker-compose tries to pull the openwhisk/controller:"latest" image instead of the "nightly" image, giving an error:
Pulling controller (openwhisk/controller:latest)...
ERROR: manifest for openwhisk/controller:latest not found

This is because during the "setup-providers" phase of the Makefile, the image prefix and image tag are missing from the "providers.env" file.
Adding the following lines to the Makefile "setup-providers" phase fixes the problem.
printf "DOCKER_IMAGE_PREFIX"="openwhisk\n" >> $(TMP_HOME)/tmp/openwhisk/providers.env

printf "DOCKER_IMAGE_TAG"="nightly\n" >> $(TMP_HOME)/tmp/openwhisk/providers.env

@phlucas96 are you on master/head?

Thanks @phlucas96 I see the delta. Do you want to submit a PR?

Ok, I will. Thanks.

closed by #272


feat: Helm Manifet Compare

🔖 Feature description
Helm apps > Configure > Manifest output (Ability to compare with Manifest output of previous deployments)
🎤 Pitch
Ability to compare helm manifest
✌ Solution
Add a button to comapre manifests same as that of compare values.
🔄 Alternative
No response
👀 Have you spent some time to check if this issue has been raised before?
I checked and didn't find similar issue
🏢 Have you read the Code of Conduct?
I have read the Code of Conduct

Fixed with attached PR


Language switcher

I'm at the stage of completion of the translation, you can add the language switcher?

Hi! Thanks for your willingness to contribute.
We currently do not plan to offer translated versions of the documentation on flarum.org. That's because it gets complicated to

keep them up-to-date,
control and maintain their quality (because we can not possibly speak all those languages)

I hope you understand that, for now, we will leave this up to community-hosted sites.
@tobscure Please close this if you agree.

Yes, I knew about it. but still translate to the end, it may be required in the future.
Thank you noticed :)
Closed.

"@tobscure Please close this if you agree."
Sorry for the carelessness:)

Which language are we talking about, by the way?

I speak in Russian, and I am not a professional translator.


[Bug] MSI + App Services needs to perform extra check

Library version used
1.14.4-beta
Java version
8
Scenario
ManagedIdentityClient - managed identity
Is this a new or an existing app?
This is a new app or experiment
Issue description and reproduction steps
Have a look at Azure Identity code here: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/identity/azure-identity/src/main/java/com/azure/identity/implementation/IdentityClient.java#L1032
Relevant code snippets
No response
Expected behavior
No response
Identity provider
Microsoft Entra ID (Work and School accounts and Personal Microsoft accounts)
Regression
No response
Solution and workarounds
Note that any solution should still offer HttpClient extensiblity. So we can't just use MSAL's internal HTTP Client stack. Needs a bit of design first.
See equivalent .NET issue: https://github.com/AzureAD/microsoft-authentication-library-for-dotnet/issues/4462

We have agreed with Azure SDK to not extend the HttpClient factory for this. It is acceptable for MSAL use it's own HttpClient that doesn't go through the extensibility pipeline.
Retry policies etc. are still required.

Fixed in https://github.com/AzureAD/microsoft-authentication-library-for-java/pull/791 and released in msal4j 1.15.0


help: Rename back button text.

Renamed back button text to Zulip instead of Home.

Testing Plan:
GIFs or Screenshots:
Before:

After:

This looks great! Two thoughts:

It would be great to un-indent the article names. That will prevent some line-wrapping as well.
I think 's can be less bold. Maybe add some transparency?

Merged, thanks @akashnimare!  @synicalsyntax FYI, I think you'll like these tweaks.
I agree both of those follow-ups could be nice; please submit them in a new PR.


Should we bundle HA provided dependencies?

Currently we include leaflet and lit in our generated map-card.js, I wonder if it would be better to leave them out given HomeAssistant provides these anyway.
WDYT @thybag?

It'd probably be nice if we could find a way to do it, as Leaflets quite a bit of the code weight.
HA's code splitting tends to make it hard to request specific resources when their internal component isn't on the page so unsure if will be possible. So far as i could see there were very few custom map cards (Aside from your own) out there, otherwise would be tempted to have a peak at how they'd done it.


Missing tar.bz2 files

Hi Ron,
Just having a little play about with some of the CPs you have here but i cant seem to find the .tar.bz2 files in the actual zip files as per the layout description.
Not sure if they have been lost somewhere but thought i would highlight it to you. Any ideas where the files are now?
Thanks

IGEL Community does not have the rights to redistribute 3rd party software.  What we provide here is everything but the 3rd party software. Most of the items here have a build script that can be run to automate the building of the tar.bz2 file.
For example, Microsoft Teams, https://github.com/IGEL-Community/IGEL-Custom-Partitions/tree/master/CP_Source/Unified_Communications/Microsoft_Teams can be built in less than a minute following this build script on Ubuntu 18.04 system https://raw.githubusercontent.com/IGEL-Community/IGEL-Custom-Partitions/master/CP_Source/Unified_Communications/Microsoft_Teams/build-teams-cp.sh
See the video here:  https://www.linkedin.com/posts/jedayres_igelos-somethingcool-microsoftteams-activity-6740750840077926400-YjSK/

Absolutely understood Ron, appreciate the quick response, i will give it a go myself using the scripts.
Thank you


add baseDataDir which could be shared between services

I'd like to override default base dataDir for all services, without having to re-state service name in each override
I'd like to do this:
let
  baseDataDir = "./.data";
in {
  services.redis."redis-1" = {
    enable = true;
    inherit baseDataDir;
  };
  services.postgres."pg-1" = {
    enable = true;
    inherit baseDataDir;
  };
}

instead of:
let
  baseDataDir = "./.data";
in {
  services.redis."redis-1" = {
    enable = true;
    dataDir = "${baseDataDir}/redis-1";
  };
  services.postgres."pg-1" = {
    enable = true;
    dataDir = "${baseDataDir}/pg-1";
  };
}

while preserving the original data dir structure
see possible implementation here

+1 for this issue. If you can raise the PR, more than happy to merge it.

here it is: https://github.com/juspay/services-flake/pull/24
thanks!

Another way to solve this, @adrian-gierakowski :
let
  imports = [ ({ name, ... }: {
    dataDir = "./.data/${name}";
  } )];
in {
  services.redis."redis-1" = {
    inherit imports
    enable = true;
  };
  services.postgres."pg-1" = {
    inherit imports;
    enable = true;
  };
}

Happy to close https://github.com/juspay/services-flake/issues/23#issuecomment-1643835355


Gamma should be allowed to be fixed

the user shouldn't be required to fit gamma

fixed in better grl


Migrating from webpack 1 to webpack 2

Since, Webpack 2 release is stable and the documentation is rich, it makes sense to migrate to webpack 2. @IamMohaiminul

Working on next release. Hope to get this on that release.

Webpack 2 is implemented in latest commit.


Support compass' inline-image

http://compass-style.org/reference/compass/helpers/inline-data/

Now need a setting to tune this (for instance, don't inline in debug mode?), and more proper sourcemap handling (sourcemap composition in progress).

Shipped in v0.1.3: can disable with transformer setting inlineImages: false.
Sourcemap-related issues (combining sourcemaps with other transforms) are tracked in issue #9.


Menu: fix click el-submenu trigger childMenu pop again bug

Please make sure these boxes are checked before submitting your PR, thank you!

[x] Make sure you follow Element's contributing guide (中文 | English | Español | Français).
[x] Make sure you are merging your commits to dev branch.
[x] Add some descriptions and refer relative issues for you PR.

The showTimeout duration of popupMenu does not need to be the same as inlineMenu.
If it
set too long, it will cause the submenu pop again bug when you quickly click on the menu.

To fix https://github.com/ElemeFE/element/issues/12512

@PanJiaChen 你是怎么测得？拉了我的分支？还是自己改了代码？

@PanJiaChen @element-bot 部署了一個 preview 版本

https://deploy-preview-14443--element.netlify.com


[FAQ] Difference between "romp" and "bev"

There should be at least in the quick start guide an short summary of the differences betwen bev and romp according to the described commands.

Sorry for the late reply. I got catched by a deadline, tired face...
I add the describtion about the difference now.

@Arthur151 No problem, thank you 🙂


Can gen-csv update the WATCH_NAMESPACE downward api destination?

Type of question
Are you asking about community best practices, how to implement a specific feature, or about general context and help around the operator-sdk?
operator-sdk
Question
What did you do?
The gen-csv task seems to copy the deploy/operator.yaml whole into the generated CSV, but the metadata.namespace value for WATCH_NAMESPACE may not be correct for the CSV.
What did you expect to see?
I expected the value to be changed to metadata.annotations['olm.targetNamespaces'] per the OLM docs.
What did you see instead? Under which circumstances?
It retained its metadata.namespace value, which is only correct when operator.yaml is deployed outside of OLM
Environment

operator-sdk version:
0.7.0

Kubernetes version information:

Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.3", GitCommit:"721bfa751924da8d1680787490c54b9179b1fed0", GitTreeState:"clean", BuildDate:"2019-02-01T20:08:12Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.0", GitCommit:"0ed33881dc4355495f623c6f22e7dd0b7632b7c0", GitTreeState:"clean", BuildDate:"2018-09-27T16:55:41Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"}

Kubernetes cluster kind: minikube

cc @estroz

@jcrossley3 @LiliC PR open: #1396


does the example bot work with locahost?

i am using your example, and have updated it with my tokens etc to get started.
whenever i access the the page via localhost or its local ip, i get Invalid oAuth2 redirect_uri
does this uri need to be publicly accessible for this example to work?

Can you verify and check if you're setting redirect URL in your Discord application developers dashboard something like this:

hello,  yes sorry i figured this out last night after posting and forgot to close this.  thank you.


Store.Chat.find error

Hi!
When i enter on a new group, and try to send message, i use Store.Chat.find(id), is returning a error:
ModelCreateError: Unknown
I tried to use this on async function which i saw on another issue:
const chatId = new Store.UserConstructor(jid_grupo, {intentionallyUsePrivateConstructor: true});
let chat = await Store.Chat.find(chatId).then((r)=>{return r;});
chat.sendMessage(message);
Returning the same error, i can't send a message to my new group.
I tried to create a new model, to add to Store.Chat.models, but not working too.

same problem here


Fixed a couple of typos

I've fixed a couple of typos in the base template for the admin.

yep. Thanks!


GROOVY-9759: Regression for accessing private static constants in clo…

…sures from Map.
Seems we have subsequent code not expecting this enriched information.

Wasn't this test case already fixed by GROOVY-9699?  I do not see any STC error for Groovy 3.0.6.SNAPSHOT.

You may be right - I'll look into it. Perhaps I was on an old branch at the time.

You are correct. I must have been on an older branch when I was running my tests. I was sure I had test on master and was basing my assumptions on that, but it works fine testing just now.


Add associate-eip to the documentation

There is currently no mention of this option in the README, I only found out about it from a StackOverflow post that mentioned it.

I would like to know more about the possible values for the --server-connect-attribute option as well. Ideally, we could add a "for example: foo to that description that shows the most common usage.
The option itself is on the page: http://docs.chef.io/plugin_knife_ec2.html#id2
It's at the very top of that list because the options are grouped by short name, which in this case is -a.

Possible values for --server-connect-attribute are 'dns_name', 'public_ip_address' and 'private_ip_address'

May I suggest to put these values in '--help'?
$ knife ec2 server create --help | grep server-connect-attribute -C 2
    -G, --groups X,Y,Z               The security groups for this server; not allowed when using VPC
    -a ATTRIBUTE,                    The EC2 server attribute to use for SSH connection. Use this attr for creating VPC instances along with --associate-eip
        --server-connect-attribute
        --spot-price PRICE           The maximum hourly USD price for the instance
        --spot-request-type TYPE     The Spot Instance request type. Possible values are 'one-time' and 'persistent', default value is 'one-time'

Using '--server-connect-attribute private_ip_address' worked for me to access one VPC instance where the DNS does not works.

@tiago-cruz-movile
Please check your VPC DNS settings, i.e DNS resolution:  and DNS hostnames:  should be set to yes
Please share your ec2 server create ... command. details with stdout logs
Thanks!


Possible to add Field meta Data ?

Hi - is it possible to extend the code generation to include some meta data so that generated model can be used to build data dictionary or run time behaviour ??
e.g.
@FeildMetaData(encrypted=true, blabla=[])
val someField : String
Thanks.

Hi Munish, have you seen the annotations section of the Configuration Guide?
It seems to me that this would cover exactly what you're asking for here.  Annotations can be added to the class or to each field, and the content can be generated from a Mustache template with access to a number of details about the class or field.
Regards, Peter

Thank you I missed it earlier.
This is what I was exactly looking for 🙏


TilesFX and JavaFXPorts for Android 5.1

Thank you for your excellent library, but I want build application for Android used JavaFXPorts and javafxmobile-plugin of Gradle. After compilation simple application with tiles, I placed it to my Android 5.1 and started, but screen black only without any other elements.
Have you tried running Tiles on Android platform?

As far as I know the current version of JavaFX ports doesn’t support Java 8 streams which I use in TilesFX. Without looking into it I’m pretty sure that this is the problem. You could try to check out the code and replace all streams by loops, compile it and give it a try. I think it won’t be a big deal to do so.
Cheers,
Gerrit

Gerrit, so sorry. I found my bug in my source code. It was not associated with TilesFX. TilesFX is ready for porting to Android 5.1 by JavaFX ports.

Regards,
Andrey

Sweet...looks good ;)


Add Axway copyright to bottom of readme

In LICENSE:
Copyright 2016 The AMP Authors => Copyright (c) 2016, Axway, Inc.

Done.


Travis CI failure - tests fail to start

Seems to be a recent failure.
https://api.travis-ci.org/v3/job/446471076/log.txt
[22:05:12] Starting 'test'...
25 10 2018 22:05:12.939:INFO [framework.detect-browsers]: Detecting browsers is disabled. The browsers of the browsers array are used.
25 10 2018 22:05:15.023:INFO [karma]: Karma v2.0.5 server started at http://<IP_ADDRESS>:9876/
25 10 2018 22:05:15.024:INFO [launcher]: Launching browser ChromeCI with unlimited concurrency
25 10 2018 22:05:15.029:INFO [launcher]: Starting browser ChromeHeadless
25 10 2018 22:05:16.485:INFO [HeadlessChrome 0.0.0 (Linux 0.0.0)]: Connected on socket FYUQ78QJTOVvdCgIAAAA with id 46055948
HeadlessChrome 0.0.0 (Linux 0.0.0) ERROR LOG: 'DEPRECATION:', 'Setting specFilter directly on Env is deprecated, please use the specFilter option in `configure`'
25 10 2018 22:05:46.615:WARN [HeadlessChrome 0.0.0 (Linux 0.0.0)]: Disconnected (1 times), because no message in 30000 ms.
HeadlessChrome 0.0.0 (Linux 0.0.0) ERROR
  Disconnected, because no message in 30000 ms.
HeadlessChrome 0.0.0 (Linux 0.0.0): Executed 0 of 0 DISCONNECTED (30.131 secs / 0 secs)

I've also seen it fail for HeadlessFirefox.

I'm seeing this very consistently in https://github.com/AnalyticalGraphicsInc/cesium/pull/7168 as well.

Ditto.  I started to see this yesterday

I just saw this same problem in one of our other projects too.
@mramato Could this have to do with Jasmine releasing a new version 21 hours ago? https://github.com/jasmine/jasmine/releases

Yep.  It's likely https://github.com/jasmine/jasmine/issues/1617


The named query for the control breaks in hub site

Describe the bug
A clear and concise description of what the bug is.
Some of the controls are not rendering,
https://hub.steampipe.io/mods/turbot/kubernetes_compliance/queries/cronjob_default_seccomp_profile_enabled
Steampipe version (steampipe -v)
Example: v0.3.0
Plugin version (steampipe plugin list)
Example: v0.5.0
To reproduce
Steps to reproduce the behavior (please include relevant code and/or commands).
Expected behavior
A clear and concise description of what you expected to happen.
Additional context
Add any other context about the problem here.

https://github.com/turbot/steampipe/issues/3405

The issue has now been fixed 👍.


I just had a second son last weekend so been pretty intense.
I will try to look at this as soon as I possibly can.
Thanks again!

@artisologic CONGRATS!!!!

Hi @moimikey for info, I have published version (1.0.3) which brings back full compatibility for the https://github.com/stars page.


Multiple nesting breaks View

Long story short: I'm nesting the Ring Progress Bar in a RelativeLayout that is nested in a LinearLayout that is nested in a ScrollView.  When I go from just a LinearLayout on the top level to having everything within a ScrollView, the ring will disappear.

I'm facing the same issue, did you ever find the solution?

Nvm found a workarround. For those that are having the same problem: hardcoding the android:layout_height to a specific dp value should solve the problem


Remove Float comparison ambiguities from generate_coords

Hi @JordiManyer,
Thanks for your work.
I have completed a first round of review. Please see my comments.
Leaving aside my concern on why do we actually need geometric information (i.e., the coordinates of the corners, whatever they are defined mathematically), my main concern while going over the proposed strategy in this PR was (but no longer is) that we are still comparing numerical values to identify vertices. But I think it is fine, because we only have two different scenarios:

The current vertex is located in exactly the same coordinate as an existing corner.
The current vertex is not located in the same coordinate as an existing corner (in such a case, we have potentially a new vertex).

I think that, in order to be even safer, one may use equality instead of norm(coord-model_coords[vertex]) > eps(T). This is because the coordinates of a corner are assigned to be bit-wise the same (i.e., a copy) to the ones of one of the vertices clashing in the same corner. The same reasoning applies to pos = findfirst(x -> norm(x-coord) < eps(T), model_coords[n_corners+1:n_vertices]).
Note that norm(coord-model_coords[vertex]) > eps(T) might be false for a vertex-corner pair which are too close to each other, but not exactly in the same position.
Let me know if you agree or not here, or I might be missing something.
Finally, on another note, I still wonder myself if I could get some symbolic information out of p4est that allows us to separate vertices from corners, but I guess this requires much more time than the one I have now.


Expose connection name

Tracing requests with dbus-monitor is much easier if you can filter by connection name.

Done in b0afae0f5d97e38eda79aa596634fe28906a876c


Add XML ingest performance tests

Add XML version of data in tests/mocks/perf-data.json.txt
Add performance tests for various ways of reading an XML sitemap
Minor fix to wait for the write streams to emit the end/finished event - this can be slightly after the read stream has emitted finished, but in practice it doesn't change the perf timings much when writing to /dev/null
Mark all of the test arrow funcs as async since they return promises and are awaited in batch()
Pass the testname to printPerf instead of repeating the case string as there were cases where this diverged (promise, stream-2) and looked like a copy pasta mistake

Note: This seems to demonstrate that there is not a clean way to hydrate a SitemapStream from an existing stream while leaving it open for further writes outside of a pipeline without having to hold all the SitemapItem[]'s in an array in memory... unless I'm missing something.

I really appreciate the contribution to the perf tests.


Working Group Calendar Links

Just noticed that the current calendar links go to here:

Is this meant to a placeholder or should it match up with the links in the other repos ? Happy to open a PR if so.

Thanks @obowersa for reporting this - the links point to the specific events in the calendar - if you sign it should show the event details and offer to save them in your calendar. I see that it may be not ideal if you don't have or don't want to use a google account, but the overall CDF public calendar is a google calendar, so I don't think there is a good way around.
One alternative is to link the CDF Public calendar<EMAIL_ADDRESS>which can be browsed online without any sign-in required, and also to specify the time and date of the meetings as text on the page - would that help?


Add temporary pixels for DMA choice screens

Task/Issue URL:
Description
Steps to test this PR
Feature 1

[ ]
[ ]

UI changes

Before
After

!(Upload before screenshot)
(Upload after screenshot)

#4543  👈
develop

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @malmstein and the rest of your teammates on  Graphite


"code":"timeout" error message

Hi,
since the latest update I have multiples times a day these error messages in my log:
[24.2.2021, 11:00:52] [Wall-e] ERR getState | {"code":"timeout"} [24.2.2021, 11:01:22] [Wall-e] ERR getState | {"code":"timeout"}
The roborock is online in the xiaomi App and I can use it on the home app.

Same problem

Hmm, it looks like a network issue to me. However, I'm working on improving the communication layer #334
Hopefully, that'll help in the future.

I don't know if this is the same but I have still red coloured error :-(
[3/4/2021, 19:19:08] [Robot Piętro] ERR getState | {"code":"timeout","device":null}
[3/4/2021, 19:19:38] [Robot Piętro] ERR connect | miio.device, next try in 2 minutes | Error: Could not connect to device, handshake timeout

Hi and thanks for the great work !
I also started getting the following similar error for the last couple of days
[Roborock] ERR connect | miio.device, next try in 2 minutes | Error: Could not connect to device, handshake timeout
However the device is connected and properly operating with Mi Home App.

While I get the fix done, does restarting homebridge fix the issues to y'all? That's usually the case for most other folks.

Yes. Restarting helps. But I need to repeat restart every day. Now I'm looking for solutions to do this automatically.

If restarting helps, I think my changes in #334 will fix this (hopefully). I need to fully test it and makes sure it works as expected before releasing it :)

Hello @afharo, many thanks for your time and efforts. This has just happened to me. While on a cleaning cycle, the timeout errors has started. (see partial logs below) Then I had restarted home bridge but nothing changed, than I have turned off-on via the power button on S5Max and home bridge timeouts stopped but still Homekit was not recognizing the Roborock. Then I have restarted home bridge and it is OK. Don't know where is the problem but Mihome was functioning properly.
[3/14/2021, 9:56:14 AM] [Xiaomi Mi Robot Vaccum 1st Generation] ERR getState | {"code":"timeout"}
[3/14/2021, 9:56:30 AM] [Xiaomi Mi Robot Vaccum 1st Generation] ACT setCleaningRoom | roborock.vacuum.s5e | Disable cleaning Room ID 18.
[3/14/2021, 9:56:30 AM] [Xiaomi Mi Robot Vaccum 1st Generation] ACT setCleaningRoom | roborock.vacuum.s5e | Start timeout to clean rooms
[3/14/2021, 9:56:30 AM] [Xiaomi Mi Robot Vaccum 1st Generation] ACT setCleaningRoom | roborock.vacuum.s5e | Disable cleaning Room ID 19.
[3/14/2021, 9:56:30 AM] [Xiaomi Mi Robot Vaccum 1st Generation] ACT setCleaningRoom | roborock.vacuum.s5e | Start timeout to clean rooms
[3/14/2021, 9:56:30 AM] [Xiaomi Mi Robot Vaccum 1st Generation] ACT setCleaningRoom | roborock.vacuum.s5e | Disable cleaning Room ID 16.
[3/14/2021, 9:56:30 AM] [Xiaomi Mi Robot Vaccum 1st Generation] ACT setCleaningRoom | roborock.vacuum.s5e | Start timeout to clean rooms
[3/14/2021, 9:56:30 AM] [Xiaomi Mi Robot Vaccum 1st Generation] ACT setCleaningRoom | roborock.vacuum.s5e | Disable cleaning Room ID 22.
[3/14/2021, 9:56:30 AM] [Xiaomi Mi Robot Vaccum 1st Generation] ACT setCleaningRoom | roborock.vacuum.s5e | Start timeout to clean rooms
[3/14/2021, 9:56:30 AM] [Xiaomi Mi Robot Vaccum 1st Generation] ACT setCleaningRoom | roborock.vacuum.s5e | Disable cleaning Room ID 21.
[3/14/2021, 9:56:30 AM] [Xiaomi Mi Robot Vaccum 1st Generation] ACT setCleaningRoom | roborock.vacuum.s5e | Start timeout to clean rooms
[3/14/2021, 9:56:30 AM] [Xiaomi Mi Robot Vaccum 1st Generation] ACT setCleaningRoom | roborock.vacuum.s5e | Disable cleaning Room ID 17.
[3/14/2021, 9:56:30 AM] [Xiaomi Mi Robot Vaccum 1st Generation] ACT setCleaningRoom | roborock.vacuum.s5e | Start timeout to clean rooms
[3/14/2021, 9:56:31 AM] [Xiaomi Mi Robot Vaccum 1st Generation] ACT setCleaningRoom | roborock.vacuum.s5e | Enable cleaning Room ID 23.
[3/14/2021, 9:56:31 AM] [Xiaomi Mi Robot Vaccum 1st Generation] ACT setCleaningRoom | roborock.vacuum.s5e | Start timeout to clean rooms
[3/14/2021, 9:56:31 AM] [Xiaomi Mi Robot Vaccum 1st Generation] ACT setCleaningRoom | roborock.vacuum.s5e | Enable cleaning Room ID 20.
[3/14/2021, 9:56:31 AM] [Xiaomi Mi Robot Vaccum 1st Generation] ACT setCleaningRoom | roborock.vacuum.s5e | Start timeout to clean rooms
[3/14/2021, 9:56:31 AM] [Xiaomi Mi Robot Vaccum 1st Generation] ACT setCleaningRoom | roborock.vacuum.s5e | Enable cleaning Room ID 24.
[3/14/2021, 9:56:31 AM] [Xiaomi Mi Robot Vaccum 1st Generation] ACT setCleaningRoom | roborock.vacuum.s5e | Start timeout to clean rooms
[3/14/2021, 9:56:31 AM] [Xiaomi Mi Robot Vaccum 1st Generation] ACT setCleaningRoom | roborock.vacuum.s5e | Enable cleaning Room ID 25.
[3/14/2021, 9:56:31 AM] [Xiaomi Mi Robot Vaccum 1st Generation] ACT setCleaningRoom | roborock.vacuum.s5e | Start timeout to clean rooms
[3/14/2021, 9:56:34 AM] [Xiaomi Mi Robot Vaccum 1st Generation] ACT setCleaning | roborock.vacuum.s5e | Set cleaning to true}
[3/14/2021, 9:56:44 AM] [Xiaomi Mi Robot Vaccum 1st Generation] ERR getState | {"code":"timeout"}
[3/14/2021, 9:56:46 AM] [Xiaomi Mi Robot Vaccum 1st Generation] ERR setCleaning | roborock.vacuum.s5e | Failed to set cleaning to true Error: Call to device timed out
at Timeout.retry [as _onTimeout] (/usr/lib/node_modules/homebridge-xiaomi-roborock-vacuum/miio/lib/network.js:519:23)
at listOnTimeout (internal/timers.js:554:17)
at processTimers (internal/timers.js:497:7) {
code: 'timeout'
}
(node:2534) UnhandledPromiseRejectionWarning: Error: Call to device timed out
at Timeout.retry [as _onTimeout] (/usr/lib/node_modules/homebridge-xiaomi-roborock-vacuum/miio/lib/network.js:519:23)
at listOnTimeout (internal/timers.js:554:17)
at processTimers (internal/timers.js:497:7)
(Use node --trace-warnings ... to show where the warning was created)
(node:2534) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:2534) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
[3/14/2021, 9:57:05 AM] [Xiaomi Mi Robot Vaccum 1st Generation] ERR setCleaning | roborock.vacuum.s5e | Failed to set cleaning to true Error: Call to device timed out
at Timeout.retry [as _onTimeout] (/usr/lib/node_modules/homebridge-xiaomi-roborock-vacuum/miio/lib/network.js:519:23)
at listOnTimeout (internal/timers.js:554:17)
at processTimers (internal/timers.js:497:7) {
code: 'timeout'
}
(node:2534) UnhandledPromiseRejectionWarning: Error: Call to device timed out
at Timeout.retry [as _onTimeout] (/usr/lib/node_modules/homebridge-xiaomi-roborock-vacuum/miio/lib/network.js:519:23)
at listOnTimeout (internal/timers.js:554:17)
at processTimers (internal/timers.js:497:7)
(node:2534) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 2)
[3/14/2021, 9:57:14 AM] [Xiaomi Mi Robot Vaccum 1st Generation] ERR getState | {"code":"timeout"}
[3/14/2021, 9:57:44 AM] [Xiaomi Mi Robot Vaccum 1st Generation] ERR getState | {"code":"timeout"}
[3/14/2021, 9:57:56 AM] [Homebridge UI] Homebridge restart request received

I've published https://github.com/homebridge-xiaomi-roborock-vacuum/homebridge-xiaomi-roborock-vacuum/releases/tag/v0.18.0. Hopefully it'll fix these issues.


fix: logic of Client.close

#8 was incomplete, update the logic. ref: https://github.com/FeatureProbe/server-sdk-java/pull/19

Codecov Report

Merging #12 (df4fe14) into main (c516a8e) will decrease coverage by 0.06%.
The diff coverage is 10.00%.

@@            Coverage Diff             @@
##             main      #12      +/-   ##
==========================================
- Coverage   85.30%   85.23%   -0.07%
==========================================
  Files          34       34
  Lines        1259     1260       +1
==========================================
  Hits         1074     1074
- Misses        185      186       +1

Impacted Files
Coverage Δ

featureprobe/client.py
84.31% <0.00%> (ø)

featureprobe/memory_data_repository.py
81.81% <0.00%> (ø)

featureprobe/default_event_processor.py
73.80% <12.50%> (-0.60%)
:arrow_down:


Missing exception/handler for '410 Gone' response

According to OSB spec, 410 Gone response could be returned when calling:

ServiceInstanceService.deleteServiceInstance()
ServiceInstanceService.getLastOperation()
ServiceInstanceBindingService.deleteServiceInstanceBinding()

However there is no code to produce this error in a core package

Hi thanks for reporting! I believe we have these cases covered already. If you are not seeing the expected behavior, then please let us know.

delete service instance
get last operation
delete service binding

Yep, i see now. It's just not consistent with the rest of code where we have exceptions and corresponding handlers. What do you think?

And actual return code for delete service instance is 422 instead of 410 because of exception handler executed first and catch block in ServiceController never reached.

@artkoshelev From the OSB API spec section on error handling:

Broker failures beyond the scope of the well-defined HTTP response codes listed above (like 410 Gone on Deprovisioning) MUST return an appropriate HTTP response code (chosen to accurately reflect the nature of the failure) and a body containing a valid JSON Object (not an array).

All of the @ExceptionHandler methods build a response with a description element in the body of the response, containing the details of the error as suggested in the spec.
The 410 Gone responses aren't technically errors (i.e. they aren't unexpected), and are returned with an empty response body. This is why they aren't handled in @ExceptionHandler methods.

Ok, thanks for clarifications!


[23.11] difftastic: 0.53.1 -> 0.54.0

Signed-off-by: Ludovico Piero<EMAIL_ADDRESS>(cherry picked from commit 0668d341012923d2a2a129959619dba6d1b48059)
Description of changes

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

What's the reason for this backport?

What's the reason for this backport?

I saw that the latest PR for this package is being backported, so I thought I should backport this as well. Feel free to close this.


Database corruption on host restart (maybe other times)

What happened (or feature request):
We're regularly seeing "can't start database" situations. The log often has something like "InnoDB: The log sequence number 1600607 in ibdata file do not match the log sequence number<PHONE_NUMBER>5 in the ib_logfiles!" and implies a crash.
Perhaps as in https://stackoverflow.com/questions/19378565/log-sequence-number-in-ibdata-files-does-not-match
This may be happening on a host restart; the db container may not have a reasonable shutdown sequence, so we end up with a corrupted db. We could delete the logfiles on start (so they'd be created automatically). There are probably other options.
What you expected to happen:
How to reproduce this:
Version: Please include the output of ddev version, docker version and the project's .ddev/config.yaml.
Anything else do we need to know:
Related source links or issues:
Please use a complexity rating of 1-5 (5 is high) for a feature request. (High complexity implies more PR planning)

Studying https://docs.docker.com/compose/compose-file/#stop_grace_period I see a stop_grace_period that we might want to increase. The default stop_signal SIGTERM that is normally sent should work OK with the db container, and mysqld is pid 1, so that's correct.

I added     stop_grace_period: 1m30s to the docker-compose.yaml in the db service, but it did not affect the behavior. However, on a stopped/broken container I did a rm ~/.ddev/<project>/mysql/ib* and then a start and it did file.

Interesting paths also might include live restore

innodb_fast_shutdown ?

Adding this to .ddev/mysql/recovery.cnf seems to completely resolve this problem:
[mysqld]
innodb_force_recovery = 1

I think we should consider just adding that to the mariadb config. See forcing recovery; This would be questionable on a prod system, but that's not what we have here. The '1" value is the gentlest, and it works for me.

It almost sounds like Docker for Mac doesn't gently stop containers before going down. If it's just a hard power off to the VM, that's really not a great situation, and it's no surprise that InnoDB is complaining about it. I think this solution works as a stopgap, but we should also file an issue upstream to stop containers nicely before killing the VM. I would also say that this solution may not be 100% effective. For instance, if Docker for Mac stops while MySQL is in the middle of writing data to the disk, I think there would be a substantially worse problem that may not be easy to recover from.

Update: The PR in https://github.com/drud/mariadb-local/pull/52 doesn't work because you can't apparently create a new db with that setting. However, The approach in https://github.com/drud/ddev/issues/748#issuecomment-375029786 does work perfectly for an existing project with db created. I think now that we understand this we'll be able to get it sorted out.

This has happened enough times that we need to look for ways to alert people to the actual cause, and help them debug or solve it. The fact that we have no control over Docker's failure in this is no excuse, we need to look for better ways.

Just have the database write everything without caching? Pretty sure you can do that, it might have performance impact.
Explain to people what the failure to start the db container might mean?

Probably the next step on this is just to rigorously demonstrate the problem, see what platforms it happens on, then file an upstream issue.

We (potentially) can recreate this issue as per this convo in the TYPO3 #ddev Slack room.

Preparation

ddev v0.19.0
docker-ce 18.03.1-ce-mac65
Drupal 7.59
Drupal 8.5.4
TYPO3 8.7.16

Testing

Drupal 7.59

Fresh unzip of release.
ddev config --docroot . --projecttype drupal7 --projectname drupal7
ddev start
ddev exec drush si standard -y
Quit Docker from Menu Bar
Start Docker Application
ddev start

Received error: "Failed to start drupal7: db service health check timed out: labels map[com.ddev.site-name:drupal7 com.docker.compose.service:db] timed out without becoming healthy, status=exited"
ddev rm && ddev start did not fix and results in the same issue.
ddev rm --remove-data && ddev start fixes but obviously the db is done.

Going to see if I can find out when/where this regression was introduced and whether or not it's also present elsewhere.

To be clear, the issue doesn't appear to be stemming from just MariaDB but there's something definitely making it difficult for a database to come back up once Docker is stop/started.

The suggestion by Timek almost worked..
Tymoteusz Motylewski [2:12 AM]
@timschreiner a workaround we use in similar cases (this happen only on one machine out of tens using ddev) is to add a file
.ddev/mysql/recovery.cnf
with
innodb_force_recovery = 1```
then call `ddev restart`, and once all is running again, change it to `innodb_force_recovery = 0`

The containers come back up but the page loads seem to hang. I think I need to toss this over to @andrewfrench to take further. But we at least have an ability to recreate and a known workable state.

@rickmanelius I think you tagged the wrong person.
Should this file only be added, when the the database is corrupted? Because when I add it by default after setting up ddev, the database does not start either. Even when it's new and empty one. Without this file, the startup when well. Until I quit docker of course.

Steps to reproduce for TYPO3 8.7

composer create-project typo3/cms-base-distribution typo3-ddev ^8
cd typo3-ddev
ddev config --docroot public --projecttype typo3 --projectname typo3v8
ddev start
ddev exec /var/www/html/vendor/bin/typo3cms install:setup --no-interaction --database-user-name=db --database-user-password=db --database-host-name=db --database-port=3306  --database-name=db  --use-existing-database --admin-user-name=admin --admin-password=admin123! --site-setup-type=site
Quit docker or restart OS (without stopping the project)
Start the project again with ddev start. You should get an error.

I was able to reproduce and confirm the issue with @codemonkey1988's steps above (thanks!), and probing with the same versions used by @rickmanelius, here's what I found:

TYPO3 v8.7 w/ ddev v0.19: db service remains unhealthy
TYPO3 v8.7 w/ ddev v0.18: db service health check times out, db service is stopped
TYPO3 v8.7 w/ ddev v0.16: db service health check times out, db service is stopped
TYPO3 v8.7 w/ ddev v0.13: all services started normally

Seems to point to some kind of regression in either ddev itself or the container images between 0.13 and 0.16. Alternatively, the health check might be more actually reporting issues that were a blind spot in previous versions.  I'll keep digging.

Any way I can restore the data on the db image?

@Tuurlijk whew. Glad that at least got you to operational. BTW, @andrewfrench has narrowed down the answerspace further and we should be able to review/address early next week. Thank you for your patience and feedback throughout this process!

I have, and to write it down for posterity:

ddev v0.14.0 with mariadb-local:v0.7.1 (default combination): issue is not reproduced
ddev v0.15.0 with mariadb-local:v0.8.0 (default combination): issue is reproduced
ddev v0.14.0 with mariadb-local:v0.8.0: issue is reproduced
ddev v0.15.0 with mariadb-local:v0.7.1: issue is not reproduced

This leads me to believe that the issue was introduced in the mariadb-local container update from v0.7.1 to v0.8.0, which took place around 8 March 2018 in the mariadb-local repository (before containers were maintained in the ddev repo).

Just noting:

It is normal for a ddev site not to come up automatically after a reboot of the system or of docker. This is a result of docker-compose's restart policy being set to "no", and it has several good effects. Normally (without database corruption) a simple ddev start will bring everything up.
The workaround with innodb_force_recovery is a one-time thing: You put that in there, start the db, and get it going, then remove that from the mysql config. It also can be upped as far as '6' for more aggressive recovery.

@jeffsheltren Would you be willing to take a look at our MariaDB config and make suggestions about how we can survive the "server belly-up with no notice" problem, corrupting the db? Docker in many cases on windows and mac just kills the container with no notice (when Docker is going down, during host shutdown, or Docker upgrade). And the user db does not fare well. Of course, Docker can do better, but can we do better at handling this? Thanks!

Added to the Stack Overflow to explain how to recover from this problem: https://stackoverflow.com/questions/49659766/my-ddev-project-says-db-service-stopped-and-i-cant-get-it-started-db-service

I'm trying to set up a standard recreation technique for this one.
Drupal6 codebase: https://github.com/rfay/satirtraining.org
Enlarged db: (ask me for the link). It's a 192MB database, with loads of garbage in the watchdog table.
This site is no longer live, has been retired. The db I can provide has been sanitized.
Recreation technique:
ddev config
ddev start
ddev import-db --src=.tarballs/satirtraining.big_watchdog.sql.gz

The import-db takes about 3:05 on my system.
Then ddev ssh -s db. I like to tail -f /var/log/mysqld.log
Then exit docker (on macOS, pull down the menu and "Quit docker"
You'll be blown out of the container immediately.
Now restart docker (Just run docker.app on macOS)
ddev list # Will show the project as stopped
ddev start

The start will fail due to broken database.
(Before beginning any new tests, start with a clean ddev rm -R)

Using the technique in https://github.com/drud/ddev/issues/748#issuecomment-400792445 I was able to (hopefully) confirm @andrewfrench 's results of the problem getting at least worse between v0.14.0 and v0.15.0.
The key change in the container between those mariadb-local versions v0.7.1 and v0.8.0 was using a starter db when starting the project from scratch. I wouldn't expect that to come into play here, since the project is not being recreated here. https://github.com/drud/mariadb-local/commit/c0d8df2ac7eeda7496e8db1983af013bbd800578
We also

added mysql overrides in v0.15.0, and
this was the version in which we changed to docker-compose 'restart' => 'no' (meaning that the project comes up stopped)

Setting docker compose 'restart' to 'always' in v0.15.0 does not fix this.

So with this process I get a happy restart of ddev after a docker down event:
ddev v0.14.0
ddev import-db (from scratch, database complete deleted before)
Hit http://satirtraining.org.ddev.local/junk
Stop docker
Start docker
ddev start (everything is fine)
ddev rm
brew switch ddev 0.20.0
ddev start (everything fine, as expected)
Stop docker
Start docker
ddev start (Everything is fine, as NOT expected)
To me, this says that something about the base db creation or base db installation has some type of problem.

#748 improves this behavior quite a lot, but the docker behavior is still wrong. There's quite a lot of diagnostic information in #748 and upstream docker-for-mac issue is https://github.com/docker/for-mac/issues/3047

thanks for this thread.
I had to use this when windows computer locked up and of course nothing i did with dockers cleaned up the crashed mess..
create a file with .ddev/mysql/innodb.cnf
[mysqld]
innodb_force_recovery = 1
site is now working again.

I do have some random crashes, recently while adding a field in a drupal content type. I couldn't see any docker restarts at that moment and the web container was still up and running.
Now I guess this might be some kind of db-size limit as our db is about 3GB excluding cash tables and thows can grow a lot too.
Thanks for the Info from this thread I can now start debugging this instead of reimporting the db everytime.

@mac-dev are you using v1.0.0? https://github.com/drud/ddev/issues/748, which went in there, should make things a lot better I'm hoping, would love to hear your experience.

Just got stuck with the same problem again with v1.0.0
Failed to restart hessen: db service health check timed out: labels map[com.ddev.site-name:hessen com.docker.compose.service:db] timed out without becoming healthy, status=exited
I've added the recovery.cnf but I guess it just takes to long.

I would love to hear how big your database is, @mac-dev, and whether anything might have happened that would affect the db container's stability. Docker on mac and windows still does not provide notice to its containers when it is shutting down, so you will definitely be better to stop or rm your projects before rebooting or exiting docker, but we're seeing better overall behavior with v1.0.0 (although we do see that the db goes through a recovery process after one of those events).

@rfay it might have been happend as you sayed during docker start / stops or windows reboots due to automated updates. I always try to stop the projects manually but sometimes I fail to do so.
DB is up to 20GB in size depending on caches. Actually I can see about 10GB in the projects mysql directory inside the users .ddev dir.
Meanwhile I changed the ram for docker to 8GB and CPUs to 4.
Trying to ddev start looked okay in the beginning but afterwards I saw that there is no connection to the db-container even it was running.
Then I tried a ddev remove but the the db-container is still there and I can't use it again.
I guess I have to start from scratch again.
I've attached some logs:
ddev_crash_logs.txt

The crash log you show ERROR: for ddev-marsupilami-db  Cannot create container for service db: Conflict. The container name "/ddev-marsupilami-db" is already in use by container "e9a2c1cb8ab03529c13ba7fdff065afb4796d7b68917e77f4ea4807f53633d67". You have to rem
means that you have a stopped container of the same name (probably for the same reason). You'll definitely have to kill it off before proceeding. A ddev rm will do that (and it doesn't kill off the db). But based on what you're describing, you have a broken db anyway, and you'll have a hard time getting it going.
Is there any possibility you have two codebases using projects with the name marsupilami? because that will cause enormous permanent pain.
I do think you're going to have never-ending pain trying to run a 20GB db in a dev environment. It's just not that likely to be successful. You may want to see if there are ways you can pare that db down to what's really important (like remove unnecessary users or content, etc.)  The reality is 20GB databases in production are not often that successful and often need special care and feeding and infrastructure.

I quess I ran into a concatenation of issues. I had to change my pw yesterday so I ran into https://github.com/docker/for-win/issues/1761
The DB files and all other didn't get mounted anymore. So I decided to start from scratch.
Meanwhile we decided to truncate some db contents and we will continue testing this.
As an alternative we did already run ddev under linux with native mariadb and modified db-connection in settings.php

The alternative running maria elsewhere on a machine that's up to it is not a bad idea. Thanks for lettime me know about https://github.com/docker/for-win/issues/1761 - I think I knew about that as a practical matter but didn't know the actual issue.
One thing you may want to consider is using mariabackup to load the db from a mount (inside the mariadb container). It is way faster than mysql against a text dump file (e.g. ddev import-db). It also will work better for using your external db server.


Recreate a standard material with NME

Would be a fantastic demo for 4.1

a PBR material if possible? :D

Not for 4.1 as some blocks will be missing but this is the goal for 4.2

I assume that the #ifdef parts of the shaders should be treated as regular "if" when converting to nme?
So, for something like:
#ifdef MULTIVIEW
	if (gl_ViewID_OVR == 0u) {
		gl_Position = viewProjection * worldPos;
	} else {
		gl_Position = viewProjectionR * worldPos;
	}
#else
	gl_Position = viewProjection * worldPos;
#endif

we would create a MULTIVIEW float constant, with either the value 0 or 1. Both parts of the #ifdef would be created in the editor and the output would be something like lerp(result2, result1, MULTIVIEW): is that what you had in mind?
Side question: I don't think gl_ViewID_OVR is available in the nme?
And what about gl_PointSize?

So we should not treat ALL the special cases. Multiview for instance is out of the equation.
Same for pointSize.
Let's chat here of the special cases but most of the time we will either ignore them or add special blocks later

Ok, going for the vertex shader first.
Should we support:

bones? I assume yes as we do have the block in NME
instances? I assume yes as we do have the block in NME
clip planes?
fog?  I assume yes as we do have the block in NME
morph targets? I assume yes as we do have the block in NME
logarithmic depth?
multiview? no / not now
shadows?  I assume yes as we do have the block in NME (Lights)
point cloud? no / not now

Here are all the defines we should handle:

NORMAL, TANGENT, UV1, UV2, VERTEXCOLOR, MAINUV1, MAINUV2, DIFFUSE, AMBIENT, OPACITY, EMISSIVE, LIGHTMAP, SPECULAR, SPECULARTERM, BUMP, PARALLAX, CLEARCOAT_BUMP, CLIPPLANE, CLIPPLANE2, CLIPPLANE3, CLIPPLANE4, CLIPPLANE5, CLIPPLANE6, FOG, REFLECTIONMAP_SKYBOX, REFLECTIONMAP_SKYBOX_TRANSFORMED, REFLECTIONMAP_EQUIRECTANGULAR_FIXED, REFLECTIONMAP_MIRROREDEQUIRECTANGULAR_FIXED, LOGARITHMICDEPTH,

those are the ones ending up as a float constant with a 0 or 1 value in the nme: can you acknowledge that all these defines must be supported?

DIFFUSEDIRECTUV, AMBIENTDIRECTUV, OPACITYDIRECTUV, EMISSIVEDIRECTUV, LIGHTMAPDIRECTUV, SPECULARDIRECTUV, BUMPDIRECTUV

those are the ones with 3 possible values, 0, 1 or 2: can you acknowledge that all these defines must be supported?

Thanks!

Clip planes: not now
log depth: not now

The reflectionMap should already be done by the reflection texture block
No support for all the xxxDirectUV. the texture node handle that differently

I have an error I don't understand: https://nme.babylonjs.com/#JDCSVX#2
This one does work.
Now, link the output of the normalWorld block to the worldNormal input of the Reflection texture block and you get this error:
Shader compilation error: VERTEX SHADER ERROR: 0:35: 'normalOutput' : undeclared identifier ERROR: 0:35: 'constructor' : not enough data provided for construction
https://nme.babylonjs.com/#JDCSVX#3
It seems it should work?

Here are my comments after creating the vertex shader:

MULTIVIEW / clip planes / log depth not handled (reminder)
Bones: no matricesIndicesExtra / matricesWeightsExtra blocks in the nme interface
It seems the MorphTargets block does not generate code inside the main function of the vertex shader as it should (I checked with "Export shaders")
Could not handle:

	#ifdef NONUNIFORMSCALING
		normalWorld = transposeMat3(inverseMat3(normalWorld));
	#endif

as there are no transposeMat / inverseMat blocks

There's a ###___ANCHOR0___### in the vertex code generated by "Export shaders" (?)
Fog: it seems it does not work yet? When doing a "Export shaders", I can see the code is here but enclosed by #ifdef FOG and there's no way to enable FOG in the nme interface (or I didn't find it).
It seems something like that can't be done (found in the bump code):

vTBN = mat3(finalWorld) * mat3(tbnTangent, tbnBitangent, tbnNormal);

we can't construct a matrix based on 3 vectors? More generally, I think we need a special block in the nme for bump, as there are a number of specific functions called in the fragment code (cotangent_frame, parallaxOcclusion, etc)?
If it can help: https://nme.babylonjs.com/#JDCSVX#4

the bump portion can be handled with the perturbNormal block
can you open an issue for https://nme.babylonjs.com/#JDCSVX#3 ?
morph target should work (like fog) but cannot be tested directly in NME (but can be tested from a playground)

Thanks.
This one also has an error:

https://nme.babylonjs.com/#JDCSVX#5
It happened when connecting the output from "Perturb normal" to the input of "Reflection texture".
If I connect the output from "Normal world" to the input of "Reflection texture", it does work.

I also have an error with this one that uses morph targets:

PG: https://www.babylonjs-playground.com/#HPV2TZ#41
Node material to use: https://nme.babylonjs.com/#6E8GQ2

Can you create an issue for each?

Ok I fixed the #JDCSVX#5

And I fixed the second :)
Nightly incoming

Cool, thanks!

I think there are too many calculations done in the vertex shader for the reflection part compared to the existing standard shaders because I get this error when wiring the two sided lighting calculation (which uses gl_frontFacing):
FrontFacingBlock must only be used in a fragment shader
https://nme.babylonjs.com/#6E8GQ2#1
In the existing standard vertex shader, there is only this code related to reflection:
#if defined(REFLECTIONMAP_EQUIRECTANGULAR_FIXED) || defined(REFLECTIONMAP_MIRROREDEQUIRECTANGULAR_FIXED)
	vDirectionW = normalize(vec3(finalWorld * vec4(positionUpdated, 0.0)));
#endif

Did you tried with the latest fix I did? I should have moved the bump computation to fragment only

Is the online nme updated with your changes?
It still does not work for me:

indeed, the last url I shared does not show any error anymore, however there is a never ending "Please wait loading" on top of the preview frame
this link still shows the same error: https://nme.babylonjs.com/#JDCSVX#5
this one too: https://www.babylonjs-playground.com/#HPV2TZ#41 (node material: https://nme.babylonjs.com/#6E8GQ2)

Yes apparently we are facing a deployment issue (cc @sebavan )

Np, I'm now using the local nme, I see the changes are in.
So, I don't have the error in the first link anymore, however I have a new one with the morph example:

This happens when you click on which node?

Hum, I think it happened when I bound:

this material: https://nme.babylonjs.com/#KPP0Z2
to this nodeMaterial: https://playground.babylonjs.com/#R7CMMJ

but I don't have the error anymore... Maybe it was a transient error.

I also added the option to flag a float as boolean (from the UI standpoint only)

I can't see the option in my local nme, have you already committed it?

I added it in the list of tasks :D

Ok :)
So, now the shader works with morph targets, however I need to know how it is supposed to work when no morph target are there. For the time being, the shader does not work, I have a uniform grey color whereas I would expect to see my reflection texture when the mesh has no morph targets.
I would have supposed that the block:

would output updated position/normal/tangent/uv if the mesh has morph targets, else would output the unmodified position/normal/tangent/uv data. But it seems it outputs 0-vector data instead. Is it expected? Should I create a float like "HASMORPHTARGETS" and take the output from "MorphTargets" if it is one, else take the standard position/normal/... values instead?
I would have thought that all the blocks "Instances", "Bones", "MorphTargets", "Perturb normal" would output modified data if the corresponding feature is enabled on the mesh  ("mesh has instances", "mesh has bones", ...) but else would unmodified but still usable data. Is that not the case ?

It should be the case: https://nme.babylonjs.com/#C39T6E

It's a problem with the #define NORMAL / UV / ... not being set correctly I think.
Try:

browse this PG: !! Online Playground does not work at the time being, so here's the code:

var scrambleUp = function(data) {
    for (index = 0; index < data.length; index ++) {
        data[index] += 0.4 * Math.random();
    }
}

var scrambleDown = function(data) {
    for (index = 0; index < data.length; index ++) {
        data[index] -= 0.4 * Math.random();
    }
}

var createScene = function () {

    // This creates a basic Babylon Scene object (non-mesh)
    var scene = new BABYLON.Scene(engine);

    // This creates and positions a free camera (non-mesh)
    var camera = new BABYLON.ArcRotateCamera("camera1", 1.14, 1.13, 10, BABYLON.Vector3.Zero(), scene);

    // This targets the camera to scene origin
    camera.setTarget(BABYLON.Vector3.Zero());

    // This attaches the camera to the canvas
    camera.attachControl(canvas, true);

    // This creates a light, aiming 0,1,0 - to the sky (non-mesh)
    var light = new BABYLON.HemisphericLight("light1", new BABYLON.Vector3(0, 1, 0), scene);

    // Default intensity is 1. Let's dim the light a small amount
    light.intensity = 0.7;

    // Our built-in 'sphere' shape. Params: name, subdivs, size, scene
    var sphere = BABYLON.Mesh.CreateSphere("sphere1", 16, 2, scene, true);

    var materialSphere = new BABYLON.StandardMaterial("mat", scene);
    materialSphere.diffuseTexture = new BABYLON.Texture("textures/misc.jpg", scene);

    var nodeMaterial = new BABYLON.NodeMaterial("node material", scene, { emitComments: true });

            scene.debugLayer.show({
              embedMode: true
            });
            scene.debugLayer.select(nodeMaterial);

    sphere.material = nodeMaterial;

    var sphere2 = BABYLON.Mesh.CreateSphere("sphere2", 16, 2, scene);
    sphere2.setEnabled(false);
    sphere2.updateMeshPositions(scrambleUp);

    /*var manager = new BABYLON.MorphTargetManager();
    sphere.morphTargetManager = manager;

    var target0 = BABYLON.MorphTarget.FromMesh(sphere2, "sphere2", 0.25);
    manager.addTarget(target0);

    angle = 0;
    scene.registerBeforeRender(function() {
        target0.influence = Math.sin(angle)*Math.sin(angle);
        angle += 0.01;
    })*/

    return scene;

};

bind this material: https://nme.babylonjs.com/#WTSLCL

it does not work, the sphere is uniformly brown. If we look at the beginning of the vertex shader in Spector:

now bind this material: https://nme.babylonjs.com/#4T6H8Y

it does work, the sphere is correctly shaded. In Spector:

bind the first material (https://nme.babylonjs.com/#WTSLCL) that did not work: it does work now.

I also think there's a problem with the "Perturb normal" block.
In the fragment shader, first line for this block is:
//Perturb normal
vec4 output7 = vec4(0.);

I think it should be initialized with the input "worldNormal" instead, so that if "BUMP" is not defined, we end up with the worldNormal unmodified. For the time being, if "BUMP" is not defined, we end up with a normal = vec4(0.) because of this line.

yep! I'm on it (thanks btw :))

I also think there's a problem with the "Perturb normal" block.

I do not think as BUMP is defined by the perturbNormal itself

It's a problem with the #define NORMAL / UV / ... not being set correctly I think.

I can repro..Looking for a fix now

Well actually no I cannot repro :(
(PG is back btw)
I do not see how this could fail: https://github.com/BabylonJS/Babylon.js/blob/master/src/Materials/Node/nodeMaterial.ts#L663
We are using the raw data of the mesh directly to flag the correct define

I do not think as BUMP is defined by the perturbNormal itself

Indeed, but how do you want the end user to use the material from the node editor?
I thought I would recreate the whole standard material with all the features in (morph, bones, instances, bump, reflection, etc) and the end user would disable some features by "some mean".
For some blocks, it's automatic because it depends on the mesh itself:

if no morph targets, the "MorphTargets" block will output unmodified position/normal/tangent/uv so it's ok to keep this block even if no morph targets
if no instances, the "Instances" block will output the world matrix given as input so it's ok to keep this block even if no instances
if no bones, the "Bones" block will output the world matrix given as input so it's ok to keep this block even if no bones

However, for other features like BUMP, it is the choice of the user to use it or not, it does not depend on the mesh itself: how do you let the user choose to use BUMP or not? Removing the block from the material seems complicated to me, I thought there would be some way to #define or not "BUMP" (and all other associated defines to BUMP: CLEARCOAT_BUMP, PARALLAX, ANISOTROPIC, PARALLAXOCCLUSION, etc).
If we keep the "Perturb normal" block even when bump is disabled, we should make sure we output the normal as given in the input.
Same thing for other features like Reflection.

Understood. The need here is to get the code with a "full fledged" std material. If the user does not want bump it can remove it from the node material.

In the future we could think about adding an "isEnabled" boolean per block

Well actually no I cannot repro :(
(PG is back btw)
I do not see how this could fail: https://github.com/BabylonJS/Babylon.js/blob/master/src/Materials/Node/nodeMaterial.ts#L663
We are using the raw data of the mesh directly to flag the correct define

It does not work for me, I did the steps I explained in the previous post in the online Playground.
What I did:

browse this PG: https://playground.babylonjs.com/#IY2BBD
open https://nme.babylonjs.com/#WTSLCL and save it locally with name "mat0.json"
click on the "Node material editor" button in the PG and load "mat0.json": the sphere is all gray

open  https://nme.babylonjs.com/#4T6H8Y and save it locally with name "mat1.json"
load "mat1.json" in place of "mat0.json" for the node material in the PG: it does work, the sphere is pink and shaded

load "mat0.json" in place of "mat1.json" for the node material in the PG: it now works, the sphere has the reflective map:

In the future we could think about adding an "isEnabled" boolean per block

Ok!

It does not work for me, I did the steps I explained in the previous post in the online Playground.

I feel like it was fixed by another fix I did :) Let's wait for the next nightly to merge and see how it goes. I will publish it asap

Ok.
Another thing, I think the answer will be "no" but I still have to ask :)
Do you plan to allow to link two vectors bullets if the source one has more components than the destination one?
For eg, my source vector is "Vertex color" which is a vec4 whereas my destination is a bullet that is a vec3. Linking them would make the shader code to use vertexcolor.xyz automatically, instead of requiring the user to put a color splitter to get the xyz output needed. If the user needs something else than the automatic conversion (which would be x / xy / xyz when the destination is vec1, vec2, vec3), he still can use a splitter.

Haha! You'll be surprised but we have all wwe need to support that :)
Node can define compatible inputs:
https://github.com/BabylonJS/Babylon.js/blob/master/src/Materials/Node/Blocks/Fragment/fragmentOutputBlock.ts#L23

Which blocks would you like to add compatible inputs? I can do it quite easliy

In fact that would be all blocks, with the rule explained above (source vector has more components than the destination vector) :p
But my use case was with "mesh.color" that I wanted to input to a multiply block.

Multiply block is an autodetect mode so it will take the format of what you have (say you have the first input connected on a vec3 then the second one must be a vec3)

Indeed, but I need to multiply the diffuse.rgb with vertexcolor (vec4). I guess I have to use the splitter then.

yes :)

I downloaded the latest source code for the nme (I did get new sources from github), but my procedure above still not work.
When I do the first step (connecting the first node material to the sphere in the PG), I get this in the console (when loading the material into the node editor):

and my sphere is gray.
I need to load the second material then reload the first material to get the right display (and this time no error in the console).

Annoying bug: when using the "DEL" key in the X/Y or Z value of a Vector3, it deletes the currently selected block!

I will fix that!

Additional request: do you have an example of a PG with a standard material with all features enabled (bump, reflection, ambient, diffuse, etc)?

Additional request: do you have an example of a PG with a standard material with all features enabled (bump, reflection, ambient, diffuse, etc)?

Unfortunately no :(

Annoying bug: when using the "DEL" key in the X/Y or Z value of a Vector3, it deletes the currently selected block!

Will be fixed by next nightly.
Checking the other issue now

Ok your repro actually works..Working on a real fix now

For the sake of testing I'm adding this construct:
BABYLON.NodeMaterial.ParseFromSnippetAsync("WTSLCL", scene).then(nodeMaterial => {})

cc @PatrickRyanMS  and @PirateJC  FYI

It is live

@Popov72 : https://playground.babylonjs.com/#IY2BBD#1

It works, and ParseFromSnippetAsync is very cool too.
Thanks!

"Is boolean" is cool too :)!

It seems CTRL+C / CTRL+V does not work for frames.
Try here: https://nme.babylonjs.com/#WTSLCL
Do you plan to also implement "undo" / "redo"?

no undo/redo for now..Maybe in the future through upworks ;)

It seems CTRL+C / CTRL+V does not work for frames.

yeah it acts weirdly...WIll check asap

Will be fixed by next nightly

Ah, we can't set the alpha value of a color4 in the GUI!

What do you mean?

I have a Color4 block and what I can edit is:

I can't edit the alpha value (not in the color chooser either).

ok make sense! will fix that

Also, I can't clamp a vector, the Clamp block assumes the input/output to be float: it should be like multiply and adapt to its input.

Some comments/questions:

How the Light information block is supposed to be used? It seems it is not needed (at least for the standard material)?
I think I don't handle Refraction, as I saw it's a block planned for nme v2?
I don't think I can handle all the things related to Fresnel?

Another one:
At the end of the fragment shader there is:
#ifdef IMAGEPROCESSINGPOSTPROCESS
	color.rgb = toLinearSpace(color.rgb);
#else
	#ifdef IMAGEPROCESSING
		color.rgb = toLinearSpace(color.rgb);
		color = applyImageProcessing(color);
	#endif
#endif

We do have a ImageProcessing node but I can't see a node for "toLinearSpace"?

How the Light information block is supposed to be used? It seems it is not needed (at least for the standard material)?

Not required for stdmaterial

I think I don't handle Refraction, as I saw it's a block planned for nme v2?

Correct

I don't think I can handle all the things related to Fresnel?

We need to create the block

We do have a ImageProcessing node but I can't see a node for "toLinearSpace"?

We need to create the block

Thanks for your answers.
This one is annoying:

Also, I can't clamp a vector, the Clamp block assumes the input/output to be float: it should be like multiply and adapt to its input.

Do you think you can so something for it, or should I create 3 clamps, one for each component? (will need to do a Vector Splitter, 3 clamps and a Vector Merger). Going to bed now, so no hurry ;)

The clamp is now autodetect ;D

I LOVE THIS! I have always split and multi-clamped but consolidation for the win!!