Black stripes in denoised image

Note: There is a $500.00USD bounty on this issue
Apparently, there are black stripes in the denoised image:

Currently, the root cause for this is unclear. I opened a question about this on StackOverflow with no success so far.
I document also some of my doings and findings in some Medium posts:

Bazelizing Open Image Denoise — Part 1: The journey begins
Bazelizing Open Image Denoise — Part 2: More details about the bazelization
Bazelizing Open Image Denoise — Part 3: A failing unit test

How to reproduce
git clone https://github.com/Vertexwahn/rules_oidn.git
cd rules_oidn
cd tests

Run example with Ubuntu 22.04:
bazel run --config=gcc11 //:example

Run example with Visual Studio 2022:
bazel run --config=vs2022 //:example

The example takes a noisy image and denoises it.
...
int main() {
    cout << "Simple denoising example" << endl;

    Image3f color = load_image_openexr("data/cornel_box.naive_diffuse.box_filter.spp128.embree.exr");
    ...

I tested the same input with https://github.com/DeclanRussell/IntelOIDenoiser and got the expected result (without black stripes). Therefore, the problem must be within my bazalization of OIDN or my supporting around it.
If I choose a constant color image, e.g.
// for debug reasons the color image can be initialized with a const color
if(true) {
    for (int x = 0; x < color.width(); ++x) {
        for (int y = 0; y < color.height(); ++y) {
            color.set_pixel(x,y,.5f, .5f, .5f);
        }
    }
}

I also get black stripes.
I also created a branch of oidn that contains a "direct"
bazelization of oidn which is almost similar to rules_oidn: https://github.com/Vertexwahn/oidn/tree/add-bazel-support. In this branch I bazelized oidnTest which contains a few test cases which all pass with success. I bazelized also oidnDenoiser. You can run it via:
# the_cornell_box.pfm is in data folder in the oidn repo
bazel run --config=gcc11 //:oidnDenoise -- --hdr /home/vertexwahn/Desktop/the_cornell_box.pfm -o /home/vertexwahn/Desktop/denoised.pfm

The generated file denoised.pfm shows the same black stripes. It seems that the black stripes are always 5 pixel wide followed by 3 correct looking color stripes.
Not sure if there are any problems with handing over memory to ISPC, or doing the Filter Operation, etc. Since I tested different image formats (OpenEXR, PFM), I assume that the error is not within my way of storing images.
Expected fix
The expected fix should identify the root cause (maybe ispc_rules, some memory layout somewhere, etc.) and provide PRs to fix it. The denoised image should look identical to the CMake build. Could be that the fix will be less than 10 lines in code - maybe single line change. In the end rules_oidn should work out of the box on Windows and Ubuntut 22.04 (as it does currently - but without the black stripes) - macOS is not expected to work since this requires a different version of oneDNN which is out of the scope of this issue.

Hi, i was also able to reproduce the issue, i'm in the process of debugging.
Only thing i was able to extract to this moment is that the issue is specifically in the filter execution. I'll let you know if i get more details for a possible fix.

Hi, i've extracted the image data after the execution of the first layer of the network in both cases, there seems to be a sharp difference between the output of the two.
I've confirmed that the tza files used by both are the same, but could this indicate an error in the loading of the weights?
== CMAKE ==

== BAZEL ==

Hi, i think i'm zeroing in on the issue, i've changed both the versions of InputReorder_kernel to just set the entire tile to a solid 1.0f value, this way:

Here the results:
== CMAKE ==

== BAZEL ==

So what i think now is that is some kind of memory alignment issue that writes wrongly the calculated values.

@Vertexwahn , it looks like @parvit goes the code-level way to fix this issue, which is fine.
I go the "technical project management" way trying to strip the "bazel" requirement, because of the high effort to support an alternate (to what the project uses) build-system.
You're risking to invest more and more time in tool/build maintenance. Take a look at:

#3

@Vertexwahn @abebeos I assumed the bazel usage was the whole point of the task, but if its not required than we could make it so the cmake generation procures and builds automtically all the dependencies.
Let me know if this is a route you are interested in, in the meantime i'll continue my investigation.

I assumed the bazel usage was the whole point of the task

It is. I usually start any task with Requirements & Feasibility analysis (the result of which will most possibly be: "MUST Requirement: Bazel). So all good, I guess.

@Vertexwahn @abebeos I assumed the bazel usage was the whole point of the task,

Exactly.
@abebeos @parvit
The goal of the Bounty is to use "pure" Bazel, not CMake.

@Vertexwahn , I looked at all this, did some trials with your sources, reviewed your initial PR etc.
What you need additionally is:

polish the projects, make reproduction far(!) easier (this allows more people to take a look fast&easy). The more eyes look at the problem, the more possible the bug is spotted.
refine any relevant PRs (make them non-intrusive) thus the target project accepts them easier
alternatively: workout a solution to create a low-maintenance fork of oidn that supports bazel

Let me know if you're willing to add an additional bounty (can be again $500) for those tasks.
My estimation is that within a timeframe of 2 weeks (not fulltime of course) this can be solved.

(i would open a project, file some issues that would need action from your side, estimated 1 hour effort. And now and then a question.)

@parvit still working on this?

My attempt is abstractly, without detail involvement.
How familiar are you with bazel? (My mindset is more around maven/cmake.)
Could you attempt to bypass the unpack step as described within https://github.com/Vertexwahn/rules_oidn/issues/8 ?

@Vertexwahn , you'll find my preliminary results here: https://github.com/Vertexwahn/rules_oidn/issues/5
The effort to apply those findings/changes to your sources are very high, so I would prefer if you would take this on (as you're fully familiar with what you wrote). Should be more efficient this way.
I still believe that the  best way to  solve this would be  a oidn fork with bazel support (one that I would keep "clean" right from the beginning, e.g. add only the minimum needed changes, always strive to use the exact same source-code that should always function with cmake). But the effort for this is beyond a $500 bounty.

the binary same

#8 verifies the tiny possibility that something went wrong during extraction. Could be verified manually, too, i guess, via file comparison.

That is exactly what i did, a binary comparison of the files.
Il Mar 14 Mar 2023, 20:00 abebeos @.***> ha scritto:

the binary same
#8 https://github.com/Vertexwahn/rules_oidn/issues/8 verifies the tiny
possibility that something went wrong during extraction. Could be verified
manually, too, i guess, via file comparison.
—
Reply to this email directly, view it on GitHub
https://github.com/Vertexwahn/rules_oidn/issues/1#issuecomment-1468666554,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AWGJD4EOU47DTPU543KK6CTW4C555ANCNFSM6AAAAAAVLGHDB4
.
You are receiving this because you were mentioned.Message ID:
@.***>

That is exactly what i did, a binary comparison of the files.

You wrote earlier:

I've confirmed that the tza files used by both are the same,

"tza" files. I'm referring to the resulting cpp/h source files
In any way, i'm awaiting @Vertexwahn here, many things need a rollback/change.
Though a full replay would be the best option.

Also those binaries are exactly the same.

Hi @Vertexwahn i have found the issue, i'll update tomorrow on the details

Also those binaries are exactly the same.

Ok, thus now we can say that

binary tza files are identical
extracted Source files are identical

i have found the issue, i'll update tomorrow on the details

@Vertexwahn, independent of this, we should do a replay, thus you can avoid similar incidents in future

The expected fix should identify the root cause

Root cause identified as "wrong source-code version used for mkl_dnn"
My cross-tests show that those versions are not compatible.
Changing your bazel build-scripts is not trivial, though, due to the used build files from "TensorFlow".
You'll find more details on how to proceed within.

https://github.com/Vertexwahn/rules_oidn/issues/7

My fix is about the missed definition in ipcs_rules of the OIDN_DNNL
define, so no version change necessary.
@Vertexwahn can you indicate which fix would you prefer as i have to
prepare the fixes in the bazel projects, if you don't think you'll accept
my solution than i can stop.
Regards,
Il Mer 15 Mar 2023, 11:34 abebeos @.***> ha scritto:

The expected fix should identify the root cause
Root cause identified as "wrong source-code version used for mkl_dnn"
My cross-tests show that those versions are not compatible.
Changing your bazel build-scripts is not trivial, though, due to the used
build files from "TensorFlow".
You'll find more details on how to proceed within.

#7 https://github.com/Vertexwahn/rules_oidn/issues/7

—
Reply to this email directly, view it on GitHub
https://github.com/Vertexwahn/rules_oidn/issues/1#issuecomment-1469756056,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AWGJD4ESTQ52UZNMYCCDJGDW4GLNDANCNFSM6AAAAAAVLGHDB4
.
You are receiving this because you were mentioned.Message ID:
@.***>

My fix is about the missed definition in ipcs_rules

if you've validated/verified your fix, the I'd just go on ('cause Vertexwahn seems to be busy with other work, no replies since days).
The "version changes" and the overall build-system replay/rewrite is anyways a necessity (= MUST requirement) to achieve production-grade bazel support. This would be independent of some direct fix. That's why I asked initially for an additional bounty within https://github.com/Vertexwahn/rules_oidn/issues/1#issuecomment-1464976927.

You can ignore what @abebeos is saying - since I follow here the policy "He who pays the piper calls the tune".

No need for such statements, my comments were always either neural or encouraging.

Changing bazel build scripts is maybe for @abebeos difficult, but not for me ;)

It's not about difficulty, but about reducing effort. Producing a simple to understand and to maintain result is the difficult (and time consuming) task.
So, even if @parvit fixes the bug, the current general status of your bazel build is not of production quality.
So, my solution path targets quality, and fixes the bug as a side effect:

#5
https://github.com/Vertexwahn/rules_oidn/issues/1#issuecomment-1464976927

The use of rules_foreign_cc should be avoided.

This is an important construct for a lower risk creation of bazel builds.

It is expected that oneDNN is used.

It is standard-procedure to use the exact versions of the original system.
Here: Bazel build must not introduce different behavior by using different versions

@Vertexwahn The three PRs that comprise my fix to this issue are attached.
My test steps (on linux):

remove the entire .cache/bazel folder
clone the repo https://github.com/parvit/rules_oidn and switch to the fix-black-stripes branch
enter the rules_oidn/tests folder
execute bazel run --config=gcc11 //:example
After the run, the file denoised.exr can be found under .cache/bazel/_bazel_root/<build-hash>/execroot/__main__/bazel-out/k8-fastbuild/bin/example.runfiles/__main__/denoised.exr

Hopefully this is all clear and reproducibile without much effort.
Regards,

Resolved by @parvit - Thanks for your help!

@parvit , nicely done.
@Vertexwahn , I do not understand why you refuse all my suggestions re enhancing the overall bazel build quality, it was stated clearly by yourself that you wish so:
within https://medium.com/@Vertexwahn/bazelizing-open-image-denoise-part-1-the-journey-begins-a16c78ea1b88 you wrote:

I personally would prefer to add Bazel support directly to the official main branch, to be honest.

This is of course a highly rational and relevant goal (bazel support right within https://github.com/OpenImageDenoise/oidn)
This is why I suggested early on to do this work in parallel to looking for a bug-fix:

#5
https://github.com/Vertexwahn/rules_oidn/issues/1#issuecomment-1464976927

Though I was initially fixated on #7, the bug would have been fixed by applying #10.
nearly all steps mentioned within #5 are necessary to achieve a oidn bazel build that is

is compatible to the cmake build of https://github.com/OpenImageDenoise/oidn
has the potential to be merged in the original project

Let me know if this would be still of interest for you.
=> A oidn bazel-build that is compatible to the original cmake build


Start application minimized

It would be nice if you included a setting to start the application minimized (in the system tray).
Edit: As far as i can see this feature does not yet exist.

This should be a relatively easy addition.

Added in release 0.3.3


Bot crashes when loading badges

First of all, thanks a lot for providing this bot to us!
I would like to report the following issue:
I run the latest version of the bot under Windows 11. Downloaded it from Github only a couple of days ago.
A user who wanted to buy sets for gems from the bot has reported to me the following behavior:
"it can not load my badges when i do !check command
the bot crashes and restarts"
The attached screenshot of the command window shows that after the !check command was entered, the bot produced a warning message claiming the sessionid had been expired. After the warning message, the red error message that the bot could not log into steamcommunity.com without first being connected to the Steam network was displayed. This behavior was not a coincidence but repeated systematically several times in a row when the user retried to enter the !check command.
This issue may or may not be related to the following behaviour I observed by myself when trying to use the bot in a test:
After entering the !check command, the bot told me that there were 15 sets available for me. However, when entering !buygems 15, the bot told me that there weren't enough sets available, although no set was sold in the mean time and all the 15 sets originally detected were actually still be there. Due to this error message, buying the 15 sets originally detected was not possible.
When entering !buygems 14 in order to test whether buying one set less would work, the bot actually did send out a trade offer about these 14 sets (which is of course the correct behaviour). The set "MTB Downhill Simulator" (appid: 475990) was not included in this offer although I had not crafted it and therefore the set was relevant for me. For this reason, would have expected this set to be included in the trade offer. Maybe this "MTB Downhill Simulator" set was the 15th set originally detected in the !check command but not recognized any more when entering the !buygems 15 command. So, maybe the bot has trouble dealing with certain sets like "MTB Downhill Simulator", causing errors.

This error in the image occurs when the script lost connection to the Steam servers but was unable to automatically reconnect. Restarting the script manually should resolve the issue.
The error with the sets of the game "MTB Downhill Simulator" unfortunately I can't reproduce it, because I don't have the cards. So I can't find the cause of the problem. Are you sure you have a complete set of this game?

Thank you very much for your reply!
You are right in that I currently do not own a complete set of the game "MTB Downhill Simulator". I sold this set recently to a user. He tried to buy it from the bot and ran into pretty much the same connection problems as shown on the above screenshot. For this reason, selling the set (and others) via the bot was not possible. We had to do trade manually instead.  I think I will have a new complete set of "MTB Downhill Simulator" ready in a few days. I'll let you know when it is ready so we can investigate whether the issue occurs again.
Given that issue with the lost connection seems to happen more frequently, is there maybe a way to improve connectivity or to renew automatically logins from time to time? My internet connection works fine. So I do not really see a reason why connections get lost so often. Restarting the script manually is not really an option if connections are lost frequently because it basically means that the bot would be frequently inoperational for longer time periods until the next manual restart happens. Users will probably not accept this.
Also, I asked the same user who ran into the issue shown in the above screenshot to redo a !check. He told me that this time the bot went offline about a minute after the !check and came back after 2 minutes or so, The user never got a response from the !check. The following screenshot shows the log from this !check. This time, there were no error message displayed but unfortunately, the !check still did not work as intended. The user who did this !check has crafted 10,000+ badges. So maybe there is a problem with badge count.

I made an update to the script to try to resolve this issue of reconnecting to Steam servers.
Regarding the issue with the !Check command, there is an issue with the data returned by the steam servers for users with many badges.
See in this test I did, the server informs that the badge level is 1, when in fact the correct level would be 5. This is what causes the error in the !check command. Unfortunately there is nothing I can do to solve this , because the data reported by the server is wrong.

Are you sure you queried the right badge type here? The json-response also says border_color=1 which usually means "foil badge". Level 1 is the correct level for the foil badge, see here:
Normal badge:
https://steamcommunity.com/id/StiGGe/gamecards/24010
Foil badge:
https://steamcommunity.com/id/StiGGe/gamecards/24010/?border=1
What is the exact query you used to retrieve this data?

Sorry, I actually made this mistake in my test. 😅

I just tried the version published a few hours ago with the fix of reconnecting to Steam.
About half an hour after I started the bot, I received the error message webLogOn() is not a function (see screenshot). Looks to me like the bot decided a renewal of the login was necessary but the login attempt failed to due to an error in the program.

I find it somewhat hard to believe that a new login actually was necessary only half an hour after the bot has been started. In these 30 minutes, the account has not been in use elsewhere so there should not have been an automatic logout. How does the bot decide about whether a new login is necessary or not?  Is this procedure reliable?

I also just checked if the bot can send out trades despite of this login issue. I did not restart the bot (so that the above error was still effective). When entering !check, the bot told me that there were 35 sets available for me. Entering !buygems 35 resulted in the error message "There are not enough sets in stock. Pleasy try again later". The same happened when entering !buygems 34. After entering !buygems 25, the bot finally sent out a trade offer about 25 sets.
First, this shows that the bot actually was logged in despite of the above errors. If it had not been logged in, sending out a trade offer would not have been possible.
Second, it shows that the problem described in my original post for the set "MTB Downhill Simulator" does not seem to be a specific problem with this set. It cannot be due to this set alone since the set was not in stock. Maybe counting sets in the !check request and in the !buygems request sometimes simply leads to different results, i.e. there might be a bug in at least one of these counting routines.

I just tried the version published a few hours ago with the fix of reconnecting to Steam.
About half an hour after I started the bot, I received the error message webLogOn() is not a function (see screenshot). Looks to me like the bot decided a renewal of the login was necessary but the login attempt failed to due to an error in the program.

I find it somewhat hard to believe that a new login actually was necessary only half an hour after the bot has been started. In these 30 minutes, the account has not been in use elsewhere so there should not have been an automatic logout. How does the bot decide about whether a new login is necessary or not? Is this procedure reliable?

The script checks every 15 minutes if you are logged in. It also monitors for connection error events.

I made a new version to fix some bugs. Check if with this new version the bugs persist.

I installed the new version #93 and retried. The error message about webLogOn() not being a function is gone. Also, the error with counting sets in the !check and !buygems request seems to be gone. !check still returns the 35 sets originally found under version #92. !buygems 35 now works under version #93 and results in a trade offer sent by the bot about these 35 sets. The same is the case for !checkone: here, !checkone finds 26 sets which can be purchased by a !buyonegems 26 command in a trade offer.
Thanks a lot for these fixes!
There still seems to be the following problem with the !check command though:
!check does not find all the sets the user actually needs. For example, !check did not retrieve a set of the game "Corroded", appid 446030. I have not crafted the badge yet. Therefore, the set was found by !checkone (i.e. it is part of the 26 sets), but for some reason not by check (i.e. it is not part of the 35 sets). Also, the bot has a complete set of the game "Egg Returns Home", appid 363790. I have crafted the badge up to level 3. Still, this set was not found by the !check command although two levels of that set are still needed.

Here is some further data on the login and connection issues. The following screenshot shows the protocol generated by version #93 during the day.

The bot checked the login to Steam regularly. Every 4 hours, it came to the conclusion that it is not logged in an tried to relog. At 19:54, a user entered a !check command. The !check command failed with the same error mesage known from previous versions (not logged in). So, despite of all attempts to relog, there is unfortunately no improvement when it comes to actually doing transactions with the bot using the !check commands.
When I do a browser login of my Steam account, the account is usually still logged in the next day. This would indicate that logins usually do not expire that frequently. For this reason, the results of the login checks do not seem reasonable to me. Maybe the bot actually is logged in and there is just something wrong with the login check.

Go to this directory %localappdata%\doctormckay and delete everything. Remove the node_modules folder and reinstall the dependencies again.
If you are running the script on your personal machine, try running it on another machine or in a virtual machine. There may be some conflict going on.

I deleted the two subfolders "node-steam-tradeoffer-manager" and "node-steamuser" in the doctormckay directory and the folder "node_modules" in the main directory of the bot. I also downloaded the new version #95 and executed the install.bat script. I do not have another machine available right now and do not know how to set up a virtual machine on Windows for letting the bot run on it. Can you maybe recommend something to me?
Before I did these upgrades, the bot generated under the old version #93 the error message shown on the following screenshot:

The error says that the bot already is logged in. However, only few minutes ago when a !check command was entered, the log said the bot was not logged in.
Under version #95 the bot now finds 65 sets in a !check command as compared to 35 under the previous version #93. This may be the correct result because there are currently 65 sets in stock and I probably do not have them maxed out yet. Definitely looks like an improvement to me. Thanks!

I recommend that you use WSL2 and run Ubuntu. See how to do it here.
Then follow the tutorial to configure the script.

Maybe there is an interference with WinAuth. I had registered the 2FA of the bot account with WinAuth in order to obtain the identitySecret and sharedSecret codes. I had WinAuth running together with the bot. Maybe WinAuth also does auto-logins and these interfere with the bot? I now removed the bot account from WinAuth and will check if this can improve things. If this does not help, I try to migration the bot to my Raspberry Pi. Unfortunately, I do not have an Ubuntu PC but maybe your suggestion can be implemented on a Raspberry Pi as well.
Here is the log from the night run under version #95 (after cleaning up the installation in the way you suggested). The issues with the connections remained. This time, the bot tried to generate a trade offer on a !buyonegems command. The offer was never sent out though. The bot finally ended up asking to enter a SteamGuard code.

I have one more suggestion to make:
For a user having crafted thousands of badges, it takes the bot more than a minute to send out an answer to a !check command. This is much longer than for other bots where this takes only several seconds, even for users with thousands of badges.
The processing time of about a minute cannot be due to loading the badge data for the thousands of badges. This can be easily verified by entering the Steam API command for loading this data in a browser. The result is usually there in a few seconds.
I do not know the bot code and therefore can only guess what might be a possible reason for the 1 minute processing time for big users. Given it is not related to the download of data, it could be due to the logic applied for matching bot sets with user badges. A simple way to do this matching is to use a nested loop iterating through the n user badges and for each user badge through the m bot sets and compare appids. A simple logic like that would require O(n*m) comparisons of appids and therefore have long processing time for big users with large number of badges n. Maybe the logic is not even implemented explicitly in the bot but hidden somewhere in an external function called by the bot which is based on this nested loop logic.
If this guess is correct, then the following approach might help to speed up the matching significantly:
Step 1: Sort the bot sets and user badges by appid
Step 2: Apply a matching logic for sorted arrays similar to the logic of the functions "intersection_destructive" or "intersect_safe" outlined in the following stackoverflow post:
https://stackoverflow.com/questions/1885557/simplest-code-for-array-intersection-in-javascript
Due to the sorting, it is only necessary to compare first elements of the sorted arrays rather than going through every single element in a nested loop. Hence, comparisons are reduced from O(m*n) down to O(m+n). This should lead to a significant improvement in performance for users with large badge numbers n.
Maybe it also would increase stability. Users with large badge numbers have told me that sometimes they do not receive an answer on their check request at all but the bot crashed (see above).
Can you check if my guess is correct and if so, whether adopting a sorted array solution can speed up things?

I wrote this script 3 years ago when I was still learning to program, so there are several problems with it. For example: performance issues, memory leak and too much code repetition. Which makes it very expensive to make major changes to this code, because the code was not written very well due to my almost zero knowledge at the time about software development and software quality.
Unfortunately at the moment I don't have much time to make major changes or write a new more optimized version.

If you have another account, test the script on it. It's happened to me that the script doesn't work on a specific account, I don't know why this happens, but it can happen.


Allow user request/response for backup

The initial version of the backup feature included the start_backup API method which started the backup process without waiting for completion and return the end result.
The user had to monitor the backup events (started, succeeded failed) in order to track the in progress operation.
The PR changes the following in order for the UX to be simpler:

change the start_backup to backup which now:

Enque a backup request
Wait for it to complete and return the result.

The BackupWatcher now monitors a request channel in addition to the sql hooks channel, so the user requests goes to that channel directly and not via the sync_requests table as before.  This allows the removal of add_sync_request method completely.
I have noticed the use of temp directory fails in Android so the code was changed to use a temporary directory inside the sdk working dir.

Is the intent to remove the backup events and return the status as the result?
Side question, in the backup_status response, does backed_up field switch to false once the SDK deems it out of date?

Is the intent to remove the backup events and return the status as the result?

No, events are useful as well so this doesn't change.

Side question, in the backup_status response, does backed_up field switch to false once the SDK deems it out of date?

SDK trigger backup whenever needed on a best effort. In cases when the automatic backup fails (network error for example) an event will be fired (BackupFailed) and this flag will be switched to false. The intention is that the application will be aware of that and may choose to invoke the backup API explicitly.

No, events are useful as well so this doesn't change.

Ah, I see. First reading I was expecting a backup_status response from the backup request. But a non-error response should be considered a successful backup.


Ce/india analytics

@sravfeyn
This provides credentials for our analytics services (namely hubspot and google analytics) for the softlayer environment, so that we can begin tracking there as well.
Dont merge until https://github.com/dimagi/commcare-hq/pull/12882 is merged.

@sravfeyn removed hold off on merge tag now that the other pr is merged


bug: TLSStore with Wildcard Certificate and sniStrict: true does not work

Welcome!

[X] Yes, I've searched similar issues on GitHub and didn't find any.
[X] Yes, I've searched similar issues on the Traefik community forum and didn't find any.

What did you do?
I am not sure if this repository or traefik/traefik-helm-chart is the correct repo to open this in, but I am under the impression this might be something in Traefik itself, not in Kubernetes (CRDs).

I am using Cloudflare (CF) Origin Certificates (Traefik can use them to talk to the CF proxies). This is nice because I do not need certificate resolvers like Let'sEncrypt. I was upgrading from v2.10.7 to 3.0.2 and first completely uninstalled all the old CRDs and Traefik, then installed the new version.
All of this worked nicely and I encountered no issues.
I also use TLSOptions:
tlsOptions:
  default:
    minVersion: VersionTLS12
    maxVersion: VersionTLS13
    sniStrict: true
    curvePreferences: [ secp521r1, secp384r1, secp256r1, x25519 ]
    alpnProtocols: [h2, h3]

Assume my domain is mydomain.test. The CF Origin Certificate is valid for *.mydomain.test and mydomain.test.
What did you see instead?
When trying to open subdomain.mydomain.test, what I am now seeing is this, though:

It seems Traefik does not use the certificates from my TLSStore anymore. Looking at the debug logs:
TLS: strict SNI enabled - No certificate found for domain: "subdomain.mydomain.test", closing connection
2024-06-16T15:29:05+02:00 DBG log/log.go:245 > http: TLS handshake error from <IP_ADDRESS>:36198: tls: no certificates configured

I am also seeing
2024-06-16T15:15:33+02:00 DBG github.com/traefik/traefik/v3/pkg/tls/tlsmanager.go:321 > No default certificate, fallback to the internal generated certificate tlsStoreName=default

which is unexpected because both the Secret and the TLSStore exist:
$ kubectl describe secrets -n ingress
Name:         cloudflare-origin-certificate-com
Namespace:    ingress
Labels:       <none>
Annotations:  <none>

Type:  kubernetes.io/tls

Data
====
tls.key:  X bytes
tls.crt:  Y bytes

$ kubectl describe tlsstores.traefik.io -n ingress
Name:         default
Namespace:    ingress
Labels:       app.kubernetes.io/instance=traefik-ingress
              app.kubernetes.io/managed-by=Helm
              app.kubernetes.io/name=traefik
              helm.sh/chart=traefik-28.3.0
Annotations:  <none>
API Version:  traefik.io/v1alpha1
Kind:         TLSStore
Metadata:
  Creation Timestamp:  2024-06-16T10:20:37Z
  Generation:          9
  Resource Version:    16866321
  UID:                 288e3946-70d5-41bb-94d3-83771c65c6a8
Spec:
  Default Certificate:
    Secret Name:  cloudflare-origin-certificate
Events:           <none>

What bugs me is the log message
TLS: strict SNI enabled - No certificate found for domain: "subdomain.mydomain.test", closing connection

I used sniStrict: true before as well, though; hence I think SNI should not be the problem here. But, when using sniStrict: false, everything starts to work. Am I misunderstanding something here? I had assumed that with a wildcard certificate, I could use sniStrict: true (like before).

related: https://github.com/traefik/traefik-helm-chart/issues/851
What version of Traefik are you using?

See previous fields for relevant configuration snippets. Here is the IngressRoute:
---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute

metadata:
  name: subdomain

spec:
  entryPoints: [ websecure ]

  routes:
    - kind: Rule
      match: Host(`subdomain.mydomain.test`)

      services:
        - name: someservice
          namespace: some-namespace-not-ingress
          port: http
          scheme: http

  tls:
    store:
      name: default
      namespace: ingress

I tried countless different configurations now, omitting the tls section in the IngressRoute, etc.
If applicable, please paste the log output in DEBUG level
2024-06-16T15:15:33+02:00 DBG github.com/traefik/traefik/v3/pkg/tls/tlsmanager.go:321 > No default certificate, fallback to the internal generated certificate tlsStoreName=default
2024-06-16T15:15:58+02:00 DBG github.com/traefik/traefik/v3/pkg/tls/tlsmanager.go:216 > TLS: strict SNI enabled - No certificate found for domain: "subdomain.mydomain.test", closing connection
2024-06-16T15:15:58+02:00 DBG log/log.go:245 > http: TLS handshake error from <IP_ADDRESS>:17586: tls: no certificates configured

@mmatur any progress on this one?

Hi @georglauterbach,
Unfortunately I didn't had the time to look at it before. Just starting to reproduce the issue on my side and Iw will come back to you.

Hi @geoffgarside,
I gave it a try and I have the same behaviour between Traefik v2.10.7 and Traefik v3.0.1.

Test 1

sniStrict set to true

---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: subdomain
spec:
  entryPoints: [ websecure ]
  routes:
    - kind: Rule
      match: Host(`subdomain.mydomain.test`)
      services:
        - name: someservice
          namespace: some-namespace-not-ingress
          port: http
          scheme: http
  tls:
    store:
      name: default
      namespace: ingress

Result: Connection closed

Test 2

sniStrict set to true

---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: subdomain
spec:
  entryPoints: [ websecure ]
  routes:
    - kind: Rule
      match: Host(`subdomain.mydomain.test`)
      services:
        - name: someservice
          namespace: some-namespace-not-ingress
          port: http
          scheme: http
  tls:
    secretName: cloudflare-origin-certificate # define a secret name to load the certificate in traefik because the when sniStrict is set to true, traefik will verify certificates loadded but not default certificate.
    store:
      name: default
      namespace: ingress

Result: It works

Test 3

sniStrict set to false

---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: subdomain
spec:
  entryPoints: [ websecure ]
  routes:
    - kind: Rule
      match: Host(`subdomain.mydomain.test`)
      services:
        - name: someservice
          namespace: some-namespace-not-ingress
          port: http
          scheme: http
  tls:
    store:
      name: default
      namespace: ingress

Result: It works

With sniStrict option set to true, Traefik will check certificates that have been loaded and will not check the default certificate.
Doc reference:

With strict SNI checking enabled, Traefik won't allow connections from clients that do not specify a server_name extension or don't match any of the configured certificates. The default certificate is irrelevant on that matter.

It the same behaviour in v2.10
This behaviour has been introduced in Traefik v1.7 and has never been changed since the merge of the PR.
I unassigned myself to have the issue popping during the next triage and be able to discuss with other maintainers about it.

I faced similar issues after doing an upgrade from v2.9.0 to v2.11.2 in helm.
With DEBUG enabled for my traefik deployment running on k8s, I could see that the default secret was no longer loaded using the TLSStore definition.
As per docs, the Kubernetes CRDs API Group traefik.containo.us has been deprecated in v2.10, and its support will end starting with Traefik v3. The API Group traefik.io has to be used instead.
Although the new traefik.io groups were applied correctly, I noted that the kubectl commands strangely still kept returning the old CRD in API group traefik.containo.us
kubectl get tlsstore -n traefik -o yaml

To check if the issue could be fixed, I rolled back Traefik to v2.9.0, but to my surprise, it still did not work
I double checked that the latest CRDs mentioned in the helm chart were already applied.
Looking at the TLSstores CRD definition properties, it also included an array object to pass a list of secret names as certificates.
apiVersion: traefik.io/v1alpha1
kind: TLSStore
metadata:
  name: default
  namespace: traefik
spec:
  certificates:
    - secretName: wildcard-tls-cert
  defaultCertificate:
    secretName: wildcard-tls-cert

After upgrading Traefik back to v2.11.2 and adding this change, the default certificate was loaded without any issues and the SNI errors were no longer logged.
I will plan the upgrade to V3 for later when more information about this issue is available.

Hello @Benly-walter and thanks for your interest in Traefik,
The issue you are facing does not seem to be related to the original one, which is only about the behavior of using the sniStrict option with the defaultCertificate.
Please could you open a new issue?

Hello @Benly-walter,
At first glance, it looks like a configuration issue during the migration. It is not the same issue, as you also use the wildcard cert as a certificate in the configured TLStore. This issue concerns the non-usage of the default certificate when the sniStrict option is enabled.
Feel free to ask for help in the community forum. It's quite active, so you're likely to find an answer to your question quickly.
If you believe I am mistaken, please open a new issue.

@kevinpollet do you have a rough estimation of when I can estimate this issue to be tackled? I'd like to turn sniStrict back on without adding the certificate explicitly to every ingress :)

Hello,
Today, it is possible to also add the default certificate to the store certificates to fix this behavior. Thus, even if the sniStrict: true option is enabled, this certificate will be used when Traefik tries to find a certificate matching the TLS connection SNI. See the following example:
tlsStore:
  default:
    certificates:
      - secretName: cloudflare-origin-certificate
    defaultCertificate:
      secretName: cloudflare-origin-certificate

We are still discussing if we want to keep the defaultCertificate to be a certificate only used when there is no SNI in the TLS connection, or if we want to fix this behavior by introducing a syntaxic sugar which will always add the defaultCertificate to the store certificates.
@georglauterbach is there any reason to not add the default certificate to the store certificates manually?

@georglauterbach is there any reason to not add the default certificate to the store certificates manually?

Oh, absolutely not :) I just wasn't aware of the defaultCertificate option. I will try to add the option today and report back.

Didn't get the time to test this, and I'll be on vacation for 2.5 weeks now. I'll report back when I'm back from vacation :)

I can now confirm that using
tlsStore:
  default:
    certificates:
      - secretName: <SECRET NAME>
    defaultCertificate:
      secretName: <SECRET NAME>

works with sniStrict: true. Using only one of certificates or defaultCertificate will not do the trick.

Great, thanks
I am closing accordingly


ashuffle: 3.10.1 -> 3.12.3

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/joshkunz/ashuffle/releases.
meta.description for ashuffle is: "Automatic library-wide shuffle for mpd"
meta.homepage for ashuffle is: "https://github.com/joshkunz/ashuffle"
meta.changelog for ashuffle is: ""
Updates performed

Version update

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
/nix/store/6dr67cij7842paqwaxgln4wfki3s2j3h-ashuffle-3.12.3/bin/ashuffle passed the binary check.
1 of 1 passed binary check by having a zero exit code.
0 of 1 passed binary check by having the new version present in output.
found 3.12.3 in filename of file in /nix/store/6dr67cij7842paqwaxgln4wfki3s2j3h-ashuffle-3.12.3

Rebuild report (if merged into master) (click to expand)

4 total rebuild path(s)

1 package rebuild(s)

1 x86_64-linux rebuild(s)
1 i686-linux rebuild(s)
1 x86_64-darwin rebuild(s)
1 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
ashuffle

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/6dr67cij7842paqwaxgln4wfki3s2j3h-ashuffle-3.12.3 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A ashuffle https://github.com/r-ryantm/nixpkgs/archive/6344193b4dd9797d25e01235f8f07874bd0cf58b.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/6dr67cij7842paqwaxgln4wfki3s2j3h-ashuffle-3.12.3
ls -la /nix/store/6dr67cij7842paqwaxgln4wfki3s2j3h-ashuffle-3.12.3/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

ashuffle

Maintainer pings
cc @tcbravo for testing.

Tested and working on NixOS unstable. Changes LGTM.


arrange according to date in increasing order from January to December

@Abhirv08  u want to do it?

@arpit456jain yeah sure


Fix coding style

I've run pycodestyle on most of our modules and the results are very bad:

a lot of white spaces
a lot of bad indentaiton

Please fix the problem.

I've fixed the coding style according to pycodestyle. Not a single warning now ;)
Closed in 79824b100d39d89774835845900b5abc15645e79!


[WIP] Changed dwd probe kubeconfigs to client certs.

What this PR does / why we need it:
Earlier the shoot-wise kubeconfigs for dependency-watchdog probes were using static tokens.
But these secrets were not rotated along with kubecfg.
So, after rotation, the dependency-watchdog probes fail for that shoot.
Which issue(s) this PR fixes:
Special notes for your reviewer:

Now client-certs are used for the probe kubeconfigs.
I have also added the code to delete old probe kubeconfig secrets contain the tokens.
Testing is pending. I will finish that tomorrow and remove the WIP prefix

Release note:

Changed dependency-watchdog probe shoot kubeconfigs to use client certs.

This is just a suggestion. We can take it up later also if we feel the need to enhance dwd. Just wanted to float the idea. :)

👍 During the first implementation, I did consider supporting all the variants of probes as supported by kubelet in livenessProbe and readinessProbe but opted for the simpler implementation to begin with 😉. Adding socket connection probe certainly makes sense.

Struggling with local setup. So, testing is taking longer than expected.

Update: I am facing more local dev setup issues for testing.  However, #1867 is already merged (Thanks, @ggaurav10!) and mitigates the problem of waking up clusters after rotation of kubeconfig.

Finally, the local setup worked enough to test this. I have tested that the probes using the new certs work and that old toke-based certs are replaced for existing shoots.
This PR is now ready for review/merge.

@rfranzke Thanks for the review! I have addressed the comments. Can you PTAL?


Wrong reporting for the CHML validation

Given the following setup:
ontrackCliSetup(
  validations: [
      [
        name: "Scans",
        chml: [
            failed: [
                level: 'CRITICAL',
                value: 1,
            ],
            warning: [
                level: 'HIGH',
                value: 1,
            ]
        ],
      ],
  ],
)
// ...
ontrackCliValidateCHML(
  stamp: 'Scans',
  critical: 0,
  high: 34,
)

We expect the result of the validation to Warning, but it's actually Failed, and the reporting of data is all mixed up:

Available in 1.0.4 (v1)


Additional Keyboard shortcuts (primarily for lenses)

It would be nice to add keyboard shortcuts for all the new lenses in this mod. For example, I like to use the Resource lens a lot, so would like to be able to assign '4' as a keyboard shortcut for it. Adding this configuration option to the base game's 'Options/Key Bindings' menu would be great.

I started to try and do this myself, but trying it on Linux is just basically a lot of guesswork, often with no success (I modified the 'ui_options_text.xml' and 'inputconfiguration.xml' files to achieve the above image).
In addition, it would be nice to be able to assign a keyboard shortcut for directly adding a 'tack'  to the map. Currently, it's a '3 click' process - 1.) click on the 'Map Tacks' button above the minimap, 2.) Click on the 'Add Tack' button, 3.) Click on the hex to add the Tack. If there was a keyboard shortcut to jump directly to step 3 (effectively combining steps 1 & 2), then that would make the process of planning my Empire a bit 'snappier'.

good call on this - I'll take a look.  I know of at least one other mod that added to the key bindings UI directly (rather than having our own like CQUI does)


New version: Tencent.START version <IP_ADDRESS>72

[X] Have you signed the Contributor License Agreement?
[X] Have you checked that there aren't other open pull requests for the same manifest update/change?

[ ] Have you tested your manifest locally with winget install --manifest <path>?
[X] Does your manifest conform to the 1.2 schema?

Microsoft Reviewers: Open in CodeFlow

@wingetbot run


Rename master branch to main

Per the rest of our org's repos we should make this change.
@v1v what do we need to do CI-wise to do so?

what do we need to do CI-wise to do so?

Search for any master references and then create a PR with the change. Change the default branch to main.
In addition, it's required to open a PR in the infra repo to change the jjbb references, and maybe also in the docs repo to change the branches, similar to -> https://github.com/elastic/docs/pull/2323
I'd say, if the first batch of changes work fine, then I'll proceed with another second batch of changes. So synthetics could be in the second batch or maybe in the third one.
There is no rush, though, if you agree we can move forward, I'm happy to give a go soon.

This has been done.  @v1v is there anything else to do here, or can we close this?


Strange characters on messages

Greetings!
I'm getting this behaviour:
PS C:\temp\NeoUdeler> python main.py
Search Keyword (input "all" to list up your subscribed courses): all
←[33m[3585042]←[39m     Master Agile Estimation: Top Techniques for Scrum & Kanban
←[33m[0]←[39m   Cancel to download
Select ←[33m[COURSE ID]←[39m to download: 3585042
Do you download ←[33m"Master Agile Estimation: Top Techniques for Scrum & Kanban"←[39m, are you sure? [y/N]: y
It is related to colorama module, it is missing one import and initialize the module, as stated in
https://linuxhint.com/colorama-python/
After fixing it, result was:
PS C:\temp\NeoUdeler> python main.py
Search Keyword (input "all" to list up your subscribed courses): all
[3585042]       Master Agile Estimation: Top Techniques for Scrum & Kanban
[0]     Cancel to download
Select [COURSE ID] to download: 3585042
Do you download "Master Agile Estimation: Top Techniques for Scrum & Kanban", are you sure? [y/N]: y
I'm using windows 10 with python 3.11.4
Thank you!

@11ange
Hello!
I am using the colorama module for terminal coloring, but it should already be imported in the main.py script.
In my environment, the terminal is displayed as shown in the attached image.

I browsed the website you cited. Can you be more specific about the modifications you made?

Alternatively, as I am a MacOS user, it may be difficult to replicate the behaviour of Windows PowerShell.

Sure! This is the code on the site:
#Import required modules
import colorama
from colorama import Fore, Back, Style
#Initialize colorama
colorama.init(autoreset=True)
#Print text using background and font colors
print(Back.RED + Fore.BLUE + "Welcome to LinuxHint")
#Add newline
print()
#Print text using background color
print(Back.GREEN + "I like programming")
And I just included on main.py (without any concern if it positioned correctly, I'm not really a python user):
import sys
import colorama
import dotenv
from colorama import Fore
from neoudeler import UdemyDownloader
#Initialize colorama
colorama.init(autoreset=True)

@11ange
Thank you. Your code might be specific to the behavior in Windows Powershell, but from colorama import Fore should already be in main.py, so it is still puzzling to me why that issue occurred.
However, to avoid similar issues, I have also made the code additions that you performed to fix the problem in the project.
These issues are essentially due to running the python script directly, and creating an executable file (.exe file) might be more preferable.
I will take this as a task for the future.
Thank you for your report. I hope you can successfully download the courses.

Great! I appreciate your efforts!
After correcting those points I discovered that the course I was trying to download is DRM protected... So, I'll try another one.
Thank you!


Update authors.yml

Updating authors.
@RomanVolkov -- what's your twitter handle?
cc @garricn and @RomanVolkov

@jessesquires here it's http://twitter.com/volkovre


Is this DeploymentFramework project dead for BizTalk 2020?

Just wondering since there's little to none options for setting up deployment on-premise

v5.8 RC1 is stable and supports BizTalk Server 2020.
There is currently no supported Visual Studio extension for VS 2019, so this is the best available workaround.


update solution and rubify

instance variable for length to remove multiple parameter passing
formula to calculate value to compare instead of hard-coding 15 so that it works for all squares of nxn.

Sorry about the previous PR! @FreddieV4

Hey @causztic awesome job. No worries about the previous PR! I probably should have double checked that the solution & challenge matched up 100%; this looks like a good solution.


fix: reduce memory and cpu usage while running background queue

part of: https://github.com/customerio/issues/issues/10950
While profiling the SDK, I noticed that the QueueRunner was using a lot of resources while the BQ executed. I found that for every background task item that was executed, a lot of new dependencies were being created.
This PR implements 1 performance improvement by re-using some dependencies while the BQ executes.

Asking for a re-review because of an addition to the implementation to avoid an edge case that could occur.

PR implementation changed so I asked for a re-review.
I updated the PR description talking about it.

@Shahroz16, 100% correct. I couldn't have put it better myself.
Communication between modules is valuable and does help us keep our SDKs lightweight and modular. There is value in it. However, I no longer believe that a QueueRunner is the best use of the hooks feature. Originally, separate QueueRunners were to try and keep the SDK extra slim by putting specific Track API calls into modules that use them. But, doing that has added more complexity then what it's worth.
Also, from internal discussions we have had about splitting our Common module into more modules, we have mentioned having a module that contains all HTTP code for communicating with the Track API. So, I see a day where all CIO Track API code is in 1 module. Therefore, another reason to not spread Track API code across modules.


Windows support

When? And what is the problem that does not make released with all others?

We fixed an issue with the login command on Windows. Could you upgrade v0.8.5 and let me know how things work?

I'm going to close this issue, but feel free to open a new one with any specific issues using the CLI on Windows.


CommunityToolkit.Mvvm NuGet package reference is missing in the project available in the Finish folder (main branch)

Cloned the repository and opened the solution available in the Finish folder (main branch).
Ended up with errors when compiled, upon analysis found that the CommunityToolkit.Mvvm NuGet package reference is missing in the project file causing the issues.
In fact, it's not added as a reference in any of the projects available in other folders.
Notify to @jamesmontemagno @jfversluis

It is added in the Diretory.build.targets.
https://github.com/dotnet-presentations/dotnet-maui-workshop/blob/main/Directory.build.targets this is found in the Solution Items when you open it in visual studio 2022

Detailed screenshot. Only the Properties Explorer reveals that the files are not physically present.

Those files are up a level in the root.

Yeah, copied those files.
If VS reported missing solution items, then would've checked for them.
The icons representation is quite misleading.
Have reported it to VS team.
Thanks for the response.


Balancer Stable Pool part 4: Event Handling and Pool Registry

Following #1006 and part of #733
We introduce the stable pool creation event listener and adapt the event handling and PoolRegistry to store stable pools along side weighted in the previously introduced enum RegisteredPool.
Test Plan
TBD...

Codecov Report

Merging #1007 (2b4196e) into stable-pool-info-fetching (2e88921) will increase coverage by 0.20%.
The diff coverage is 77.77%.

@@                      Coverage Diff                      @@
##           stable-pool-info-fetching    #1007      +/-   ##
=============================================================
+ Coverage                      64.98%   65.19%   +0.20%
=============================================================
  Files                            122      122
  Lines                          22908    23311     +403
=============================================================
+ Hits                           14887    15197     +310
- Misses                          8021     8114      +93

So I have worked on removing this try_into stuff by simply keeping the weighted and stable pools separated. This also resulted in removing the enum RegisteredPoolentirely and does make things quite a bit nicer.
It remains to deduplicate some of the code blocks for stable and weighted pools (hopefully) using generics.

I still feel like these changes don't address the issue I had brought up here: #1007 (comment)
However, I don't want to block merging this, and we can revist refactoring how we do Balancer liquidity once we have stable pool support.

I will make a follow up PR on this specifically.


CORS Support?

@nathanclevenger I got a question over email about whether count.do is supposed to support CORS for browser-based fetches. I know that we've been directing people to write server-based applications so that CORS restrictions don't apply, but is that the same advice for .do domains?

@Nathan @Samuel is this resolved?
Bryant Skarda
Co-founder, CEO
LinkedIn ( https://www.linkedin.com/in/bryantskarda/ ) | Twitter ( https://twitter.com/BryantSkarda )
On Sun, Jun 2 2024 at 10:37 PM, sam-lippert < @.*** > wrote:

Assigned #5 ( https://github.com/drivly/count.do/issues/5 ) to @bryant22 (
https://github.com/bryant22 ).
—
Reply to this email directly, view it on GitHub (
https://github.com/drivly/count.do/issues/5#event-13014018118 ) , or unsubscribe
(
https://github.com/notifications/unsubscribe-auth/AT3SDHEQMP7LWGC4VXHUVL3ZFPJHPAVCNFSM6AAAAABIVTVZCCVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJTGAYTIMBRHAYTCOA
).
You are receiving this because you were assigned. Message ID: <drivly/count
. do/issue/5/issue_event/13014018118 @ github. com>


build is failing on 'Parsing error: The keyword 'export' is reserved'

I am trying to build the application but it fails on yarn bundle with following error.
I't tried the master and the v0.1.1-alpha.4 tag - same result.
Please advise.
this is my environment:
user@5490f1f4b285:/home/user$ yarn --version
1.22.1
user@5490f1f4b285:/home/user$ npm --version
6.14.4
user@5490f1f4b285:/home/user$ node --version
v12.16.2
user@5490f1f4b285:/home/user$ uname -a
Linux 5490f1f4b285 4.15.0-96-generic #97-Ubuntu SMP Wed Apr 1 03:25:46 UTC 2020 x86_64 GNU/Linux
user@5490f1f4b285:/home/user$ git status
On branch v0.1.1-alpha.4

and here is the failure
> yarn install
....
OK
> yarn bundle
....
lerna success run Ran npm script 'build' in 8 packages in 220.2s:
lerna success - @backstage/cli
lerna success - @backstage/core
lerna success - @backstage/test-utils-core
lerna success - @backstage/test-utils
lerna success - @backstage/theme
lerna success - @backstage/plugin-home-page
lerna success - @backstage/plugin-lighthouse
lerna success - @backstage/plugin-welcome
$ backstage-cli app:build
Creating an optimized production build...
Failed to compile.

./src/plugins.ts
  Line 16:1:  Parsing error: The keyword 'export' is reserved

Command 'react-scripts' exited with code 1

error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed.
Exit code: 1
Command: /usr/local/bin/node
Arguments: /src/.yarn/releases/yarn-1.22.1.js bundle
Directory: /src/packages/app
Output:

info Visit https://yarnpkg.com/en/docs/cli/workspace for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

with --verbose
lerna success - @backstage/plugin-welcome
$ backstage-cli app:build
Creating an optimized production build...
Failed to compile.

./src/plugins.ts
  Line 16:1:  Parsing error: The keyword 'export' is reserved

Command 'react-scripts' exited with code 1

error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed.
Exit code: 1
Command: /usr/local/bin/node
Arguments: /src/.yarn/releases/yarn-1.22.1.js bundle
Directory: /src/packages/app
Output:

info Visit https://yarnpkg.com/en/docs/cli/workspace for documentation about this command.
verbose 92.322723954 Error: Command failed with exit code 1.
    at /src/.yarn/releases/yarn-1.22.1.js:42956:38
    at Generator.throw (<anonymous>)
    at step (/src/.yarn/releases/yarn-1.22.1.js:543:36)
    at /src/.yarn/releases/yarn-1.22.1.js:558:21
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
user@5490f1f4b285:/home/user/packages$

I've just deleted entire folder, cloned the repo and run everything from scratch, and it is not reproduced anymore. probably some glitch on my environment.


INFINITY-2204 Exercise new upgrade flows

Includes a rename workaround: Cosmos doesn't support upgrades across package names, which causes problems for our "beta-pkg" => "pkg" flow.
To get around this, the test will perform surgery on the Scheduler's labels to make it appear to be from the target package.

Rather than depending on this name hack, I'm going to update the PR to just use beta-foo for our in-development package names.

Breaking out tests for just hello-world in a separate PR. This one will come later with wider changes.

@nickbp
"Cosmos doesn't support upgrades across package names, which causes problems for our "beta-pkg" => "pkg" flow."
$ docs kafka update start --options=options.json --package-version=2.0.0-<IP_ADDRESS>
So this will not work anymore?

The name of the CLI itself doesn't matter as long as the --name=x parameter is correct. The command just triggers an RPC to cosmos containing the name of the service to be updated, not the name of the CLI itself. Technically you could do a beta-cassandra update using a beta-kafka CLI.


Transparrent background before the soft keyboard appears

It is not an issue but rather a help entry.
Use case: when the soft keyboard is opened programmatically, navigating to the next activity, there is a short period when the home screen of Android (desktop screen) is visible.
If someone experiences the same problem, I have solved it by adding a listener to configs, and changing the window color.

creating config
SlidrConfig config = new SlidrConfig.Builder()
.listener(new SlidrListener() {
private float mPercent;
 @Override
 public void onSlideStateChanged(int state) {
 	// means that NavigatingBack, changing the color leads to a flash
 	if (mPercent == 0 && state == ViewDragHelper.STATE_IDLE) {
 		return;
 	}
 	ActivityUtils.setAppTheme(BaseActivity.this, state);
 }

 @Override
 public void onSlideChange(float percent) {
 	mPercent = percent;
 }

 @Override
 public void onSlideOpened() {}

 @Override
 	public void onSlideClosed() {}
 }).build();

ActivityUtils.setAppTheme(BaseActivity.this, state) implementation
public static void setAppTheme(Activity activity, int state) {
// If any color is set in STATE_SETTLING, it leads to a flash
if (state == ViewDragHelper.STATE_SETTLING) {
return;
}
final @ColorRes int color = getWindowColor(state);
ColorDrawable colorDrawable = new ColorDrawable(ContextCompat.getColor(activity, color));
activity.getWindow().setBackgroundDrawable(colorDrawable);
}

getWindowColor(state) implementation
@ColorRes
private static int getWindowColor(int state) {
switch (state) {
case ViewDragHelper.STATE_IDLE:
return R.color.colorMaterialLight;
case ViewDragHelper.STATE_DRAGGING:
return android.R.color.transparent;
default:
String errorMessage = String.format(Locale.getDefault(), "Cannot resolve WindowColor for state: %d", state);
throw new IllegalArgumentException(errorMessage);
}
}

I hope it is helpful, but if someone finds a better solution, please share :)

@MihailsKuzmins, thanks for your solution, but I think it would be better if we set background of decorView rather than setting background of window directly, so I replaced this:
activity.getWindow().setBackgroundDrawable(colorDrawable);
to this:
activity.getWindow().getDecorView().setBackground(colorDrawable);
so that we could use this method on the onCreate method of Activity.

@Rhtyme interesting idea. Thank you for sharing. I'm not such an Android expert to judge which solution is a better one. Under the hood getWindow().setBackgroundDrawable(colorDrawable) calls this bunch of code.

They call some other methods rather than simply setting the background. But if you solution works as well, I cannot see a reason why not using it :)


how I use redis-7.* in redismod

How I use redis-7.* in redismod
Is there any way to upgrade the redis version?

i created one for myself you can find it docker pull woss/redismod:7.2.4


feat(list-editor): reposition a child node amongst its siblings (DSP-1340)

resolves DSP-1340

I discovered I was using quite an old version of npm which was creating conflicting package-lock.json files. Should be fixed now.

@kilchenmann This PR adds these two buttons to the "action bubble" and the up and down arrows hide if the node cannot move in that direction. I did not implement it as drag and drop for version 1. Explained in more detail here.


chore: Add connection metadata for incoming connection from GCS and fix typo in v2/main.tf

This PR makes below changes,

Fixes typo in v2/main.tf for creating GCS volume
Adds new connection metadata coming from GCS
Adds new connection metadata coming from Mysql (similar to existing postgresql module)

/gcbrun

/gcbrun


Bom versioning question

1st: cool that there is a BOM now
2nd: Question: what is your versioning idea? coupling the extension versions to the axon-framework release versions seems a bit strange to me ... when 4.5 main release is out and we adopt for example extension-kotlin to support new features of 4.5 this will result in a new version of extension-kotlin and thus will require a new bom release ...
Possible Solutions:

provide two boms: one just containing the "main" axon-framework components released under 4.5 (or 4.4.7, ...) and one that is released more frequently, referring to the first one and updating extension versions
a versioning scheme that contains the main framework version but supports an additional version counter to reflect intermediate changes extensions, which could either be a four digit version (<IP_ADDRESS>) or a suffixed version (1.2.3-A4.4.6) where the part after the "-" is the main framework version and the part before the continuing  version of the bom artifact.

Did you consider this? What is the plan, what's your opinion?

Hi Jan,
as a matter of fact that is the exact issue we are discussing internally currently and the main reason why the BOM is not released on Maven Central yet. I'm personally in favour of something similar to the second option you propose but it has its drawbacks.
As for the plan, I think we'll need a bit more time to consider all options and decide what will be the best (taking into account both usage convenience and release effort) approach. Of course your feedback is much appreciated. Should you have any more ideas or concerns please do not hesitate to share them.

Hi Jan,
as a matter of fact that is the exact issue we are discussing internally currently and the main reason why the BOM is not released on Maven Central yet. I'm personally in favour of something similar to the second option you propose but it has its drawbacks.
As for the plan, I think we'll need a bit more time to consider all options and decide what will be the best (taking into account both usage convenience and release effort) approach. Of course your feedback is much appreciated. Should you have any more ideas or concerns please do not hesitate to share them.

Please have a look at: https://github.com/AxonFramework/AxonFramework/pull/1707
I would prefer it over this additional repository.
This repository could still provide a bom that is more like the spring-boot-dependencies pom ... so it will reference the axon-framework-bom but also extensions, 3rd party libs and so on ...
but https://github.com/AxonFramework/AxonFramework/issues/1200 will be fixed by the PR ..

Can't wait to get it from central...

Closing this issue, as it's a question.
By the way, currently working on getting this guy out there so it can be used by everybody :-)


Agent NodeInfoWorker crash on container die event

Presumably happens when a container dies before the agent has published its node stats for the first time.
kontena-agent | E, [2017-01-31T14:06:09.870981 #1] ERROR -- : Actor crashed!
kontena-agent | NoMethodError: undefined method `+' for nil:NilClass
kontena-agent | 	/app/lib/kontena/workers/node_info_worker.rb:109:in `on_container_event'
kontena-agent | 	/usr/lib/ruby/gems/2.3.0/gems/celluloid-0.17.3/lib/celluloid/calls.rb:28:in `public_send'
kontena-agent | 	/usr/lib/ruby/gems/2.3.0/gems/celluloid-0.17.3/lib/celluloid/calls.rb:28:in `dispatch'
kontena-agent | 	/usr/lib/ruby/gems/2.3.0/gems/celluloid-0.17.3/lib/celluloid/call/async.rb:7:in `dispatch'
kontena-agent | 	/usr/lib/ruby/gems/2.3.0/gems/celluloid-0.17.3/lib/celluloid/cell.rb:50:in `block in dispatch'
kontena-agent | 	/usr/lib/ruby/gems/2.3.0/gems/celluloid-0.17.3/lib/celluloid/cell.rb:76:in `block in task'
kontena-agent | 	/usr/lib/ruby/gems/2.3.0/gems/celluloid-0.17.3/lib/celluloid/actor.rb:339:in `block in task'
kontena-agent | 	/usr/lib/ruby/gems/2.3.0/gems/celluloid-0.17.3/lib/celluloid/task.rb:44:in `block in initialize'
kontena-agent | 	/usr/lib/ruby/gems/2.3.0/gems/celluloid-0.17.3/lib/celluloid/task/fibered.rb:14:in `block in create'

ping @jakolehm re  #1589


Support to show icons in more places (tabs, open editors, quick open)

VSCode Version: 1.51

It would be great if VScode supported an option to show the icons in open tabs and editors.

The release notes does talk about eventually supporting icons for tabs.
"Open editors" here refer to the opened editors as opposed to open editors section on the side bar in #11287

Icons are now showing in these additional locations once you pick an icon theme:

open editors view
tabs
title in editor when tabs are disabled
search results
problems
quick open / editor pickers

I wish we were able to configure its behavior on where icons can show up.

@infinnie where do you want to not see icons?

@bpasero wherever it seems too visually cluttered.

@infinnie like?

@bpasero I think what @infinnie is trying to say is that it'd be nice to be able to turn the icons on or off in each location based on one or more config entries.
That is, if I run a small screen with a lot of tabs for example, maybe I'm not keen on showing the icons on the tabs themselves.
I'm not sure how hard that would be to add support for a config entry per-icon-location (or maybe just one config entry that takes an array/dict/etc.), but I believe that's what he's suggesting.

@joshsleeper BTW I am female. What a pity it is for GitHub not to be able to let us config and display our sexes (or genders?)

@infinnie And I've been trying so hard to be more gender neutral with my comments too! Bah on me, super sorry about that, I promise it wasn't ill intended.

@joshsleeper for that reason there is a setting to disable icons in tabs. that is the only setting we provide for now. I am still trying to find out where else @infinnie would want to configure this.

@bpasero That would be fine!

Ok then it looks like we are good.

Glad to see this setting! It wasn't immediately clear to me what the setting ".showIcons" actually did, even after reading the comment above it. Maybe "workbench.editor.showTabIcons" would be better.

@jtlowe no, the icons also show up when tabs are disabled

@bpasero I see what you're saying. My last thought would be something like: "Controls if opened editors should show with an icon next to their title or not." I don't know if this would have helped me initially though. I need to improve on my vscode terminology. Thanks!


Request for Adding Monitoring Flag to Enable Monitoring on VM Create

Preconditions

[X] No need to upgrade Python SDK or the Python SDK is ready.

Related command
az vm create --enable-azure-monitor-metrics
Resource Provider
Microsoft.Compute/VirtualMachines, Microsoft.Compute/VirtualMachinesScaleSets
Description of Feature or Work Requested
The Azure Monitor team, in partnership with the HPC team is aiming to deliver a comprehensive [Observability solution] for AI/ML workloads running on HPC virtual machines and virtual machine scale sets. I am reaching out to get some assistance with updating VM/VMSS create CLI with a --enable-azure-monitor-metrics flag which will provide customers more flexibility for enabling this monitoring solution on VM/VMSS resources.
I have created a one pager doc(see below) that summarizes our ask, please can you provide with some guidance on next steps? Looking forward to your response!
[ Enabling monitoring using Azure CLI and Powershell Spec Draft.docx]
Minimum API Version Required
N/A
Swagger PR link / SDK link
N/A
Request Example
No response
Target Date
2025-03-01
PM Contact
dakpan
Engineer Contact
ananilku
Additional context
 Enabling monitoring using Azure CLI and Powershell Spec Draft.docx

Thank you for opening this issue, we will look into it.

@Daramfon10 May I ask if this feature request is related to security? Please note that currently, due to the CLI team being busy with the security wave and prioritizing security in this semester, so all the feature requests unrelated to security will be postponed. cc PM @Jacekey23

Hello @zhoxing-ms - Thank you for the response, this is not related to security.


Autoselect current season/episode in game creation modal

@hndrewaall I'll work on this. I think I lack the privilege to assign myself to it.


upgrade solid eslint plugin & upgrade eslint

#28  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @szamanr and the rest of your teammates on  Graphite

Merge activity

Apr 29, 9:40 AM GMT+2: @szamanr merged this pull request with Graphite.


missing copyright statement?

I could not find copyright statement anywhere in this project...
Could you document Copyright please?
Thanks.
https://github.com/thecodeteam/goscaleio/blob/master/LICENSE#L189

This repo is no longer maintained and I am closing out issues before archiving it.
The project is now maintained by Dell at https://github.com/dell/goscaleio


Choice?

I am completely sold on this revolutionary method of versioning software and intend to use it immediately in everything.
I just have one slight issue though: what flavour of set theory are we using here? It's all fine and dandy as long as our versions are finite sets, but I can imagine some Enterprises needing infinite version sets quite quickly and then we need to agree exactly what axioms we're using.
In order to keep this issue focused, are we accepting the Axiom of Choice in SetVer or not?
Once that is settled, then we will probably need to deal with issues about constructivism but I'll leave that for later...

Thanks for your consideration, and more importantly, thanks for fully embracing SetVer!
We've had a meeting with the Council about this, and there were a number of differing opinions. While there were a few council members that advocated for dropping the axiom of infinity and "sticking to sets that actually exist," they have since been terminated. The overall concensus is that ZF is "radical" and "tubuler [sic]".
Choice remains controversial. As a compromise, we've decided to standardize ZF in the next version of SetVer, and to declare the use of the axiom undefined behavior. Later versions may or may not change this political stance.

Version {{}} now addresses this!

Choice is automatically true in SetVer, because SetVer only consists of hereditarily finite sets. You can't write down an infinite set as a finite string of { and }'s. So it is not a model of ZF, but a model of ZF - infinite (+ ¬infinity). This contradicts with the changes introduced in {{}}, which claims that SetVer is to a model of ZF.

Choice is automatically true in SetVer, because SetVer only consists of hereditarily finite sets. You can't write down an infinite set as a finite string of { and }'s. So it is not a model of ZF, but a model of ZF - infinite (+ ¬infinity). This contradicts with the change introduced in {{}}, which claims that SetVer is a model of ZF.

Thank you for contacting SetVer customer support. We will process your inquiry and report back in                business days.


Fix LGTM and flake8 warnings

There were some annoying LGTM and flake8 warnings being reported for this project.  I realized they were all from one python file, so I went through and fixed them.

This pull request fixes 7 alerts when merging 6ddedead9d4ae8cea721ffdf72706e6e2ab4929c into 47dbb932d724cb3d7249cf85dfdd2dd3f01257c2 - view on LGTM.com
fixed alerts:

5 for Unused import
1 for Testing equality to None
1 for Unnecessary 'else' clause in loop

Comment posted by LGTM.com

This pull request fixes 8 alerts when merging dd1fd9fb16038eca248746070a4d5a224f22d938 into 47dbb932d724cb3d7249cf85dfdd2dd3f01257c2 - view on LGTM.com
fixed alerts:

5 for Unused import
2 for Testing equality to None
1 for Unnecessary 'else' clause in loop

Comment posted by LGTM.com


Publish in JOSS

Karthik Ram describes the streamlined submission process for Journal of Open Source Software. Briefly, what's needed is:

[ ] A short Markdown paper paper.md with a title, summary, author names, affiliations, and key references. See this example.
[ ] A metadata file paper.json describing the software. This script automates the generation of this metadata.

Consider publishing instead in an AI or ML journal.


Alt-Enter but command not sent...

Hi my Atom only install ionide and key binding resolver panel shows the alt-enter is FSI:Send-Selection.
But the selected fsharp code didn't been sent to fsi panel...

i have the same problem...

Should be fixed in ionide-fsi 1.1.4

Having same issue with 1.1.4.  It will open FSI panel but the FSI Send commands fail silently.  Any debugging tips for how I could help?

As a temporary workaround, start FSI first using FSI: Open command. Send selection should work without any problems after FSI is started

FSI: Open opens 2 FSI windows.  Subsequent FSI: Send operations don't appear to do anything

Uhh... can't reproduce this behavior. Do you have any output in FSI Window (standard FSI welcoming text) ?

yep, I get the fsi welcome text but nothing else.  The correct fsi path is in the settings.  Anything else I can do to get hints?

I see the exact same thing as reidev275. I am running on OSX and version 1.1.4 of Ionide. Please let me know if there is anything I can add to help pinpoint or resolve the issue.

@Krzysztof-Cieslak This is resolved for me this morning.  Perhaps I had something that didn't shut down properly last week?  Not sure but glad it's working


fix(costmodel): fit and finish text updates

bugfix #1304

Codecov Report

Merging #1310 into master will decrease coverage by 0.06%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           master   #1310      +/-   ##
=========================================
- Coverage   75.46%   75.4%   -0.07%
=========================================
  Files         226     227       +1
  Lines        4540    4561      +21
  Branches      872     879       +7
=========================================
+ Hits         3426    3439      +13
- Misses        984     992       +8
  Partials      130     130

Impacted Files
Coverage Δ

src/utils/getItemLabel.ts
45.45% <0%> (ø)

src/utils/getComputedAwsReportItems.ts
91.83% <0%> (+0.34%)
:arrow_up:

src/utils/getComputedAzureReportItems.ts
89.36% <0%> (+0.47%)
:arrow_up:

src/utils/getComputedOcpCloudReportItems.ts
23.33% <0%> (+0.52%)
:arrow_up:

src/utils/getComputedOcpReportItems.ts
68.42% <0%> (+1.75%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6f1ebb4...9abd6ca. Read the comment docs.


Update Boost python rules

Firs commit creates a libboost-python-dev key and copies the content of libboost-python into it
The second updates libboost-python to not include all the development headers. Note that this is changing the rules of the key existing before this PR
Note: on debian/ubuntu: libboost-python is a hybrid package providing libraries for both python2 and python3. It's unclear to me what rule to use for system that have different packages for python2 and python3 e.g. fedora. So I opted to both to be the closest to ubuntu/debian behavior

Note that this is changing the rules of the key existing before this PR

It would be good to check how packages use this key and how likely this change will result in regressions.

It would be good to check how packages use this key and how likely this change will result in regressions.

As far as I can tell it's not used by any released package on any active ROS distribution.
It was added fairly recently: https://github.com/ros/rosdistro/pull/21570
Maybe @JWhitleyWork can provide more context on their use of this key

@mikaelarguedas / @dirk-thomas It's currently used by https://github.com/fzi-forschungszentrum-informatik/Lanelet2. This is a pretty major part of the Autoware project and it would be a breaking change. Would one of you mind coordinating with the maintainers of that repo to implement the change at the same time and keep me in the loop? We will need to update our reference to that repo to keep from breaking our builds once the change is in place.

Sounds good. Do you happen to know of any other repo than that one using it ?
To be fair, it's unlikely to break as the majority of ROS packages bring in the boost headers (all the way down to ros_comm). For example lanelet_python depends directly on lanelet_core that <depend>boost</depend> so it pulls libboost-all-dev.
I opened https://github.com/fzi-forschungszentrum-informatik/Lanelet2/pull/88 for correctness


feat(option-labels): add custom option labels

Description
Adds prop to pass in a map of methods to option labels.
Contribution guidelines
For contribution guidelines, styleguide, and other helpful information please
see the CONTRIBUTING.md file in the root of this project.

:rocket: PR was released in v2.25.1 :rocket:


Import users from OpenLDAP - Unable to locate user

Laravel Version: 5.8.4
Adldap2-Laravel Version: 6.0.5
PHP Version: 7.2.2
LDAP Type: OpenLDAP

Description:
I'm trying to import users from OpenLDAP using the command php artisan adldap:import.
I can find the users on the OpenLDAP server, but nothing has been imported or syncronized.

In the log i found this error:
local.ERROR: Unable to import user Gabriele Bellucci. Unable to locate a user without a Email
In the ldap_auth.php i have this configuration for the ldap identifiers:
'ldap' => [             'locate_users_by' => 'Email',             'bind_users_by' => 'distinguishedname',         ]
I have an email field for the use in the OpenLDAP server, so i can't understand what i'am doing wrong.

Can someone help me?
Thanks

Hi @bellu,
I the Email attribute you're showing is actually named something else behind the scenes. Usually it is mail (lowercase).
I would try dumping your users attributes and seeing what you find. Try this:
use Adldap\Laravel\Facades\Adldap;

$user = Adldap::search()->find('gabriele.bullucci@test.com');

dd($user->getAttributes());

Hi @stevebauman, thanks for the quick reply.
The name of the attribute is mail and not Email, thanks. Now the import is working fine!

Awesome, no problem @bellu! Glad you were able to resolve it :smile:


Python API:

As a robot user, I would like the pipette to hold its z-position between two actions in the same well, so that it can complete protocols more efficiently.
Acceptance Criteria

If robot is performing two sequential actions in the same well, the robot should maintain its z-position (i.e., not raise the pipette out of the well) between the actions.

-- Example: The pipette should remain in the same position when executing the following command, and not move between mixing (one aspirate and dispense) and aspirating its transfer volume:
    pipette.transfer(
        100,
        plate.wells('A1'),
        plate.wells('A2'),
        mix_before=(1, 50))

As an exception case, the robot should move (raise out of the well) if creating an air gap before or after an aspirate.

@btmorr
The above should handle Kristin's request for an 'aspirate with prewet' command.
Worth thinking about eventually creating a 'prewet' alias for 'mix_before' (defaulted to a single aspirate/dispense cycle).

Issue closed by #839 (please re-open if I'm wrong here)


How metadata is being used during similarity search and query?

I have 3 pdf files in my directory and I "documentized", added metadata, split, embed and store them in pinecone, like this:
loader = DirectoryLoader('data/dir', glob="**/*.pdf", loader_cls=UnstructuredPDFLoader)
data = loader.load()

#I added company names explicitly for now
data[0].metadata["company"]="Apple"
data[1].metadata["company"]="Miscrosoft"
data[2].metadata["company"]="Tesla"

text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=200)
texts = text_splitter.split_documents(data)

embeddings = OpenAIEmbeddings(openai_api_key=OPENAI_API_KEY)

pinecone.init(
    api_key=PINECONE_API_KEY,
    environment=PINECONE_API_ENV
)

metadatas = []
for text in texts:
    metadatas.append({
        "company": text.metadata["company"]
    })

Pinecone.from_texts([t.page_content for t in texts], embeddings, index_name=index_name, metadatas=metadatas)

I want to build a Q&A system, so that I will mention a company name in my query and pinecon should look for the documents having company A in the metadata. Here what I have:
pinecone.init(
    api_key=PINECONE_API_KEY,
    environment=PINECONE_API_ENV
)
index_name = "index"
embeddings = OpenAIEmbeddings(openai_api_key=OPENAI_API_KEY)

docsearch = Pinecone.from_existing_index(index_name=index_name, embedding=embeddings)

llm = ChatOpenAI(temperature=0, openai_api_key=OPENAI_API_KEY)
chain = load_qa_chain(llm, chain_type="stuff")

query = "What is the total revenue of Apple?"
docs = docsearch.similarity_search(query, include_metadata=True)

res = chain.run(input_documents=docs, question=query)
print(res)

However, there are still document chunks from non-Apple documents in the output of docs. What am I doing wrong here and how do I utilize the information in metadata both on doc_search and chat-gpt query (If possible)? Thanks

You can pass metadata filter as a dictionary
docs = docsearch.similarity_search(query, {"company":"Apple"})

@egils-mtx For the following code,
docs = docsearch.similarity_search(query, {"url": "langchain.readthedocs.io"})

I am getting the error.
TypeError: '>' not supported between instances of 'dict' and 'int'

I have upgraded the chroma library.

@egils-mtx The thing is I don't know in advance what company will be used in a query. I provided the code above just as an example. Moreover, I don't even know If a query will contain a company name or not. I was expecting the query --> filter_by_metadata type of behavior to happen under the hood, without my intervention. It seems like there is no such a functionality so far.

Hello @mzhadigerov, can you share info. how did you solve your problem?

I am also curious! I dont see any solution to this in the thread. Could you please provide information on how to deal with this?

+1

+1

Check this - https://python.langchain.com/docs/modules/data_connection/retrievers/self_query/
I have not tried it yet but looks like it solves the problem you are facing.

@egils-mtx what's the best way to pass on multiple "values" for metadata?
For example take this metadata key/value pair:
{"category": "game"}
now what if i want to add two categories? With or without comma?
{"category": "game, adult"}
or
{"category": "game adult"}
And can I query them later using "eq" or a similar filter to query for a metadata that contains BOTH game AND Adult?

I used this docs = vectordb.similarity_search(     question,     k=3,     filter={"source":"docs/cs229_lectures/MachineLearning-Lecture03.pdf"} )


Can't get working in GitLab-CE

Don't know if it's a GitLab problem or an omniauth-openid-connect problem, but I wasn't able to get it working, despite being able to get two other omniauth providers working on GitLab-CE just fine (github, google_oauth2), so I can't assume it's a GitLab problem.  Also, it's rather difficult to tell where the problem is.
Anyway, posting it here just in case:
https://gitlab.com/gitlab-org/gitlab-ce/issues/3019

I have managed to get GitLab-CE work with openid-connect after doing the fix for the issue #57.
For the moment the working configuration was with a keycloak.org server and explicit configuration of endpoints. The issuer + discovery didn't worked on a first try.
@odigity Can you check if the fix is also applying for you?

@iconoeugen I m struggling to make gitlab work with keycloak; wondering if you can please share details how you did it? How does your config looks like?


Class variable state leaked/persisted between examples

[!NOTE]
Please let me know if this issue is not created in the correct rspec project

Subject of the issue

Using class variables inside tests can lead to inconsistent behavior. Indeed, modifying a class variable in one test modifies it for everyone. I would expect each example to be independent of each other, and I wouldn't expect adding/modifying/deleting another unrelated example to influence my current test.
From my understanding, we don't clear the memory for class variables whenever we start a new example. That's maybe intended, but I'd be curious to know why we decided to go into that direction.
Do you know if there's a way to actually clean this up after examples?
Thanks for your help!
Your environment

Ruby version: 3.3.1
rspec-core version: 3.13.0

Steps to reproduce

# frozen_string_literal: true

begin
  require 'bundler/inline'
rescue LoadError => e
  warn 'Bundler version 1.10 or later is required. Please update your Bundler'
  raise e
end

gemfile(true) do
  source 'https://rubygems.org'

  gem 'rspec', '3.13.0'
  gem 'rspec-core', '3.13.0'
end

puts "Ruby version is: #{RUBY_VERSION}"
require 'rspec/autorun'

class A
  class << self
    attr_reader :foo

    def bar
      @foo ||= 0
      @foo += 1
    end
  end
end

RSpec.describe 'A' do
  context 'when A.bar is called' do
    before { A.bar }

    it { expect(A.foo).to eq(1) }
  end

  context 'when A.bar is called again' do
    before { A.bar }

    it { expect(A.foo).to eq(1) }
  end
end

Expected behavior

I would expect both examples to pass (even the second one), as I believe it should be independent and isolated from the first one.
Actual behavior

The second example fails, as the class variable @foo is not reset between examples.

Detecting and revering changes on a vast number of class objects in the object space, global variables, constants etc between each example is hard if even possible, error-prone and would have an insane overhead.
If you need isolation, use https://github.com/rspec/rspec-support/blob/8cbf7f9aecbefacd74da16ec2e713358ff72925b/lib/rspec/support/spec/in_sub_process.rb
Check https://github.com/search?q=org%3Arspec in_sub_process&type=code for usage examples.

Thanks for the quick answer! I was indeed expecting this, but wanted to make sure, in case I missed something.
Thanks!

This is an example of global state and we can't make any atttempt to reset this for you but I can suggest some approaches to help test it:
You can add a "reset" to this global state and call it between each spec in a hook, we actually do this ourselves in places.
You could also change how this state is encapsulated and instead of using multiple class variables you have a normal instance of a class which you test, but then have a single "class instance" of this class in reality, you can see an example of this with our "RSpec::World".


make getAllWords public

The method is used inside the ViewModel, it should be public to be accessed.

I signed it!

Since everything is in the same module, there's no need to make the method public.


fix: remove unnecessary code

Codecov Report

Merging #13 (fb125ba) into main (c21938c) will decrease coverage by 0.09%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main      #13      +/-   ##
==========================================
- Coverage   72.13%   72.03%   -0.10%
==========================================
  Files          11       11
  Lines         567      565       -2
==========================================
- Hits          409      407       -2
  Misses        158      158

Impacted Files
Coverage Δ

seqal/query_strategies.py
53.54% <ø> (-0.73%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c21938c...fb125ba. Read the comment docs.


Inline javascript - method does not exists

I'm loading a fixture with a layout that yields a content block :scripts at the bottom. The :scripts content block contains some inline javascript that I set in a partial - something like this:
<% content_for :scripts do %>
<script>
    var goTo = function(path){
        window.location.href = path;
    }

    $('body').on('click', function(){
        goTo("/");
    });
</script>
<% end %>

This works great in my app but when I run my tests the goTo function does not exist, however, the functions in application.js are all there. How can I fix this?

You could render :foo, layout: true to render the layout with that particular template. You might also be able to pull the the script tag into a partial and load both, ie MagicLamp.load("foos/show", "layouts/script").
But, I'd recommend that you avoid using the content_for script blocks and instead use something like Rails JSKit so you can keep all of your JavaScript in your application.js.

Let me know if that helps.

I do load the layout with the template and in the inspector I can see the javascript but when I call those functions they are not defined.  What's weird is all of the functions in application.js are defined.

Ah I see that application.js is being included by the actual test runner page. So it seems that I can only load html in a fixture.

It might be worth it to just stub your goTo function since its execution will likely disrupt your tests anyway (it certainly would if you're using Teaspoon as your test runner).

@malissa is MagicLamp still giving you any trouble?

I just refactored my javascript to get around it :) thanks!


Implement Data Manager Methods

Fill out the methods for calling the Fitbit API in the Data Manager

We should add methods for data like heart rate, sleep data, weight, and various goals for a start. If doable, we can also implement methods for data like food and water data.
See the Fitbit client for specifications on how to use the client:
https://python-fitbit.readthedocs.io/en/latest/


fix memory leak in managed connection

there is a go routine leak if disconnect is invoked on a managed connection that is not connected.
func printEvents(wg *sync.WaitGroup, id int, t *slack.RTM) {
	defer wg.Done()
	for msg := range t.IncomingEvents {
		log.Printf("id(%d): %+v", id, msg.Data)
		switch msg.Data.(type) {
		case *slack.DisconnectedEvent:
			return
		default:
		}
		time.Sleep(time.Second)
	}
}

func main() {
	wg := &sync.WaitGroup{}
	for i := 0; i < 10; i++ {
		wg.Add(1)
		c := slack.New("token")
		// c.SetDebug(true)
		rtm := c.NewRTMWithOptions(&slack.RTMOptions{})
		go rtm.ManageConnection()
		if err := rtm.Disconnect(); err != nil {
			log.Println("disconnect error", err)
		}
		log.Println("disconnect completed")
		go printEvents(wg, i, rtm)
	}
	wg.Wait()
}

@catherinetcai, @dvdplm mind take a look at this if you get a chance.

LGTM and good catch. Out of curiosity, did you detect a leak in production and set out to find it or how did you get to this?

leak in production unfortunately. I noticed it a long time ago actually then forgot about it because we were deploying often enough to that subsystem it wasn't having any real impact.

…because we were deploying often enough…

Ha! Been there. :)