1.20.0-SNAPSHOT Fails on mongodb cluster query

Describe the bug
Drill is not able to query mongodb cluster, it throws a NPE
2021-10-28 16:21:52,535 [1e8531df-4e21-f062-707b-b9cbb72c619e:foreman] INFO  o.a.d.e.s.mongo.MongoStoragePlugin - Created srv protocol connection to [address=mongoclusterDNSisOK:27017, user=USERalsoOK].
2021-10-28 16:21:52,535 [1e8531df-4e21-f062-707b-b9cbb72c619e:foreman] INFO  o.a.d.e.s.mongo.MongoStoragePlugin - Number of open connections 1.
2021-10-28 16:21:53,165 [1e8531df-4e21-f062-707b-b9cbb72c619e:foreman] WARN  o.a.d.e.s.m.s.MongoSchemaFactory - Failure while getting collection names from 'config'. Command failed with error 13 (Unauthorized): 'not authorized on config to execute command ...
2021-10-28 16:21:53,208 [1e8531df-4e21-f062-707b-b9cbb72c619e:foreman] WARN  o.a.d.e.s.m.s.MongoSchemaFactory - Failure while getting collection names from 'local'. Command failed with error 13 (Unauthorized): 'not authorized on local to execute command ...
2021-10-28 16:21:58,711 [1e8531df-4e21-f062-707b-b9cbb72c619e:frag:1:0] INFO  o.a.d.e.w.fragment.FragmentExecutor - 1e8531df-4e21-f062-707b-b9cbb72c619e:1:0: State change requested AWAITING_ALLOCATION --> FINISHED
2021-10-28 16:21:58,711 [1e8531df-4e21-f062-707b-b9cbb72c619e:frag:1:0] INFO  o.a.d.e.w.f.FragmentStatusReporter - 1e8531df-4e21-f062-707b-b9cbb72c619e:1:0: State to report: FINISHED
2021-10-28 16:21:58,711 [drill-executor-6] ERROR o.a.d.exec.server.BootStrapContext - org.apache.drill.exec.work.WorkManager$WorkerBee$1.run() leaked an exception.
java.lang.NullPointerException: null
	at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:347)
	at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
	at java.base/java.lang.Thread.run(Thread.java:832)

The DBs unauthorized are cluster-private stuff, these warning lines are also as DEBUG at 1.18.0 logs but after the query is starting on the requested DB without problems.
To Reproduce
Steps to reproduce the behavior:

Create a mongodb cluster storage
Try a query
Check the logs

Expected behavior
Running the query
Additional context
[version] apache/drill:master-openjdk-14
[DIGEST] 55e6551a8523 (2021/10/28 5:23pm)
[OS] docker

apache drill> show databases;
+-----------------------------+
|         SCHEMA_NAME         |
+-----------------------------+
| cp.default                  |
| dfs.default                 |
| dfs.root                    |
| dfs.tmp                     |
| information_schema          |
| mongo_atlas.admin           |
| mongo_atlas.local           |
| mongo_atlas.myfirstdatabase |
| sys                         |
+-----------------------------+
9 rows selected (0.318 seconds)

apache drill> use mongo_atlas.myfirstdatabase;
+------+---------------------------------------------------------+
|  ok  |                         summary                         |
+------+---------------------------------------------------------+
| true | Default schema changed to [mongo_atlas.myfirstdatabase] |
+------+---------------------------------------------------------+
1 row selected (0.099 seconds)

apache drill (mongo_atlas.myfirstdatabase)> show tables;
+-----------------------------+------------+
|        TABLE_SCHEMA         | TABLE_NAME |
+-----------------------------+------------+
| mongo_atlas.myfirstdatabase | test       |
+-----------------------------+------------+
1 row selected (0.248 seconds)

apache drill (mongo_atlas.myfirstdatabase)> select * from test;
+-----------------------------+-------------------------+
|             _id             |          date           |
+-----------------------------+-------------------------+
| a"j\xC3;)\xFFc\x06\xB3j\xA9 | 2021-08-22 23:18:27.258 |
+-----------------------------+-------------------------+
1 row selected (2.196 seconds)

@javier-nieto-nexplore I cannot reproduce the same set of problems on the cluster (3.6, replica set) and Atlas cloud (4.4, free). I think the difference is Database Access :

User Name
Authentication Method
MongoDB Roles
Resources

test
SCRAM
atlasAdmin@admin
All Resources

Hi @luocooong
Tested with admin user an same result, I would say it could be something about collection size.
I've just done these tests:

silly collection with 1 field -> OK
complex collection (flat document) with 10K docs -> OK
complex collection (flat doc) with 160K docs -> NPE

@javier-nieto-nexplore Thanks for the explanation. Could you please post the log file (as an attachment)? Please mask any privacy information, thanks.

Hi @luocooong
logs are a bit dry :( but I realized that this is related to a config param (planner.slice_target), looks like it has to be greater than collection doc count, not tested in dist mode though

2021-11-03 14:27:40,553 [1e7d63a2-9c2f-c629-d5f1-50c47d2ff3b1:foreman] INFO  o.a.drill.exec.work.foreman.Foreman - Query text for query with id 1e7d63a2-9c2f-c629-d5f1-50c47d2ff3b1 issued by anonymous: select STREET from mongo_local.fake_data_test.frisco
2021-11-03 14:27:41,848 [1e7d63a2-9c2f-c629-d5f1-50c47d2ff3b1:frag:1:0] INFO  o.a.d.e.w.fragment.FragmentExecutor - 1e7d63a2-9c2f-c629-d5f1-50c47d2ff3b1:1:0: State change requested AWAITING_ALLOCATION --> FINISHED
2021-11-03 14:27:41,848 [1e7d63a2-9c2f-c629-d5f1-50c47d2ff3b1:frag:1:0] INFO  o.a.d.e.w.f.FragmentStatusReporter - 1e7d63a2-9c2f-c629-d5f1-50c47d2ff3b1:1:0: State to report: FINISHED
2021-11-03 14:27:41,848 [drill-executor-7] ERROR o.a.d.exec.server.BootStrapContext - org.apache.drill.exec.work.WorkManager$WorkerBee$1.run() leaked an exception.
java.lang.NullPointerException: null
	at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:347)
	at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
	at java.base/java.lang.Thread.run(Thread.java:832)

@javier-nieto-nexplore
It might be good to document this in the Drill docs.

@cgivre  do you think it's the expected behavior?

@javier-nieto-nexplore
I guess my question is does this fix the issue when you change the setting?  If so, then we should document this and tell the user that if you have a collection with more documents than the planner slice target allows, you should increase that.  I am curious as to how running this in distributed mode affects this issue.
IMHO, regardless, we should improve the error message so that if this happens, a user knows how to fix it.

Hi @cgivre fully agree NPE is not a great hint, it took me some time to find a trail
we can note down this as a known issue but for me that's not an 100% acceptable solution, how high should you go with the slice_target to cover your needs i.e.:1M,10M?
also I'm a bit concerned about possible the side effects as we don't know the root cause of the issue.

Hi @luocooong here's some log with several query runs vs. local mongo with dummy data, changing the slice_target parameter made it OK/KO and going through the MongoScanBatchCreator log or not, also the fragment id has some 1:0/0:0
Hope it helps
error_log.txt


MsgPack BUFFER_SHORTAGE

I was looking at the socket.io-redis messages that are going across our Redis instance using the redis-cli and I am seeing packets that seem to be malformed (missing a final byte).  Can someone please help me understand this.
redis-cli
localhost:6379> psubscribe *
Reading messages... (press Ctrl-C to quit)

"psubscribe"
"*"
(integer) 1
"pmessage"
"*"
"socket.io#/#VR6DcrGDTeFCJP2yAAAY#"
"\x93\xa6w9PJlb\x83\xa3nsp\xa1/\xa4data\xa6\xa4type\x02\x82\xa5rooms\x91\xb4FOcyd8f4OtGcAjoVAADq\xa5flags\xc4"

The last item in the message is the "flags" field which says that it is of type \xc4.  If I look at the MessagePack protocol, that says that '\xc4' is for an bin8 field and it expects a least one additional byte (to say it has 0 length).
bin 8 stores a byte array whose length is up to (2^8)-1 bytes:
+--------+--------+========+
|  0xc4  |XXXXXXXX|  data  |
+--------+--------+========+
Processing the message like below produces a BUFFER_SHORTAGE error from msgpack-lite.
out = mpl.decode(message);

message :: socket.io#/#VR6DcrGDTeFCJP2yAAAY#
.../sio-parse/node_modules/msgpack-lite/lib/flex-buffer.js:64
if (end > this.buffer.length) throw new Error(BUFFER_SHORTAGE);
If I add a '\x00' to the message payload, I can decode it successfully.
c = Buffer.from([0]);
d = Buffer.concat([message, c]);
out = mpl.decode(d);

['w9PJlb', {'nsp': '/', 'type': 2, 'data': ''}, {'flags': , 'rooms': ['FOcyd8f4OtGcAjoVAADq']}]
This is different than the straight decode done in the socket.io-redis code.  I am confused by this and was hoping someone could help me.
// from socket.io-redis/index.js (140)
var args = msgpack.decode(msg);
PS - I also have an implementation in Python that needs the same additional '\x00' in order to parse the messages that come from Redis.

Do you know where that packet comes from?
With io.to(<socketid>).emit('data'), the following packet gets sent over Redis pub/sub:
1) "pmessage"
2) "*"
3) "socket.io#/#VAT6-pmu-9yZ-lIWAAAA#"
4) "\x93\xa6tlD92v\x83\xa4type\x02\xa4data\x91\xa4data\xa3nsp\xa1/\x82\xa5rooms\x91\xb4VAT6-pmu-9yZ-lIWAAAA\xa5flags\xc0"

\xc0 being the encoding for undefined (ref)
Which version of msgpack-lite are you using?

Argh.... I forgot to check the versions of everything before starting this.  Sorry about that.
Turns out I am using socket.io-redis 1.0.0 and it uses msgpack-js 0.3.0.
Here is a nice comment from the msgpack-js website:

Also I've added a type for undefined that works just like the null type.
undefined  11000100  0xc4

That seems to be what I am concerned about.  I will upgrade everything and I expect that all will be well.
Its good to know that you can't mix versions because of the change in msgpack libraries.


test, refactor: Use the same shutdown path as in bitcoin-qt binary

Since bitcoin-core/gui#336, the bitcoin-qt shutdown process has been streamlined, and initiating of another event loop is no longer required.
This PR applies the same approach for tests.

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts
Reviewers, this pull request conflicts with the following ones:

#149 (Intro: Have user choose assumevalid by luke-jr)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.


Don't request release notes from closed PRs

Summary
If a PR is already closed, chewbacca should not request release notes, as the PR might be older then our new process for release notes. Blocking new PRs from getting merged is enough to enforce that every new PR gets it's release notes.
Ticket Link
None
Release Note
Don't request release notes from closed PRs

@cpanato Does this change sound reasonable to you?


Improve Member Directory Filter 👥

Issue Reserved for Bloomberg Mentorship Program
Description
On the Directory tab of the Member Profile, we currently have a filter where you can filter through members based on Company, Graduation Year, Hometown, etc. However, to do the filtering you must first press on the filter icon, then select one filter (such as ‘Company’) then select the last filter (such as ‘Amazon’), and finally press ‘Add Filter’. This is a lot of steps, and we’d like to make the interface simpler for a couple of filters. In particular, ‘Company’ and ‘School’ are two filters that particularly relevant, so in addition to the existing filter with other options, we’d like to also have buttons for ‘Company’ and ‘School’ that we can select, see a list of schools/companies, select one of those, and then have the filtered view of members present.

Acceptance Criteria

Create dropdown elements for ‘School’ and ‘Company’ where, when selected, they will list all the schools or companies. Then, when one of the schools/companies is selected, the view should update to
Add these dropdown elements to be aligned with the current filter icon
Remove ‘Company’ and ‘School’ from the existing filter

Additional Context
See how the layout of the directory page is implemented here.
Examine the way the Filter Directory dropdown is implemented in the file here. This will be very useful in understanding how to have a filter that can be applied to modify the other contents on the page (i.e. which members are displayed).

I’m interested in working on this issue :)

Update: Gonna start working on it with my mentor today


"Withstands Explosions" containers dupe items when broken with explosive pickaxe

If you attempt to mine a steel chest, obsidian chest, steel safe or obsidian safe (all say they "withstands explosions") with an explosive pickaxe, the contents are dropped on the ground around the container.  The container is left in tact with the original contents still inside it, thereby duplicating its contents with each attempted mine of the item.
[13:46:32 INFO]: Paper v1_15_R1 (143)
[13:46:32 INFO]:
[13:46:32 INFO]: CS-CoreLib vDEV - 83 (git 8081bb4f)
[13:46:32 INFO]: Slimefun vDEV - 470 (git bfcbda10)
[13:46:32 INFO]:
[13:46:32 INFO]: Installed Addons (14)
[13:46:32 INFO]:  DyedBackpacks vDEV - 4 (git 99945ff4)
[13:46:32 INFO]:  SoundMuffler vDEV - 8 (git b700523d)
[13:46:32 INFO]:  SlimefunOreChunks vDEV - 7 (git 1edd355c)
[13:46:32 INFO]:  HotbarPets vDEV - 26 (git 7489ae00)
[13:46:32 INFO]:  ExtraGear vDEV - 7 (git 9e953653)
[13:46:32 INFO]:  ExoticGarden vDEV - 37 (git 76bd242f)
[13:46:32 INFO]:  Barrels vDEV - 11 (git abecba9c)
[13:46:32 INFO]:  ElectricSpawners vDEV - 9 (git 9cf22acb)
[13:46:32 INFO]:  ExtraHeads vDEV - 12 (git 51e97ace)
[13:46:32 INFO]:  SoulJars vDEV - 12 (git 6b475952)
[13:46:32 INFO]:  MobCapturer vDEV - 7 (git b61d3fbf)
[13:46:32 INFO]:  SlimyTreeTaps vDEV - 9 (git 55c9996f)
[13:46:32 INFO]:  PrivateStorage vDEV - 11 (git b064863b)
[13:46:32 INFO]:  ChestTerminal vDEV - 26 (git 5b0db2b7)

https://github.com/John000708/Barrels/issues/33 Same issue in barrels.
And yeah, I can confirm this. @TheBusyBiscuit could you please fix it? Players are using it to dupe items...

Btw still wasn't fixed, just mentioning that it maybe could be confusing when it was mentioned as fixed.

Btw still wasn't fixed, just mentioning that it maybe could be confusing when it was mentioned as fixed.

Nowhere was this ever mentioned as fixed...
And this issue is still open...

Still the one issue that couldn't get fixed even if it's one of the most important, well.

Still the one issue that couldn't get fixed even if it's one of the most important, well.

Still the one issue that couldn't get fixed even if it's one of the most important, well.

I wish there is an update to this.

Looks like it's finally fixed.

Will this PR https://github.com/Slimefun/Slimefun4/pull/2859 can help this problem?

Verified this is resolved, thanks so much!


Is Dice6 entropy calculated wrong?

Hello, when calculating dice rolls for 100 rolls of the dice it is saying that 100 rolls is only 169 bits of entropy.  However log2(6^100) = 258.  Not 169.

See entropy.js L26-29 and this comment in #435 to understand why a single dice roll is less than log2(6) bits of entropy (whole conversation in 435 is worth reading, the topic definitely evaded my own intuition and is pretty fascinating).

I'm siding with @ZenDragonX  on this one.
Using the DICE entropy string "123456123456" notice the filtered entropy goes to "123450123450".  So ONLY 6 wraps?  If the symbols are {1,2,3,4,5,6} then 6 should be valid.  Perhaps filtered base6 should read "012345012345" but wrapping ONLY 6, while leaving the rest unchanged seems weird to me.
You can spot-check this against : https://github.com/Coldcard/firmware/blob/master/docs/rolls.py to show the "wrapping 6 bug".  If you try "1234512345" on both you get:
^(tool aim drum remain amazing present strike tattoo fox guilt vessel protect salute forward galaxy hidden long chunk humor lumber upset news build behave)
Subtle... but I really do think this is bugged whenever 6 shows up in the entropy string.


[BEAM-10961] enable strict dependency checking for sdks/java/extensions/sorter

Split from #13686
R: @ibzib

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

Run Java PreCommit

Run Java PreCommit


BRANDING: Allow to change title,ico, default motd

Feature Request
Feature
Add ability to change AMP:

title, ico

root@testiniiis:/home/amp/.ampdata/instances# grep -R "Application Management Panel" ADS01 AMPCache-2114.zip MCJAVAEDITION
ADS01/WebRoot/manifest.json:  "name": "AMP - Application Management Panel",
ADS01/WebRoot/AMP.html:    <title>AMP - Application Management Panel</title>
ADS01/Plugins/srcdsModule/WebRoot/manifest.json:  "name": "AMP - Application Management Panel",
ADS01/Plugins/srcdsModule/WebRoot/AMP.html:    <title>AMP - Application Management Panel</title>
MCJAVAEDITION/WebRoot/manifest.json:  "name": "AMP - Application Management Panel",
MCJAVAEDITION/WebRoot/AMP.html:    <title>AMP - Application Management Panel</title>
MCJAVAEDITION/Plugins/srcdsModule/WebRoot/manifest.json:  "name": "AMP - Application Management Panel",
MCJAVAEDITION/Plugins/srcdsModule/WebRoot/AMP.html:    <title>AMP - Application Management Panel</title>

default game server motd for all game server types

root@testiniiis:/home/amp/.ampdata/instances# grep -R "Powered by AMP" ADS01 AMPCache-2114.zip MCJAVAEDITION
ADS01/Plugins/ADSModule/GenericTemplates/valheim.kvp:App.CommandLineVars={"ServerName": "My Valheim Server - Powered by AMP", "WorldName": "PoweredByAMP", "ServerPassword": "12345"}
ADS01/Plugins/ADSModule/GenericTemplates/valheimplus.kvp:App.CommandLineVars={"ServerName": "My Valheim Server - Powered by AMP", "WorldName": "PoweredByAMP", "ServerPassword": "12345"}
MCJAVAEDITION/Plugins/ADSModule/GenericTemplates/valheim.kvp:App.CommandLineVars={"ServerName": "My Valheim Server - Powered by AMP", "WorldName": "PoweredByAMP", "ServerPassword": "12345"}
MCJAVAEDITION/Plugins/ADSModule/GenericTemplates/valheimplus.kvp:App.CommandLineVars={"ServerName": "My Valheim Server - Powered by AMP", "WorldName": "PoweredByAMP", "ServerPassword": "12345"}
MCJAVAEDITION/MinecraftModule.kvp:Minecraft.ServerMOTD=Powered by AMP
MCJAVAEDITION/Minecraft/server.properties:motd=Powered by AMP

There's already branding options available if your license allows it, I can't see why you wouldn't be allowed to change the defaults of game server motd settings or even the WebUI title/icon

Alright, i had a good laugh at this theme and branding.
On Sat, May 22, 2021, 3:01 AM Kyle @.***> wrote:

There's already branding options available if your license allows it, I
can't see why you wouldn't be allowed to change the defaults of game server
motd settings or even the WebUI title/icon
[image: image]
https://user-images.githubusercontent.com/25551312/119217700-c7ed1000-bad3-11eb-9d72-3161aa6782be.png
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/CubeCoders/AMP/issues/462#issuecomment-846365629, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAL67K462EY4UK272PUMFR3TO5JGHANCNFSM45I7KJPA
.

Glad I could manage that.
I'd like to +1 this feature. I'd do it myself but.. yeah

They opened this change request on my advice, there's some extra branding options they need that don't currently exist.

Latest build has an option to change the Title, the manifest changes should be done via the Overlays feature.

Sorry to necro but any chance that these settings can be synchronised across all game instances including targets and controller? If I change the branding on my controller I would expect all instances under its control to adopt the same settings, perhaps with an option to toggle this functionality per instance.
Cheers

That's what the new instance defaults are for :)

That's what the new instance defaults are for :)

In the controller configuration I can't see any options to enable anything under new instance defaults:

I need configurations such as the background image URL to sync to all instances, or at least be parsed to newly created instances:

I just tested creating a new instance and these values were not set under branding. I have branding enabled in both my controller ADS and my target ADS (both configured manually), and yet, newly created instances still have default branding.
What am I missing here? @PhonicUK

The final section of this wiki page : https://github.com/CubeCoders/AMP/wiki/Configuring-AMP-for-Enterprise-or-Network-Usage


2 Companies : Clumio and Enphase added

Hi @draco-malfoy  !! I added companies to the List and made a PR. Please review and accept it.

@all-contributors please add @Suhas1998 for documentation


Update app-content-store for Training environment

Add backend to build app-content-store in the Training environment.
Add parameters to select which domain to use with the DNS records (Training
does not use the stack domain).

I think we want the external ELB to be created optionally, so we can test if we can remove it on all the environments, similar to commit 5f0bf677a394699e1363c71b0775a93a4f8d42e1

Ah thanks for checking @afda16, I see what you mean.  I'll change this here and in the others, then run it by you again


Easier cactbot customization

From the discussion on #152, I wanted to make a feature here to discuss making cactbot customization easier.  It'd be helpful to know what kinds of customization folks find themselves doing a lot.  I'd also love to know what's hard (and you wish was easier), and what's impossible that you wish could be possible.

I wish i could move timeline and popup text separately on my screen instead of editing .css file

Not sure if that's limited by the OverlayPlugin, but resizing (Dragging corners to resize) for OopsyRaidsy would be sweet without doing css. IIRC the Kagerou parser skin does this.

I wish I could resize the timebars because sometimes, the translation does not fit.

I don't know if this is possible within Cactbot, but it would be really awesome if I could choose an audio device to output its trigger sounds to! (like usb headphones or voice meeter)  The SlimDX Audio plugin doesn't seem to do the job reliably :(

@chocoreb Take a look at #120.  CheVolume might do what you want.

It would be nice to add an option directly in the GUI to change X,Y coordinates. I play on 2 screens and sometimes I switch to one screen only.
If, for exemple, the raidboss window is on the other screen, I can't move it easilly.

Changing the size, colours of the bars and the text inside as well as the font text and its formatting. Basic stuff. Separating timeline from popup would be nice too.

As a contributor myself the only request I have is a button or option within ACT (within the cactbot overlay plugin) to switch languages or that cactbot automatically switches to the language set in the FFXIV-Parsing-Plugin (which everyone needs anyway for ACT to work in general) :)

@meciuPL You should be able to do all of that now with CSS customization.  Are you asking for that to be easier?

I wish I could easily choose which direction to grow when new bars are added (i.e fix the popup position and have the bars grow upward when needed), as well as inverting the sorting order (independently of the grow direction).

It would be really nice to be able to toggle different sounds on an off, as an example the "game on" for ready check I previously would always disable by removing in the JSON array.  Now that the files are hosted remotely by default; it takes more effort to remove.

i think a lot of people would benefit from a guided styling experience instead of having to overwrite the css files.
kagerou's overlay does a great job of walking non technical people through styling the UI, and also has an advanced tab which lets power users update the css itself directly.

Individual triggers can be toggled off.  You just need to add some lines into your local cactbot/user/raidboss.js file, even when you're using remote files.  See https://github.com/quisquous/cactbot/blob/master/AdvancedCactbot.md#customizing-behavior for more details.
In particular, add these lines:
Options.DisabledTriggers = {
  'General Ready check': true,
}

That's cool, i'll do that.  Thank you @quisquous .  Along the lines of the issue title "Easier Cactbot customization"; an interface tab on ACT to do that would be super cool; however I also understand that is a non-trivial amount of effort.
Again, thank you for the link for customization on the remote files 👍

I was also thinking about the customization functionality and starting writing a small UI today for the primary settings currently I broke it out from the primary addons and created another module to support this. Here is the start of what I have so far:
https://gyazo.com/ccb2875bfb6d9379aeff2c824aaa1831
I also have a branch I am trying to work this out of myself, also wondering about feedback and stuff or if something like this is even useful for people

@unfaiyted :heart_eyes: Oh wow, that's some nice-looking UI.  I would be super supportive of this kind of feature.  If you want to file a separate issue that we can talk about this config UI there without crowding out this more brainstormy issue, that might be a good way to keep it focused and move forward.
I think the real catch is the question of how you save all of the settings.  Maybe we would need to send some json back to C# and write that out on disk.  Maybe something that writes out cactbot/user/raidboss-autoconfig.js so that folks can still do manual customization, but this UI could generate the same sort of input that folks are now doing manually.  That's just a thought, though.  Maybe you have some alternative idea of how to persist and reload this data.

Based on some comments i've read so far, is there a way to change the sound for something like Ready Check using the cactbot/user/ folder? I noticed there's an option to disable the trigger, but not one to change it. I've made it work by changing the cactbot/ui/raidboss/data/triggers/general.js file to get it to play a custom sound, but when I tried putting a copy of general.js in my cactbot/user/, it doesn't work.
Is there a current implementation for this? I could just add the following lines into cactbot/ui/raidboss/raidboss.js
ReadyCheckSound: '../../resources/sounds/<name>/<name.ogg>'
and maybe get it to work that way but that's pretty much the same as changing general.js.

@Serin-leetee Yes, you need to do something like https://github.com/quisquous/cactbot/blob/master/user/raidboss-example.js#L198.  Let me know if you run into any issues.

Its not quite working, here's what I'm doing:
I believe the id for Ready check is 'General Ready Check' so in the user folder i'm using this to test it. also a side, the Test Laugh trigger that this was based on also doesn't work for me. The test poke tts works though.
  'General Ready Check': {
    SpeechAlert: false,
    SoundAlert: true,
    TextAlert: false,
    SoundOverride: '../../resources/sounds/WeakAuras/CowMooing.ogg',
    VolumeOverride: 0.2,
  },

Oh that did the trick. Dang I should have noticed it earlier but I totally didn't recognize the lowercase c in the general.js file.

i like to customize some of the callouts like my group in a12s doesnt use triangle and box we just move to our spot or in a11s i like to chase the head and do left right from there. i dont know how to edit the .js, right now i just disable it and do a tts from act itself
the other thing is i want to change the style of timelines but i dont know how to edit.
the best i can mention is kagerou overlay. it's noob friendly. they have coresponding tabs for it.

I'd like it if I could put my own trigger files in the user folder to override ones in the data/triggers folder. I haven't played in a while so I don't know if this feature already exists or not.
Typically when a new boss comes out I would write my own trigger files and once I'm used to them, I don't really want to relearn new callouts from the ones pushed on github.
Typically I just back mine up and after updating to the latest release I just then overwrite all the trigger files with my own, but it would be nice to just be able to extract over everything and not have to do it myself every time.

@Zinterax I think the easiest thing to do to support that would be for you to add some lines to your user raidboss file to disable all the normal triggers, e.g. https://github.com/quisquous/cactbot/blob/master/AdvancedCactbot.md#customizing-behavior.  Then you can copy and paste the whole trigger block that you would have put into ui/raidboss/data/triggers/something.js into the Options.Triggers array, like https://github.com/quisquous/cactbot/blob/master/AdvancedCactbot.md#adding-custom-triggers.
Maybe in the future with @unfaiyted's config generator, it will be easier to disable all the triggers at once.

As a user with customized triggers, I would like to have a streamlined process for updating Cactbot and maintaining my previous customization.
As a technical user, I can see 3 options for this:
#####Option A

Edit directly on the "source" data files (under cactbot/ui/raidboss/data/[patch]/[raidtype]/file.js),
Commit these intentional changes to git, e.g. branch 'custom'
Download cactbot and overwrite the old cactbot folder with the new one.
Branch, commit new cactbot changes, e.g. branch 'new'
Merge 'custom' onto 'new'

#####Option B

Edit every custom changes in the ~/user/raidboss.js config file. This creates for a huge file that's impossible to read over time
Lay down
Try not to cry
Cry a lot

#####Option C

Edit each encounter's customized changes in its own ~/user/encounter.js
import these into the main ~/user/raidboss.js config file

It'd be nice if we could have some feedback, suggestions, and guidance on this issue (especially for newer users).

Personally I find it easier as a "new user" to be modifying existing words as opposed to writing on a blank sheet. Option C is clearly the cleanest and most desirable approach, but there's that moment when you open the new file and just aren't really sure what to write.
I think a hybrid approach for A and C would be best. Either something like copying the original .js trigger file to the ~/user folder then start hacking away at it, or copy-pasting the relevant sections into a pre-made template then modify as desired.

Out of curiosity, what is most of your customization? New triggers? Disabling triggers? Changing text on triggers?
One thing that might help is that I landed a new config UI which will be in the next release: https://github.com/quisquous/cactbot/pull/880#issuecomment-568863244

Option A

Edit directly on the "source" data files (under cactbot/ui/raidboss/data/[patch]/[raidtype]/file.js),
Commit these intentional changes to git, e.g. branch 'custom'
Download cactbot and overwrite the old cactbot folder with the new one.
Branch, commit new cactbot changes, e.g. branch 'new'
Merge 'custom' onto 'new'

If you are doing this, you should add quisquous/cactbot as a git remote and steps 345 should just be to git pull and merge into your current branch.
I don't know that I'd recommend doing this approach, but if you're comfortable with resolving git conflicts, it's certainly an option.

Option B

Edit every custom changes in the ~/user/raidboss.js config file. This creates for a huge file that's impossible to read over time
Lay down
Try not to cry
Cry a lot

Sorry that this is such a pain.
Regarding the huge file, I think the right fix for that is something like https://github.com/quisquous/cactbot/issues/860 where the user folder could support subfolders.  This is something I would really like to have done, but haven't gotten to it yet.  Please feel free to contribute if you have time and are interested.  (I think this would address Option C for you as well.)
Would that solve your pain, or do you need more than that?

Most of the mods involve overriding the tts text and changing the trigger conditions so that it's more useful (and less confusing) for a group setting. Cactbot trigger files are written for personal consumption, so a lot of it needs to be changed, and GroupTTS is just one of those things. I can't override condition in the config file, so my personal use case requires direct modification of the source files.
Example of a big mod: Limit Cut in Cactbot reads the number for the user and gives them a 5s warning before they get cleaved. A group would then benefit from a 5s warning before every pair of cleaves.
I'm personally fine with managing versions of the files, git conflict isn't a big problem since I'll just opt to keep my version over any updates until things break. I just feel that the use case of Cactbot as a group alert is only going to get more popular and other people won't be able to cope.

Regarding the huge file, I think the right fix for that is something like #860 where the user folder could support subfolders.

This is what I have in mind for Option C as well. I haven't looked into the source code for cactbot too deeply since I lack proficiency in js, but I can certainly try to cook something up once I understand how the resources are loaded.

I think this is fixed now with the config panel.


[WIP] Upgrade Moq version

@smitpatel Don't forget to do the open-source dance with @Eilon 🕺 🕺

@ajcvickers - Thanks for reminder. This is still not fixing build. Seems like some breaking change in dotnet SDK/CLI

Moq 4.7.8 is approved.

New version doesn't work.


[Backport v2.10] Add "v27.4.x" options for "Docker Install URL"

backport for issue: https://github.com/rancher/dashboard/issues/12909

created here by mistake. created it in dashboard repo.


Updated sawtooth-kubernetes-default-pbft.yaml to comply with kubernetes version 1.16+

Updated sawtooth-kubernetes-default-pbft.yaml to comply with kubernetes version 1.16+
https://kubernetes.io/blog/2019/07/18/api-deprecations-in-1-16/

I will look at it next weekend, this week will be hard for me.
These changes worked fine for me when I tested it.


Support .NET Standard?

This merge is a harder conversation. 😁
To get to .NET Standard we need to first upgrade to using dotnet restore and dotnet build (or MSBuild 15, or VS 2017).  The cool thing about this new format is (if you have the targeting pack installed) you can also add net35 to the TargetFrameworks and automatically build all three at once.
To add .NET Standard support, we have to remove anything that doesn't work there -- which includes WPF and System.Drawing. So we must write a manual conversion for HSL/HSV to replace calls to WPF's System.Windows.Color and System.Drawing.Color which don't exist in .NET Standard. And we must move all of the color profile stuff to it's own assembly -- which only works on Windows on .Net 4.5
Note that I did have to change the Hue/Saturation on the Silver test to get it to pass. I'm not totally sure why my algorithm gets a different value, but Hue and Saturation are irrelevant for Silver, right? Perceptually, they're the same...

Awesome! I want to take a look at the HSL/HSV thing before I merge - I got the numbers from somewhere originally so I wanted to try cross-checking against another source. Thanks for working on this!

Yeah, please review thoroughly 😉
These are implemented based on the explanations of how it's supposed to work on Wikipedia (with some reference to other language implementations from all over), but it definitely took me more than one try to get it right.
That one Silver thing is just a mystery to me. I'm not sure how everything else can be the same, and that one be so very different. I was guessing the value you have is from the EasyRGB web converter, which seemed like it was picking arbitrary values in the middle of a color (i.e. 60/150/240/300) when things were close to neutral saturation.
I'd feel much better about it if the saturation values were the same, but on a scale of 0-100,  0.7 and 0.02 are imperceptibly different from 0, so I don't know ...

Everything looks good to me, thanks for your patience!

Will you publish a new nuget package @THEjoezack or is the currently some blocking issue w/ regards to multiple SDK versions?


maxListeners

Hi,
I was having an issue where node was complaining about a potential memory leak because too many listeners were being added for the same event.
I saw that this was fixed in Biff (https://github.com/FormidableLabs/biff/commit/04edf8acab28c3e035a52905e312b7e7f4f3870d) but not in this repo.
Is there a particular reason for that? Could we apply the same fix here?

You can setMaxlisteners directly onto the store object yourself to prevent the error bubbling but I'm not sure why we keep the default max listeners on stores, I guess it's quite useful to catch memory leaks from components not being unmounted or being unmounted incorrectly

The fix will find its way back here, I've just been swamped at my full time.

I should have pointed out that I already fixed it the way that @tomatau described it.
But now I'm second-guessing the way I've structured my components, instead of having every GridItem listen to a Store, I should probably have the GridView listen to it and propagate the changes to the items via a property?
Anyhow, I can whip up a PR for the Store if you want.


Encryption support

Hi,
Does LiteDB encrypt the .db file by default?  I can't see any documentation on the support for encryption in the wiki.  Are there any examples of how it is done?  Note that I am using v2.0.
Thanks.

Hi @dgodwin1175 , no, if you want encrypt your datafile you must provide "password=" keyword in connection string.
https://github.com/mbdavid/LiteDB/wiki/How-LiteDB-Works#connection-string

@mbdavid I think the URL should be updated to:
https://github.com/mbdavid/LiteDB/wiki/Connection-String


[firebase_ml_vision] Flip detected barcode coordinates on Android

Android uses reversed coordinates, flip those coordinates to draw a detection box correctly.

Hi @gi097,
Where did you see/experience that they were flipped? I didn't need to flip coordinates in the example app.

I faced this issue when running the example code (the code I changed here) on a Motorola Moto G4.

Oh, thats right. I was thinking this was an issue not a PR.
Anyways, I tested on a G4 and was unable to reproduce the bug. It may have been fixed by this PR?

@bparrishMines, I don't have time to test it now. Since the problem seems to be solved, I will close this PR. Thanks.


docs: Update quick start doc for user assigned identity

Previous code will return Unable to get object id from principal name.
Reason for Change:
Using --spn only works after a period of time ~60s, using --object-id works immediately.

Requirements

[ ] squashed commits
[ ] included documentation
[ ] added unit tests and e2e tests (if applicable).

Issue Fixed:

Please answer the following questions with yes/no:
Does this change contain code from or inspired by another project? If so, did you notify the maintainers and provide attribution?

[ ] yes
[ ] no

Notes for Reviewers:

/azp run Azure Workload Identity CI

/azp run Azure Workload Identity CI


Improper folder specification; file missing / extra file present. See documentation

Found bam file: /home/i439h/projects/heroes-aya-pools/AG_Thongjuea/Result/10x_multiomics/CellRanger_ARC/1016715_pool3_hg38/outs/gex_possorted_bam.bam for genotyping.
Wed Mar 20 12:38:37 CET 2024: Found file of barcodes to be parsed: /home/i439h/projects/heroes-aya-pools/AG_Thongjuea/Code/bulkWGS/Sadeghi/mgatk/barcodes.tsv
Wed Mar 20 12:38:39 CET 2024: User specified mitochondrial genome matches .bam file
Wed Mar 20 12:43:06 CET 2024: Finished determining/splitting barcodes for genotyping.
Wed Mar 20 12:43:12 CET 2024: Genotyping samples with 20 threads
Error in checkGrep(grep(".A.txt", files)) :
Improper folder specification; file missing / extra file present. See documentation
Calls: importMito -> checkGrep
Execution halted

Dear @caleblareau , have you found the solution?
best
Iman

Did you find a work around for this issue?


MediaFoundationReader ready to play event?

Hi,
I am currently experimenting of using MediaFoundationReader along with WaveOutEvent to stream  audio from URIs and I was wondering whether there is an event or a property to check which signals that the playback can start.
Also, for a .wav file I noticed that there is some delay in the playback, even after the WaveOutEvent.PlaybackState get status PlaybackState.Playing
Thanks

I imagine there is a buffering built in to Media Foundation when you play from a URI which accounts for this delay. I'm not aware of an event you can use, but there may be one if you are willing to explore the MF documentation.


Only Pulls 1st Page Products

If you have more products than the 50 displayed on the first page of the query then only the first page will show up.
I was able to add a limit=250 to the query, but this is a bit of a hack fix and seems to only allow that value as the top limit.

Is it possible you PM me your shopify credentials of the API you use for the plugin? Or maybe create a separate app inside of shopify-ui with less authorizations. Because I do not have an account to test/reproduce this with.

Would just like to add that I have the same issue.
It looks like the API now returns results in paginated form with links to the next (and prev) pages in the response header (the key is 'Link'). You also need to switch to the latest version of the API to get the links returned in the header.
Not sure why the functionality changed in the older version of the API as well as the new version?
I have a really hacky temporary fix - it is not advisable to use this in production but it demonstrates the issue at least. It's just proof of concept. It would be wise to either use a while loop with a reliable break function or maybe run another query to get max results and then use that to control the expected number of loops in order to avoid it looping too long. At the moment it's got a hard coded cap of 10... not pretty.
`/**

Created by PhpStorm.
User: nmaier
Date: 22.07.18
Time: 15:08
*/

namespace shopify\services;
use yii\base\Component;
class ShopifyService extends Component
{
/**

Get all products from shopify account.

@param array $options

@return bool
*/
public function getProducts($options = array())
{
$all_products = array();
$next_link = null;
// Theoretically this should be a do while loop because we don't
// know how many iterations it's going to take.
// Am actually using a for loop so if the break clause fails
// we don't end up in infinite loop territory.
for ($i = 0; $i <= 10; $i++) {
// Make the request and add any found products to the master array.
$response = $this->getProductsPageResponse($options, $next_link);
$items = json_decode($response->getBody()->getContents(), true);
$products = $items['products'];
$all_products = array_merge($all_products, $products);
// See if we have any link headers.
$next_link_found = false;
if ($header_links = $response->getHeader('Link')) {
$header_link_array = explode(', ', $header_links[0]);
foreach ($header_link_array as $header_link) {
if (strpos($header_link, 'rel="next"') !== false) {
$next_link_found = true;
// Need to sanitize the link.
$link_explode = explode('>', $header_link);
$link = $link_explode[0];
$link = substr($link, 9);
// Assign the new link to the old link variable.
$next_link = $link;
}
}
// If no next link then break out of loop.
if (!$next_link_found) {
break;
}
}
}
// Return all the products from all the pages.
return $all_products;
}

/**
 * get all products from one page of a shopify account
 *
 * @param array $options
 * @param string $url_override
 * @return bool
 */
public function getProductsPageResponse($options = array(), $url_override = null)
{
  $settings = \shopify\Shopify::getInstance()->getSettings();

  $query = http_build_query($options);
  // $url = $this->getShopifyUrl($settings->allProductsEndpoint . '?' . $query, $settings);

  if (!is_null($url_override)) {
    $url = 'https://' . $settings->apiKey . ':' . $settings->password . '@' . $url_override;
  }
  else {
    $url = 'https://' . $settings->apiKey . ':' . $settings->password . '@' . $settings->hostname . '/admin/api/2019-10/products.json';
  }

  try {
      $client = new \GuzzleHttp\Client();
      $response = $client->request('GET', $url);
      if ($response->getStatusCode() !== 200) {
          return false;
      }
      return $response;
  } catch(\Exception $e) {
      return false;
  }
}

/**
 * Get specific product from Shopify
 *
 * @param array $options
 * @return bool
 */
public function getProductById($options = array())
{
    $settings = \shopify\Shopify::getInstance()->getSettings();

    $id = $options['id'];
    $fields = isset($options['fields']) ? '?fields=' . $options['fields'] : '';
    $url = $this->getShopifyUrl($settings->singleProductEndpoint . $id . '.json' . $fields, $settings);

    try {
        $client = new \GuzzleHttp\Client();
        $response = $client->request('GET', $url);

        if ($response->getStatusCode() !== 200) {
            return false;
        }

        $items = json_decode($response->getBody()->getContents(), true);

        return $items['product'];
    } catch(\Exception $e) {
        return false;
    }
}

/**
 * @param $endpoint
 * @param \shopify\models\settings $settings
 * @return string
 */
private function getShopifyUrl($endpoint, \shopify\models\settings $settings)
{
    return 'https://' . $settings->apiKey . ':' . $settings->password . '@' . $settings->hostname . '/' . $endpoint;
}

}
`

@nmaier95, any progress on this? I love the plugin but we have lots of products and only getting the first page is not enough.


fix gettest error

As it turns out, even if it looks redundant, the explicit path to jsi18n is necessary in the list of urls, otherwise the frontend get a javascript error about not finding gettext().
(for info this line was removed during the pull request review).

Hm, interesting. Do you know under what circumstances this error happens? I didn't see anything broken when I tried out the site with your previous changes. Also, I don't see why the two paths would behave differently, as they seem like they should be the same except that one has caching. I'd like to be sure this is the right fix, and ideally also understand what the root cause is.
(If this does turn out to be necessary, please add a comment explaining that, so it doesn't get accidentally removed again in the future.)

You can see the problem on the home page of a puzzle hunt: hover over the start/end time of the hunt and there's a javascript to show them to you in your local time. That's no longer working.
I don't have an explanation of why it's allowed to have 2 entries for the same url, but both make a difference: one makes it work, the other caches the javascript in the browser (you can see that in the developer tools). I got this from the doc, but I don't know why it has to be that way.
I read on stack overflow that the order matters, and the /js18 url should be put at the top of the list, but that didn't fix the problem for me.

Yeah, I'm saying I can run the server without this PR and everything is fine; js18n is loaded correctly and the hover works as well. I'm sorry to block this but I want to dig further into why this is happening for you later today -- maybe it's somehow dependent on the Django version, or only appears when running in a non-English language?

I'm building a US hunt site and that's how I noticed it.

It's still working for me, no matter what I try.
Documentation here and here only show one entry in urls. Can you link the doc page and stack overflow page you mentioned earlier?
Is it possible your Django cache is misconfigured? If you go to http://lemm.as:8000/, does it work for you?

Your site works for me. Mine runs on heroku, maybe that's where the difference comes from?


Python 3.11 support?

It would be nice to have a release with Python 3.11 support.

:+1: See the linked issues from coincidentally named 0.11 release: https://github.com/zarr-developers/numcodecs/issues/377

Closing to consolidate

Thanks! Github could really use a "closed in favor of (link to other issue)" option...


Create a tile that allows you to change the screen fading time

I like the way the quick tile works more than the one on LineageOS.
If someone wants to implement this, please add a second quick tile that implements this behavior.


[Docs]: Next.js App Router internationalized navigation

Link to page
https://next-intl-docs.vercel.app/docs/routing/navigation
Describe the problem
The docs are out of date as createSharedPathnamesNavigation no longer exists in the latest versions.

It definitely exists in the latest versions. Are you using an old version of next-intl?

Hmm.. I was on v3.3.1 when I posted this and was getting an error. Now with 3.3.2 released a couple days ago it seems to work!
Whatever happened, good news!
Thanks!


Module genetator does not respect config path

in modules config I changed path to configs to lowercase, and after generation i saw both config and Config folders (on screenshot) https://imgur.com/n9KGpiN

Hello,
Sorry for the delayed response.
I just checked this out, and it seems to only do this with casing changes.
However, the path sent to the laravel Finder, receives the correct path with modules lowercase for instance.
So I assume this is OS related and not the laravel filesystem itself.


Explain more the update os screen

Avoid duplicates

[x] This issue has not already been raised before
[x] If you are proposing a new feature, please do so in CWA-Wishlist

Current Implementation
Version 1.12.1 introduced an update os screen, which is shown to users on iOS 13 versions below 13.7.
At the moment, this screen only says:

Please update your iOS.
You need the current iOS version to install the Corona-Warn-App. To do so, open the settings in your iPhone and select General -> Software Update.

Suggested Enhancement
Add a link to a FAQ entry (for example: https://www.coronawarn.app/en/faq/#ios135) to give the user an explanation why this is needed.
Expected Benefits
More user really updating their iOS version and not just uninstalling CWA (when reading the comments on the App Store, it seems like the most users will just delete the app).

Related Issue
#2032

@Ein-Tim
Thank you! I've forwarded the suggestion to the devs. We'll see what we can do about it.

Corona-Warn-App Open Source Team

There are more suggestions what should be shown on this screen to mitigate the fact / decision that cwa has dropped support for ENF v1 and for the affected iOS versions:

a link to list of other European corona warning apps, which could be an alternative for a user who does not want to upgrade iOS
a hint that not all these apps exchange keys with every other country, and most don't accept infection reported from another country.

See the discussion at https://github.com/corona-warn-app/cwa-app-ios/issues/2032 , especially this comment and this.

@ndegendogo ok, ticket updated.

Internal Tracking ID: EXPOSUREAPP-5275

There's now a FAQ entry which could be linked to from this screen:
https://www.coronawarn.app/de/faq/#update_iOS
or
https://www.coronawarn.app/en/faq/#update_iOS

I consider this issue as obsolete since the most people who were seeing the update os screen most likely uninstalled the app or updated to a supported iOS version.
Therefore I'm closing this issue now.


Facing issues while importing 'network' from 'tensorflow.python.keras.engine'

On Tensorflow 2.3.0
Anaconda Environment.
from tf_agents.environments import suite_gym

ImportError                               Traceback (most recent call last)
<ipython-input-8-b9cd8ac2787f> in <module>
----> 1 from tf_agents.environments import suite_gym
      2
      3 # env = suite_gym.load("Breakout-v4")
      4 # env

~\AppData\Roaming\Python\Python38\site-packages\tf_agents\environments\__init__.py in <module>
     24 from tf_agents.environments import tf_py_environment
     25 from tf_agents.environments import trajectory_replay
---> 26 from tf_agents.environments import utils
     27 from tf_agents.environments import wrappers

~\AppData\Roaming\Python\Python38\site-packages\tf_agents\environments\utils.py in <module>
     23 from tf_agents.environments import tf_environment
     24 from tf_agents.environments import tf_py_environment
---> 25 from tf_agents.policies import random_py_policy
     26 from tf_agents.specs import array_spec
     27

~\AppData\Roaming\Python\Python38\site-packages\tf_agents\policies\__init__.py in <module>
     16 """Policies Module."""
     17
---> 18 from tf_agents.policies import actor_policy
     19 from tf_agents.policies import boltzmann_policy
     20 from tf_agents.policies import epsilon_greedy_policy

~\AppData\Roaming\Python\Python38\site-packages\tf_agents\policies\actor_policy.py in <module>
     27 import tensorflow_probability as tfp
     28
---> 29 from tf_agents.networks import network
     30 from tf_agents.policies import tf_policy
     31 from tf_agents.specs import tensor_spec

~\AppData\Roaming\Python\Python38\site-packages\tf_agents\networks\__init__.py in <module>
     16 """Networks Module."""
     17
---> 18 from tf_agents.networks import actor_distribution_network
     19 from tf_agents.networks import actor_distribution_rnn_network
     20 from tf_agents.networks import bias_layer

~\AppData\Roaming\Python\Python38\site-packages\tf_agents\networks\actor_distribution_network.py in <module>
     24 import tensorflow as tf  # pylint: disable=g-explicit-tensorflow-version-import
     25
---> 26 from tf_agents.networks import categorical_projection_network
     27 from tf_agents.networks import encoding_network
     28 from tf_agents.networks import network

~\AppData\Roaming\Python\Python38\site-packages\tf_agents\networks\categorical_projection_network.py in <module>
     24 import tensorflow_probability as tfp
     25
---> 26 from tf_agents.networks import network
     27 from tf_agents.networks import utils
     28 from tf_agents.specs import distribution_spec

~\AppData\Roaming\Python\Python38\site-packages\tf_agents\networks\network.py in <module>
     31
     32 # pylint:disable=g-direct-tensorflow-import
---> 33 from tensorflow.python.keras.engine import network as keras_network  # TF internal
     34 from tensorflow.python.training.tracking import base  # TF internal
     35 from tensorflow.python.util import tf_decorator  # TF internal

ImportError: cannot import name 'network' from 'tensorflow.python.keras.engine' (C:\Users\asus\anaconda3\envs\tf2\lib\site-packages\tensorflow\python\keras\engine\__init__.py)

Tried Reinstalling TF. The issue still persists.
Tried running on TF 2.2, it asks me to upgrade.

@vybhav72954 can we change this to bug rather than "type:others" please?

@vybhav72954,
Could you please install the tf-agents-nightly package. With the nightly version, I was able to import the module without any issues. Please find the gist of it here. Thanks!

I think pip install tf-agents==0.5.0 is enought

installing the tf-agents-nightly package does not solve the problem . if you  install it you get an error  on
from tf_agents.agents.dqn import dqn_agent

I don't get any error on from tf_agents.agents.dqn import dqn_agent with  tf-agents-nightly

How i resolved the issue :
uninstall version tf_agents 0.5.0  ===>   pip uninstall tf-agents
install the nightly version 0.6.0 at the moment ==> pip install tf-agents-nightly

Kindly give me a few days, as my laptop has blown up and I don't have access to another system cause of the pandemic. I'll update you by the weekend. Sorry for the inconvenience.
@bhack I had tried that, but wasn't aware about the Version issue, thnks

@vybhav72954  Yes cause TF version in colab i.e. to run @amahendrakar  gist was updated as default to 2.3.0 probably  just yesterday/today

Thank you, it works @amahendrakar
Though am asking too much, but is possible can you put some light on what was wrong here?

Though am asking too much, but is possible can you put some light on what was wrong here?

@vybhav72954,
Seems like the issue was caused due to TensorFlow and TF-Agents version incompatibility. For more information, please take a look at this link. Thanks!

+1 looks like you need to use a newer version of TF-Agents.


CAMPD Contact Us

Definition of Ready

[x] Does this meet the INVEST criteria?
[x] Does this have a clearly defined user?
[x] Does this have an estimate?
[x] Does this have acceptance criteria?
[x] Does this have sufficient context (wireframes, business rules, etc.)?
[x] Is this traceable to a feature, epic, or PI?

User Story
As a CAMPD user I want to view the contact us page So that that I can submit an issue to the EPA
Acceptance Criteria
Given that a user has accessed the Contact Us Page
When the functionality is displayed
Then the page renders in accordance with the wireframes (see context)
And is consistent with the Contact Us Content document
Given that a user has selected Contact us on the utility nav
When the functionality is displayed
Then the contact us page renders in accordance with the wireframes (see context)
Given that a user has accessed the FAQ page
When submit a help ticket is selected
Then the user is redirected to the contact us page
Given that a user has accessed the tutorials page
When the contact us link is selected
Then the user is redirected to the contact us page
Given that a user has filled out the contact us form
When the form is submitted
Then the form is sent the the CAMPD contact us email (see context)
Developer Notes
As apart of the scope of this ticket a request should be sent to the CAMPD contact us email.
Swagger page for the API endpoint to Send Email for Contact Us page
https://easey-dev.app.cloud.gov/quartz/api/swagger
Contact Us Email
https://easey-dev.app.cloud.gov/quartz/api/triggers/notifications/emails
please set the purpose property to Contact CAMPD Support when using this from the Contact Us
These are protected by an API Key that must be passed in the x-api-key header
CAMPD contact us Emails to<EMAIL_ADDRESS>the subject should be Comment Type and the message (body) should be the Comment
Does this Story require 508 Testing?

[ ] Implementing a new authoring tool that will produce code / content for the application
[ ] Implementing a 3rd party widget that will be part of the application
[ ] Adding new / updating custom functionality
[x] Adding new / updating a page template
[ ] Adding new / replacing existing functionality (unless the replacement has already passed 508 testing)
[ ] Full Site Review prior to a scheduled EPA 508 review
[ ] Planning a video that will appear on the application
[ ] Producing a video that will appear on the application
[ ] Producing Training materials
[ ] Producing documents

Context
(User journeys, design mockups, architectural diagrams, business rules, data element definitions)

Contact Us
Contact Us (dev)

Contact Us Content document
Testing
Test Case: TC498
Tester: @JanellC
508 Testing:
Tester: @aprematta
Definition of Done

[x] Have all Acceptance Criteria been agreed to and validated by the scrum team?
[x] Has the code been unit tested and peer reviewed?
[x] Have the functional tests been executed?
[x] Have all defects been reviewed and dispositioned, resolved, or deferred?
[ ] Has the user story been accepted by the Product Owner?
[x] Is the user story ready to be deployed to the test/staging environments?

TC498 passed @tlageman-epa . This ticket is ready for 508 testing @aprematta


Private repository - Use multiple accounts on one machine

Supporting this would be awesome. At the moment you cannot use multiple scopes on one machine.
I think the scenario that a team can consist of more than 1 company or private person is not infrequent. I have the case that we collaborate in a team where many people host their code in private repos.
Thanks!

The title of this issue seems to say something differently than its description. We can't warn you that you're not "allowed" to download packages, since we would simply get a 404 in that case from the registry.
Having multiple registries is a different issue.
Please clarify.

@alexanderGugel is it clear?


Add possiblity to change websocket ping interval

Sets ping client ping interval without changing configs.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.0 out of 21 committers have signed the CLA.:x: zhangyanjiaoesp:x: jack0c:x: xiongweichao:x: JiaLWang:x: wmy-espressif:x: Espressif-liuuuu:x: Isl2017:x: Weijian-Espressif:x: MaxwellAlan:x: muhaidong:x: Icarus113:x: OttoWinter:x: suda-morris:x: o-marshmallow:x: igrr:x: sarveshb14:x: KonstantinKondrashov:x: mahavirj:x: laukik-hase:x: zikalino:x: esp-cjhmuhaidong seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


VMware PowerCLI 6.5 Compatibility

The vSwitch Security plugin does not detect PowerCLI correctly and reports:
..start calculating vSwitch Security by Justin Mercier, Sam McGeown, John Sneddon v1.2 [58 of 92]
WARNING: PowerCLi version installed is lower than 5.1 Release 2
The version I have installed is: "VMware PowerCLI 6.5 Release 1 build 4624819" which doesn't match the version check of this plugin:
if (((Get-PowerCLIVersion) -match "VMware vSphere PowerCLI (.*) build ([0-9]+)"))
I'm going to change this line to eliminate vSphere from the string to match and I believe it will be ok.

Fixed in dev branch.
https://raw.githubusercontent.com/alanrenouf/vCheck-vSphere/dev/Plugins/50 Network/98 vSwitch Security.ps1


gnome-terminal prefs didn't install cleanly on Gnome 3.8+

The default profile is "Unnamed"

Need to install via dconf http://unix.stackexchange.com/a/133920


Custom Bot Name per Application (One WebHook)

Is this possible? Custom Bot Name per Application (One WebHook)
or
Can I use the render to dynamically add project name to the top of the message?
Core 2
<PackageReference Include="Serilog.Sinks.Slack.Core" Version="0.1.3-beta" />

I would say that, if Slack WebHooks API supports it, it should be easy to implement. Let me check and I'll come back to it.

Try and use this with the latest version:
var log = new LoggerConfiguration().WriteTo.Slack("https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX", username: "Test User", iconEmoji: Serilog.Sinks.Slack.Core.Sinks.Slack.Client.Emoji.MonkeyFace).CreateLogger();

That's cool so I can use one webhook and that will set the username which would be the name of the bot and the icon?

Seems to be working, very nice option set.

Side question: Is there anyway to have it not crash if the hook URL is empty e.g. from settings? Stuggling with the new way of doing logging in Core 2 and having conditional log methods.


test with ansible 2.7

Currently we test upstream in Jenkins with Ansible 2.5 and 2.6, as discussed in #2901
Please add testing for Ansible 2.7. Their 2.7.0rc3 prerelease is available.
Ansible 2.5 will go end-of-life after Ansible 2.7 is released (https://access.redhat.com/support/policy/updates/ansible-engine)

This is duplicate of #3148 and will be addressed by #3151
I'm closing this


fix mismatch in http/https protocol

This should fix issues where API reference page will not load when there's a protocol mismatch when this page is accessed with https and the swagger ui JS loads the swagger reference page using http.
I've not been able to get the public API reference page for months now due to this error

🚢


mef_eline isn't allocating/releasing available_tags for UNIs

When a EVPL is created, only NNIs are allocating vlans from available_tags, UNIs vlans are entirely relying on _is_duplicated_evc(evc) to try to detect as conflict as mentioned on issue #323, which also works, but this is an inconsistency implementation detail that makes things hard especially in the future when vlan range will be introduced, if we were to coherently also be allocating and releasing available_vlans for UNIs when an EVC is being created/deleted/updated, then to check if a VLAN is still available to be created is just a matter of checking if the tag or range of tags are available, which is simpler than iterating over all circuits and still compare (which is also OK, even though it also has a worse time complexity but as vlan ranges are introduced it's becoming evident for vlan ranges the current implementation will make things harder than it should be, so it's probably worth it allocating/releasing the VLANs for UNIs when tags are in place to simplify code maintenance)
Example:

Notice that dpid interface [1] s1-eth1, available_tags len doesn't change after creating an EVPL, only for NNIs vlans are being allocated/released

kytos $> len(controller.switches[dpid].interfaces[1].available_tags)
Out[9]: 4095

kytos $> len(controller.switches[dpid].interfaces[4].available_tags)
Out[10]: 4095

kytos $>

kytos $> 2023-05-05 14:29:30,195 - INFO [kytos.napps.kytos/flow_manager] (AnyIO worker thread) Send FlowMod from request dpid: 00:00:00:00:00:00:00:01, command: delete, force: True, flo
ws_dict: {'flows': [{'cookie':<PHONE_NUMBER>8004794693, 'cookie_mask':<PHONE_NUMBER>3709551615}], 'force': True}
2023-05-05 14:29:30,198 - INFO [uvicorn.access] (MainThread) <IP_ADDRESS>:36304 - "POST /api/kytos/flow_manager/v2/delete/00%3A00%3A00%3A00%3A00%3A00%3A00%3A01 HTTP/1.1" 202
2023-05-05 14:29:30,205 - INFO [kytos.napps.kytos/flow_manager] (AnyIO worker thread) Send FlowMod from request dpid: 00:00:00:00:00:00:00:03, command: delete, force: True, flows_dict:
{'flows': [{'cookie':<PHONE_NUMBER>8004794693, 'cookie_mask':<PHONE_NUMBER>3709551615}], 'force': True}
2023-05-05 14:29:30,207 - INFO [uvicorn.access] (MainThread) <IP_ADDRESS>:36316 - "POST /api/kytos/flow_manager/v2/delete/00%3A00%3A00%3A00%3A00%3A00%3A00%3A03 HTTP/1.1" 202
2023-05-05 14:29:30,216 - INFO [uvicorn.access] (MainThread) <IP_ADDRESS>:36326 - "POST /api/kytos/pathfinder/v2/ HTTP/1.1" 200
2023-05-05 14:29:30,236 - INFO [kytos.napps.kytos/flow_manager] (AnyIO worker thread) Send FlowMod from request dpid: 00:00:00:00:00:00:00:01, command: add, force: False, flows_dict: {'
flows': [{'match': {'in_port': 1, 'dl_vlan': 11}, 'cookie':<PHONE_NUMBER>8004794693, 'actions': [{'action_type': 'set_vlan', 'vlan_id': 11}, {'action_type': 'push_vlan', 'tag_type': 's'},
{'action_type': 'set_vlan', 'vlan_id': 2}, {'action_type': 'output', 'port': 4}], 'owner': 'mef_eline', 'table_group': 'evpl', 'table_id': 0, 'priority': 20000}, {'match': {'in_port': 4
, 'dl_vlan': 2}, 'cookie':<PHONE_NUMBER>8004794693, 'actions': [{'action_type': 'pop_vlan'}, {'action_type': 'output', 'port': 1}], 'owner': 'mef_eline', 'table_group': 'evpl', 'table_id':
 0, 'priority': 20000}], 'force': False}
2023-05-05 14:29:30,245 - INFO [uvicorn.access] (MainThread) <IP_ADDRESS>:36342 - "POST /api/kytos/flow_manager/v2/flows/00%3A00%3A00%3A00%3A00%3A00%3A00%3A01 HTTP/1.1" 202
2023-05-05 14:29:30,254 - INFO [kytos.napps.kytos/flow_manager] (AnyIO worker thread) Send FlowMod from request dpid: 00:00:00:00:00:00:00:03, command: add, force: False, flows_dict: {'
flows': [{'match': {'in_port': 1, 'dl_vlan': 11}, 'cookie':<PHONE_NUMBER>8004794693, 'actions': [{'action_type': 'set_vlan', 'vlan_id': 11}, {'action_type': 'push_vlan', 'tag_type': 's'},
{'action_type': 'set_vlan', 'vlan_id': 2}, {'action_type': 'output', 'port': 3}], 'owner': 'mef_eline', 'table_group': 'evpl', 'table_id': 0, 'priority': 20000}, {'match': {'in_port': 3
, 'dl_vlan': 2}, 'cookie':<PHONE_NUMBER>8004794693, 'actions': [{'action_type': 'pop_vlan'}, {'action_type': 'output', 'port': 1}], 'owner': 'mef_eline', 'table_group': 'evpl', 'table_id':
 0, 'priority': 20000}], 'force': False}
2023-05-05 14:29:30,263 - INFO [uvicorn.access] (MainThread) <IP_ADDRESS>:36352 - "POST /api/kytos/flow_manager/v2/flows/00%3A00%3A00%3A00%3A00%3A00%3A00%3A03 HTTP/1.1" 202
2023-05-05 14:29:30,272 - INFO [kytos.napps.kytos/mef_eline] (AnyIO worker thread) EVC(14a051874e0545, evpl2) was deployed.
2023-05-05 14:29:30,278 - INFO [uvicorn.access] (MainThread) <IP_ADDRESS>:36294 - "POST /api/kytos/mef_eline/v2/evc/ HTTP/1.1" 201
2023-05-05 14:29:30,281 - INFO [uvicorn.access] (MainThread) <IP_ADDRESS>:36362 - "POST /api/kytos/pathfinder/v2/ HTTP/1.1" 200
2023-05-05 14:29:30,306 - INFO [kytos.napps.kytos/flow_manager] (AnyIO worker thread) Send FlowMod from request dpid: 00:00:00:00:00:00:00:02, command: add, force: False, flows_dict: {'
flows': [{'match': {'in_port': 2, 'dl_vlan': 2}, 'cookie':<PHONE_NUMBER>8004794693, 'actions': [{'action_type': 'set_vlan', 'vlan_id': 2}, {'action_type': 'output', 'port': 3}], 'owner': '
mef_eline', 'table_group': 'evpl', 'table_id': 0, 'priority': 20000}, {'match': {'in_port': 3, 'dl_vlan': 2}, 'cookie':<PHONE_NUMBER>8004794693, 'actions': [{'action_type': 'set_vlan', 'vl
an_id': 2}, {'action_type': 'output', 'port': 2}], 'owner': 'mef_eline', 'table_group': 'evpl', 'table_id': 0, 'priority': 20000}], 'force': False}
2023-05-05 14:29:30,347 - INFO [uvicorn.access] (MainThread) <IP_ADDRESS>:36372 - "POST /api/kytos/flow_manager/v2/flows/00%3A00%3A00%3A00%3A00%3A00%3A00%3A02 HTTP/1.1" 202
2023-05-05 14:29:30,360 - INFO [kytos.napps.kytos/flow_manager] (AnyIO worker thread) Send FlowMod from request dpid: 00:00:00:00:00:00:00:01, command: add, force: False, flows_dict: {'
flows': [{'match': {'in_port': 3, 'dl_vlan': 2}, 'cookie':<PHONE_NUMBER>8004794693, 'actions': [{'action_type': 'pop_vlan'}, {'action_type': 'output', 'port': 1}], 'owner': 'mef_eline', 't
able_group': 'evpl', 'table_id': 0, 'priority': 20000}], 'force': False}
2023-05-05 14:29:30,371 - INFO [uvicorn.access] (MainThread) <IP_ADDRESS>:36388 - "POST /api/kytos/flow_manager/v2/flows/00%3A00%3A00%3A00%3A00%3A00%3A00%3A01 HTTP/1.1" 202
2023-05-05 14:29:30,379 - INFO [kytos.napps.kytos/flow_manager] (AnyIO worker thread) Send FlowMod from request dpid: 00:00:00:00:00:00:00:03, command: add, force: False, flows_dict: {'
flows': [{'match': {'in_port': 2, 'dl_vlan': 2}, 'cookie':<PHONE_NUMBER>8004794693, 'actions': [{'action_type': 'pop_vlan'}, {'action_type': 'output', 'port': 1}], 'owner': 'mef_eline', 't
able_group': 'evpl', 'table_id': 0, 'priority': 20000}], 'force': False}
2023-05-05 14:29:30,389 - INFO [uvicorn.access] (MainThread) <IP_ADDRESS>:36404 - "POST /api/kytos/flow_manager/v2/flows/00%3A00%3A00%3A00%3A00%3A00%3A00%3A03 HTTP/1.1" 202
2023-05-05 14:29:30,399 - INFO [kytos.napps.kytos/mef_eline] (thread_pool_app_7) Failover path for EVC(14a051874e0545, evpl2) was deployed.
kytos $>

kytos $> len(controller.switches[dpid].interfaces[1].available_tags)
Out[11]: 4095

kytos $> len(controller.switches[dpid].interfaces[4].available_tags)
Out[12]: 4094

@viniarck we are planing to use this feature in production, and the potential risks of having overlapping usage of a tag for NNI/UNI is quite important. I believe we should increase the priority of this issue and even change it's label to a bug instead of enhancement (since it creates the possibility of duplication, I would consider as a bug)

As a workaround, we could do the following on the kytos console:
kytos $> from kytos.core.interface import TAG, TAGType
kytos $> controller.switches["00:00:00:00:00:00:00:01"].interfaces[2].use_tag(TAG(TAGType.VLAN, 1899))
Out[21]: True

@viniarck @Alopalao we are starting to have some problems related to this issue in a deploy we have and we might need to think about a workaround solution for this problem (backporting to previous releases). The problems we are seeing is due to NNI being used as UNIs to some EVCs, and the pool of VLANs are reaching the range of user-vlans, which are creating conflicting flows for Service VLANs and for Customer VLANs.

@italovalcy, right. @Alopalao's PR https://github.com/kytos-ng/mef_eline/pull/371, is also taking an opportunity to fix another update issue, and it's also leveraging his other PR on kytos https://github.com/kytos-ng/kytos/pull/409 that's fixing vlan pool and using the new methods (that are already being prepared to also work with a range of tags) when allocating and releasing vlans.
If only allocating/deallocating the vlans solve the problem then we could backport only this part to 2023.1 (and potentially to 2022.3 if you can't afford to upgrade to 2023.1 soon), and then provide a migration script to update DB collection interface_details available_vlans accordingly. Does it solve the problem or you need more than this? @Alopalao if you could make an assessment here or if you have other ideas, thanks.
Side note: backporting all of the data model changes would be very difficult and it's a significant effort in terms of development and exploratory tests (since it impacts horizontally mef_eline and topology too). We'll need to find a balance in terms of effort and minimizing risks while providing a solution for this problem (and if we need more than the previous paragraph then also assess if we either cut the release early and or also check the viability or an anticipated prod deployment).

Talked about this with Italo I think a temporal solution would be better for this case. kytos, mef_eline and topology would need to have these changes so UNI can get their TAGs allocated. This way we will reduce the process of making sure the changes with data model and changes with lists of ranges of TAGs are working correctly. These changes are doomed to be deprecated later though.


Replace iframes with wrappers

Closes #160
Testing PR:
OSSM Console plugin in Openshift should behave the same way as with iframes. If you install the plugin via operator takes into account cluster param issue (https://github.com/kiali/kiali/issues/6322).
IstioConfigListPage has been refactored and now it shows Validation summary as in Kiali and namespace reference link:

Note: To make it work I have modified 3 files of Kiali application. This will be addressed in https://github.com/kiali/kiali/issues/6266 to avoid these modifications in the future.

If the kialiUrl is getting removed, then this issue can be closed since it involves the kialiUrl setting:

https://github.com/kiali/openshift-servicemesh-plugin/issues/28

operator work to remove kiali.url is done here: https://github.com/ferhoyos/openshift-servicemesh-plugin/pull/4

now that the new frontend code is in master, I can't build this PR to test it... lots of errors about default exports. So I'll wait for updates to this PR before testing
example of error (there are many like this):
ERROR in ./src/openshift/utils/Reducer.ts:16:8
TS1192: Module '"/home/jmazzite/source/kiali/openshift-servicemesh-plugin/plugin/src/kiali/reducers/MessageCenter"' has no default export.
    14 | import tour from 'reducers/TourState';
    15 | import namespaces from 'reducers/NamespaceState';
  > 16 | import Messages from 'reducers/MessageCenter';

Compilation errors are fixed. @jmazzitelli you can test PR again.

BTW... @ferhoyos while looking up how to install this via make, I noticed this in the README that is incorrect : https://github.com/kiali/openshift-servicemesh-plugin/pull/172


UHF-10713: Fix update hook if module is not installed of config is missing

UHF-10713

What was done
Update hooks fails on paatokset: https://github.com/City-of-Helsinki/helsinki-paatokset/actions/runs/12369282693/job/34520859820.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 88.48%. Comparing base (fa87b6a) to head (1d42064).
Report is 1 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main     #194      +/-   ##
============================================
- Coverage     88.63%   88.48%   -0.15%
  Complexity      504      504
============================================
  Files            76       76
  Lines          2033     2033
============================================
- Hits           1802     1799       -3
- Misses          231      234       +3

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Configurable min and max zoom for thumbnail

Allows passing min/max zoom when thumbnail is generated.

Coverage increased (+0.07%) to 87.429% when pulling 51302afc3820b49e12779d7dbef41b6ec4b61051 on config-zoom into ce81b32e34bcf91ef5ba64acc1e43d91798129ea on configure-encoding-options.