Fail to build Mixtral-8x22B-v0.1 with w4a16

System Info
tensorrt                  10.0.1
tensorrt-cu12             10.0.1
tensorrt-cu12-bindings    10.0.1
tensorrt-cu12-libs        10.0.1
tensorrt-llm              0.10.0.dev2024050700
on 8*A100 40G
Who can help?
@byshiue
Information

[X] The official example scripts
[ ] My own modified scripts

Tasks

[X] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
[ ] My own task or dataset (give details below)

Reproduction
set -e

export MODEL_DIR=/mnt/memory
export MODEL_OUTPUT_DIR=/app/models/tmp
export MODEL_NAME=Mixtral-8x22B-v0.1
export LD_LIBRARY_PATH=/usr/local/tensorrt/lib:$LD_LIBRARY_PATH
export PATH=/usr/local/tensorrt/bin:$PATH
export DTYPE=bfloat16
export PYTHONPATH=/app/tensorrt-llm:$PYTHONPATH

export CUDA_VISIBLE_DEVICES=0,1,2,3

python ../llama/convert_checkpoint.py \
    --model_dir $MODEL_DIR/${MODEL_NAME} \
    --output_dir $MODEL_OUTPUT_DIR/trt_models/${MODEL_NAME}/w4a16/4-gpu-tp \
    --dtype $DTYPE \
    --use_weight_only \
    --weight_only_precision int4 \
    --tp_size 4 \
    --load_model_on_cpu

trtllm-build \
    --checkpoint_dir $MODEL_OUTPUT_DIR/trt_models/${MODEL_NAME}/w4a16/4-gpu-tp \
    --output_dir $MODEL_OUTPUT_DIR/trt_engines/${MODEL_NAME}/w4a16/4-gpu-tp \
    --gemm_plugin $DTYPE \
    --gpt_attention_plugin $DTYPE \
    --max_batch_size 1 \
    --max_input_len 2048 \
    --max_output_len 1024

Expected behavior
build success
actual behavior
trtllm-build with following error
[05/20/2024-01:09:58] [TRT] [I] Global timing cache in use. Profiling results in this builder pass will be stored.
[05/20/2024-01:10:02] [TRT] [I] [GraphReduction] The approximate region cut reduction algorithm is called.
[05/20/2024-01:10:02] [TRT] [I] Detected 13 inputs and 1 output network tensors.
[05/20/2024-01:10:03] [TRT] [E] 2: Assertion engine failed.
[05/20/2024-01:10:03] [TRT] [E] 2: [wtsEngineUtils.cpp::buildWtsEngine::104] Error Code 2: Internal Error (Assertion engine failed. )
[05/20/2024-01:10:03] [TRT-LLM] [E] Engine building failed, please check the error log.
[05/20/2024-01:10:03] [TRT-LLM] [I] Total time of building all engines: 00:00:25

additional notes
is Mixtral-8x22B-v0.1 with w4a16 supported right now? where I can get the error log?

@byshiue I know Mixtral 8x7b does not support for int4_awq now and you are working on it.
I can build Mixtral 8x7b with w4a16, so this is why I create the new issue for Mixtral 8x22b to support w4a16 here.

If you want to mention pure weight only w4a16, it is not supported, too.


Footer is overlay - not stuck to bottom of page

I don't know what causes it, but if you go to a longer page - for instance the new /settings page on #19, the footer doesn't stick to the bottom of the page, it goes to the bottom of the screen and blocks content. It should just go to the bottom of the page, and appear only when you scroll down far enough.
Idk the fix (boo css) but I'll assign it to you since you wrote it @spkane31

Closed by #39

Fixed in #45


Add labrinth cargo run documentation

Add documentation for running the API itself, which could puzzle some users.

Thanks for the contribution!


How to debug?

Backend and frontend :)
Tried using electron-debug, but I guess this is only for the frontend.
ps. I am a total newbie in this framework, I tried the obvious github repository search, google queries and a couple non-obvious.. if this info is already explained please point to the link, I'll be happy to contribute with a wiki page after I succeed :)

if you are on windows press Windows+R and then type cmd then press enter. Then navigate using the cd command (ex. cd C:/Users/yourname/Downloads/YakYak), and start yakyak with yakyak.exe. if you are on Linux, open the terminal, depending on your operating system, usually ctr+alt+t and then using the cd command (ex. cd /home/yourname/Downloads/YakYak), and start YakYak with the ./yakyak command. Then paste the log in Github.

I was asking to actually pause the process and inspect variables. Specifically in main.coffee (backend)
With the ui the inspector can be called directly in the window.

If you want an easy solution, you can log any variable using console.log anywhere. If it's a front-end variable, it will be in inspector's console. If it is backend, it will be in your terminal.

Added information to the wiki page, I guess this issue can be closed
https://github.com/yakyak/yakyak/wiki


Added RegexMatch to catch more JS files.

2 files changed.

README.md
runner/subjs/runner.go

I didn't see this PR until now...looks good!

Thanks!


Unregistered Terra user gets "unauthorized" error on Samples and Files tabs

Prior to login the data browser should check if the user is registered with Terra and, if not, redirect them through that process.

Also see Databiosphere/azul-3253.

Following up on https://github.com/DataBiosphere/azul/issues/3253#issuecomment-885244655 - the proposed solution for that will allow all registered Terra users to see public snapshots.
However, logged-in-but-not-registered-with-Terra-users would still not have access. The Data Browser should check if the user is registered with Terra and, if not, redirect them through a Terra registration workflow. A logged-in-but-not-registered user should not be able to select any of the project/samples/files tab (and so would not see the "unauthorized" error - they would only be able to logout (go back to being an anonymous user) or complete the Terra registration workflow.

##APIs to explore
Dev endpoints worth exploring: https://firecloud-orchestration.dsde-dev.broadinstitute.org/#/Profile/me (check registration status),
creating a profile (https://firecloud-orchestration.dsde-dev.broadinstitute.org/#/Profile/setProfile)
https://sam.dsde-dev.broadinstitute.org/#/Users/getUserStatus


Refactor

Hi there,
this is a somewhat heavy-handed rework - feel free to ignore, it is simply what I needed for our production environment

merge ui content into consul
binaries into /usr/sbin
data into /var/lib
add config-files and directories declarations to fpm
add jenkins BUILD_NUMBER translating into RPM iteration
add (manual) test script
script refactor
bug fixes

Thanks for the contribution. That's indeed a lot. I will take a look at it and probably do some cherry picking. Some of it are great ideas, some conflict with our own needs for the production environment, such as putting the executable into /usr/sbin.

Hi Joerg,
if the bin path is the only problem - I can compromise on that one. Main reason for the change was to root out at least one rpmlint error (consul.x86_64: E: dir-or-file-in-usr-local /usr/local/bin/consul) but it's not the end of the world. Pls let me know if there are any other showstoppers.
Steffen


virtblkioctl: Test running examples

Providing VIRTual BLocK IOCTLing test running input examples and corresponding output excerpts.

:+1: She looks good!


The order of GradeLevel is not correct

We have declared:
protected enum GradeLevel { FirstYear = 1, SecondYear, ThirdYear, FourthYear };
Why is the result in the order of SecondYear, ThirdYear, FirstYear, FourthYear?
Should it be FirstYear, SecondYear, ThirdYear, FourthYear? Is this a bug? What should I do if I want it in the declared enum order?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 728b8960-bd9b-6e47-0697-1ede60319dcc
Version Independent ID: f2080e9b-3a61-ec89-eb34-b61aff1641fa
Content: Create a nested group (LINQ in C#)
Content Source: docs/csharp/linq/create-a-nested-group.md
Product: dotnet-csharp
Technology: csharp-linq
GitHub Login: @BillWagner
Microsoft Alias: wiwagn

Never mind, I have fixed it. But the original one does have a bug. Here is the solution:
var queryNestedGroups = from student in students
group student by student.Year into newgroup
from interiorGroup in
(from orderedStudent in newgroup
group orderedStudent by orderedStudent.LastName)
group interiorGroup by newgroup.Key into outerGroup
orderby outerGroup.Key
select outerGroup;
This orderby clause will force the outer group to be ordered according to FirstYear, SecondYear, ThirdYear, FourthYear.

Thanks for posting the solution @JeffgLiu
I'll close this as answered.


enhance coverage testings

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] I have read the CONTRIBUTING document.
[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have added tests to cover my changes.
[x] All new and existing tests passed.

Coverage increased (+0.4%) to 96.986% when pulling 7f1072d93552bce94340c8336157a5d851d1860c on dev into 593a158df084f6d6941132197fe8956aa34fb16b on master.


Added checking for symtable line number to fix the issue of name conflict

The issue is that when there is a method name "top" then the return value of name would be None.
This issue will possibly be the cause of AttritbuteError: NoneType discussed in https://github.com/vitsalis/PyCG/issues/13

Nice, thanks!


i2c matster EEPROM Selective (Random) Read problem

there is problem while working with random read of EEPROM in ESP32 idf,  I have tested EEPROM by i2c scanner  and its detecting also i can write multiple byte to EEPROM, i'am facing problem while reading multiple bye from EEPROM. and its end with timeout error(error code 0x107). one interesting thing is as soon as ESP boot once it will read data then after SDA & SCL signal goes to low(i have probed in DSO) then after even write function also end-up with time out error, if we read other i2c device in same bus it will also end with timeout error. all this happening when we call repeated start after addressing required read location. and this repeated start is required here to stop write operation and start with read process. i need urgent help for this problem. i have also tested nkolban sir DS1307 RTC code in that he is following same and it will work fine. but for EEPROM its not working and i have just tested with  same nkolban RTC code and i have attached text file here.
i2c_eeprom issue.txt

@Neil Kolban sir can you help on this issue?

Paging @nkolban ... :)

@projectgus sir can you tell me solution for this problem?

Howdy @santhoshEsp32,
Can you explain more about "EEPROM" ... what EEPROM is this?    What is the nature of the I2C slave you are interacting with?   Can you attach a signal analyzer to the bus and validate that the correct I2C signals are being sent and received ... see:
https://www.youtube.com/watch?v=CE4-T53Bhu0
If you are getting timeout errors, then that means that the ESP32 I2C is expecting to read data with an acknowledgement and that acknowledgement is not happening within the timeout period.  We can look at the circuit all day long but without a trace of the signals, we can't go much further.   Fortunately logic analyzers are very cheap ... about $6 (US) ... so I strongly recommend getting one.  Without a logic analyzer, we are debugging in the dark.

Hello Kolban sir !thank you for your reply . I have used at24c32  atmel eeprom and also i have tested fm24l64 FRAM from cypres semiconductor memory in im facing same problem. Some time it will work fine . And each time when i reboot it will read once then its not gona read . As you said its problem with ack mostly if that is the problem then how it can read at 1st boot ? And yes sir i have alredy ordered logic analyser its on the way !. Once again thank you sir !!

There is another problem listed here ... https://github.com/espressif/esp-idf/issues/393
This problem sounds similar to yours.  The work around for the other problem that might be applicable to you is to always do a "ping" of the I2C target device when you start.
for example
i2c_cmd_handle_t cmd = i2c_cmd_link_create();
ESP_ERROR_CHECK(i2c_master_start(cmd));
ESP_ERROR_CHECK(i2c_master_write_byte(cmd, (i2c_add << 1) | I2C_MASTER_WRITE, 1 /* expect ack */));
ESP_ERROR_CHECK(i2c_master_stop(cmd));
i2c_master_cmd_begin(I2C_NUM_0, cmd, 1000/portTICK_PERIOD_MS);
i2c_cmd_link_delete(cmd);

Just do this one time after I2C is initialized.  The chances are that this ping request may fail but that's ok, in that subsequent I2C calls will then work.  Study issue #393 for details.

it was same issue #337 , problem got fixed !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.I tired before and that time i was not successful because supplied wrong function . thanks @nkolban .:p

@santhoshEsp32 Good news.  Make sure you follow github process and if the issue is resolved that you close out this github issue so you don't spend folk's time checking resolved issues.

@nkolban yes i was waiting for your reply !


Added 1.8 json fixture on readme

We added 1.8 fixture cbv/fixtures but it not updated in readme

Thanks :+1:


transport socket extension: gRPC TSI based ALTS support

Description:
The proposal is to upstream the ALTS support based on gRPC TSI (transport socket interface) implementation to Envoy main repo.
The current implementation have two part:

gRPC TSI based TransportSocket implementation which may be reused to implement other gRPC based transport security (e.g. TLS)
ALTS specific TransportSocketConfigFactory to instantiate the TransportSocket.

cc @alyssawilk @htuch
[optional Relevant Links:]
Existing implementation:
https://github.com/istio/proxy/tree/master/src/envoy/alts

I believe this falls neatly under our new extension policy, so I think this should be good to go.

I think this is not done yet.


First pull req for IPCHECK

Colored output checking all the playit's ips to see if any are blocked by ISPs

im not sure i did this right... im not using github too much 😭


Revert "Automated cherry pick of #8806"

Reverts mattermost/mattermost-webapp#8954
Opening this so that this is ready to go in case https://github.com/mattermost/mattermost-webapp/pull/8963 doesn't make it in by tomorrow.
I'll merge this PR tomorrow morning as needed.
cc @michelengelen

/release-note-none


Use Markdown for descriptions, observations and action texts

Aimed at fixing #56

Coverage decreased (-0.2%) to 49.958% when pulling 0611ed05b4cf226bfc8aa4b339b550fed61eea48 on markdown into ca3e47602e42b098b2e50b4f81d16e7e7280fb44 on master.

Current coverage is 43.49% (diff: 0.00%)

Merging #220 into master will decrease coverage by 0.21%

@@             master       #220   diff @@
==========================================
  Files            34         34
  Lines          1416       1423     +7
  Methods         193        193
  Messages          0          0
  Branches        167        170     +3
==========================================
  Hits            619        619
- Misses          797        804     +7
  Partials          0          0

Powered by Codecov. Last update ca3e476...0611ed0

Coverage decreased (-0.2%) to 49.958% when pulling d4b0a504ca78670661ecfd58624cb0eaf5678499 on markdown into ca3e47602e42b098b2e50b4f81d16e7e7280fb44 on master.

Coverage decreased (-0.2%) to 49.958% when pulling 72e68019b072ca7891bc255c29afc9d426e3aa53 on markdown into ca3e47602e42b098b2e50b4f81d16e7e7280fb44 on master.

Coverage decreased (-0.2%) to 49.958% when pulling 189ded824bbb798b02f39b61acbd837947bf3675 on markdown into ca3e47602e42b098b2e50b4f81d16e7e7280fb44 on master.

Coverage decreased (-0.2%) to 49.958% when pulling 4df2326df7abf929f7d4d1dff2c4b6abbc1e10d5 on markdown into ca3e47602e42b098b2e50b4f81d16e7e7280fb44 on master.

Coverage decreased (-0.2%) to 49.958% when pulling 43c1c0df22e83c9e74afdb3f792960c5f2cfa344 on markdown into ca3e47602e42b098b2e50b4f81d16e7e7280fb44 on master.


Deleted items not being removed since update

Hey @Rias500 - we updated your plugin (thanks!) the other day and we've noticed that deleted entries are not being removed from Algolia so our entry count and Algolia count is off. Any chance you can help sort? We use FeedMe to import stock and has been working perfectly in hand with Scout.

We too have noticed that disabled entries are being left in the Algolia index. I imagine these are similar issues.

Indeed a regression caused by a change in the previous release, should be fixed in 1.1.3

Awesome, thanks @Rias500


Unable to use wetty Docker container to connect to an ssh Docker container

Describe the bug
I'm unable to use the wetty web shell to connect to another Docker container that I've spun up to serve as an SSH server. I've confirmed that I can connect to the Docker container via SSH using my host, but I'm unable to access it via the wetty Docker container/web shell.
To Reproduce
Steps to reproduce the behavior:

I have the following Dockerfile:

FROM ubuntu:latest
RUN apt update && apt install  openssh-server sudo -y
RUN useradd -rm -d /home/ubuntu -s /bin/bash -g root -G sudo -u 1000 test
RUN  echo 'test:test' | chpasswd
RUN service ssh start
EXPOSE 22
CMD ["/usr/sbin/sshd","-D"]

Build the ssh Docker container with docker build -f Dockerfile -t ssh-test . and run using docker run -it -p 22:22 ssh-test

Verify that you can connect to the container via ssh (by trying to connect via ssh from the host to the container):

(base) andrew@Andrews-MacBook-Pro webshell % ssh test@localhost
test@localhost's password:
Welcome to Ubuntu 22.04 LTS (GNU/Linux 5.10.76-linuxkit x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

This system has been minimized by removing packages and content that are
not required on a system that users do not log into.

To restore this content, you can run the 'unminimize' command.

The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

test@d6858280da2a:~$

Now spin up the wetty container docker run --rm -p 3000:3000 wettyoss/wetty --ssh-host=<IP_ADDRESS> --ssh-port=22

Now access the wetty terminal at localhost:3000, and insert the test username and test password.

I get an incorrect login via wetty but I've confirmed that I'm able to ssh into the ssh-test Docker container from my host.

Expected behavior
I would expect that I can access the ssh-test container via SSH using the wetty web shell.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: macOS
Browser Chrome
Version 100.0.4896.127

The wetty shell is the equivalent to running a docker exec into a container; the shell is running inside that container, and if you try to connect to localhost, that will be localhost of the container itself (in which no ssh server is running). To connect to the port 22 that is mapped for the other container, you can try using the special host.docker.internal hostname that's available on docker desktop; https://docs.docker.com/desktop/mac/networking/#i-want-to-connect-from-a-container-to-a-service-on-the-host
Alternatively, have both containers connected to the same internal docker network, in which case you can connect with the other container using its own IP address  or hostname.

Ahhhh okay that makes sense, thank you I'll try your advice and report back. Thanks for the prompt response! 🙏

This worked exactly as you said! Thank you so much, I was banging my head against the wall for a while.

Yes it can be confusing, as localhost is not always "localhost" 😂 (you're definitely not the first one getting confused!)
Glad this helped 👍

🙏 🙏 🙏 🙏 🙏 🙏 🙏 🙏 🙏 🙏 🙏 🙏 🙏 🙏 🙏

I hope I can see this issue one hour ago😂


Tab issue with FluentWPF:Window

When fw:Window is used, it tabbing through controls break. After reaching to last control, it takes additional 4 tabs to reach the 1st control. Seems like 3 tabs entries are focusing or highlighting something thats not visible? Probably due to some isTabStop? I have added video to show the difference between Window and fw:Window and how its affecting
https://user-images.githubusercontent.com/5356702/134038171-06f48a48-7567-445e-abfd-7c2838b7a56c.mp4

Out of 3 tabs that didn't select anything I was able to remove 1 by adding isTabStop="False" to ContentControl in Window.xaml Still trying to see where else it needs to be added

Hi, @SlackingVeteran
Thanks for the information. I'll review it later 😃

Fixed by #138


Add appendAll (:++), add (+) and addAll (++) operations

With prependAll added we currently have the following membership affecting operations:

prepend (aliased as +:)
append (aliased as :+)
prependAll (aliased as ++:)
concat(aliased as ++)

For symmetry the following operations should be added:

appendAll (aliased as :++)
add (aliased as +)
addAll (aliased as ++, alias for concat)

Further, the operations add/addAll/concat could then be defined in Iterable (thus available for Set) as they do not entail any sequential positioning, while prepend/prependAll, append/appendAll could be defined in Seq since they do in fact entail sequential positioning (thus not avaliable for Set). For a specific Seq implementation concat could either be implemented as an alias for prependAll or appendAll depending on which is more efficient (e.g. the former for List and the latter for ImmutableArray).

In Set collections we have + (and its alphabetic name is incl instead of add to not be confused with the add of the Java collections -- which mutates the collection in place). I don’t think we should have + on Seq collections.
I don’t think we should have different names for ++ (concat, addAll, appendAll). It could make sense to remove ++ on Seq collections and have only :++ (appendAll) and ++: (prependAll), by symmetry with :+ and +: but that will require one more step for existing code bases to migrate to the strawman (specifically, replacing ++ with :++).

I see your point regarding add/addAll. I still see a real value in having appendAll for Seq collections though. If removing concat from Seq as you described is considered a too big a change for ease-of-migration reasons, I believe adding appendAll (aliased to :++) while still keeping concat in Seq(thus having both methods available for Seq collections) is the best choice (i.e. symmetry and ease of discovery beats DRY in this case IMHO).

I agree.  Symmetry beats DRY.  But that is more true of :++ than appendAll, I think.  concat is straightforward enough.

Closed in #184.


Renamed component not recognized

This issue has been moved from a ticket on Developer Community.

I renamed my Counter.razor component to App.razor and then updated my root component registration to refer to App instead of Counter, but App wasn't recognized:
Severity	Code	Description	Project	File	Line	Suppression State
Error	CS0246	The type or namespace name 'App' could not be found (are you missing a using directive or an assembly reference?)	BlazorApp1.Client	C:\Users\daroth\source\repos\BlazorApp1\BlazorApp1\Client\Program.cs	6	Active
The project builds cleanly in spite of this intellisense error.

Rebuilding didn't help, but restarting VS resolved the issue

Original Comments
Feedback Bot on 10/7/2021, 11:38 PM:
We have directed your feedback to the appropriate engineering team for further evaluation. The team will review the feedback and notify you about the next steps.

Original Solutions
(no solutions)

Closing as dupe of https://github.com/dotnet/razor/issues/8541


Remove errant backticks in CentOS-7/cms/wordpress-nginx.yml

Fix for #25
CC: @ryanpq

This fixes the passwords being displayed, but there is still "tr: Broken Pipe" and "tr: Write Error" along with WordPress not having the proper permissions to write to its directory.
If you were to try and delete a plugin, or upgrade WordPress (through the site) it would ask for FTP information, since it can't write. There is no option for SFTP.

Doesn't fix the issue of the WordPress directory not having the proper ownership/write permissions.
Should line 94:
- chown -Rf nginx.nginx /var/www/html/*
Look like this

chown -Rf nginx:nginx /var/www/html/*

Using a dot (".") rather than a colon (":") is actually valid:
asb@asb-do:~$ touch foo
asb@asb-do:~$ ls -la foo
-rw-rw-r-- 1 asb asb 0 Dec  1 10:01 foo
asb@asb-do:~$ sudo chown root.root foo
asb@asb-do:~$ ls -la foo
-rw-rw-r-- 1 root root 0 Dec  1 10:01 foo

I think the issue with installing plugins is different. It think define( 'FS_METHOD', 'direct' );
@ryanpq can you take a look at this?

:+1:  Looking into it.

@ryanpq do you think it would be better to create a "wp-user" and add them to the nginx group?

@ryanpq any luck with looking into the ftp socket method?


Transparent images.

I wasn't sure if I should make a new issue for this, or include it in the 1.0.0 issue. Sorry if this is the wrong spot.
Anyway, I have a png image with a transparent background that shows as transparent as the application icon, and anywhere else I use it, with the exception of this splash screen. Instead it shows the transparent layer with the white/gray grid you see in photoshop.
Do you have any idea why that might be the case?
My splashscreen options:
splashScreenOpts: {       width: 256,       height: 256,       transparent: true     }

This is the perfect location to post your issue!
Interesting. The example in this project uses an image with a transparent background, which you can see displays as expected. If you point to my example svg img in your project, do you correctly see the transparency? Also what version of electron are you using?

I am encountering the same issue. Using the latest version of electron (1.8.4). After converting my .png to a .svg format, it works correctly (although I got some size issues atm). Same behaviour occurs when using .ico file (which also has the transparent background)

On a Mac, I just tested a png with a transparent background using electron v1.8.4 and had no issue. I just pushed a temporary branch which uses a png version of my icon when running npm run example-dynamic, in case you want to see if your code is any different and if it works in your environment.
This issue is likely not an issue with this module. It isn't doing anything special for transparency, all the options are just given to electron to do the heavy lifting. For reference, transparent windows have limitations.

Thanks for the example, I just tested this on my mac (haven't tried windows, can try later today), and indeed, the dynamic one works. However, if you load the png directly in the simple splash screen as file instead of the svg, it does not work properly. I guess this is due to how electron shows the loaded template.
I will adjust my application tonight to try the dynamic splash screen.

@Tommatheussen , To be clear, it doesn't matter if you use initSplashScreen or initDynamicSplashScreen, rather it depends on the templateUrl that's given. Pointing directly to an SVG will work because how SVG work, but if you have an image like a PNG, you'll have to embed it in an HTML file. My dynamic example already had this setup so that's what I used in that branch. I'll see if I can improve my doc verbiage on this parameter.
I've thought about adding a convenience option where you provide the image instead of the templateUrl and I wrap the image in an html file for you...
For reference, this is what I'm doing with the parameter:
splashScreen.loadURL(`file://${xConfig.templateUrl}`);

So I just tested an SVG on Electron 1.8.2 (I would assume it works on 1.8.4), and it works nicely.
mainWindow = Splashscreen.initSplashScreen({
    windowOpts: windowOptions,
    templateUrl: path.join(__dirname, 'assets/icons/win/', 'icon.svg'),
    delay: 0,
    minVisible: 1500,
    splashScreenOpts: {
      width: 256,
      height: 256,
      transparent: true
    }
  })

PNG still isn't working correctly.
On 1.8.4 the PNG still doesn't work as expected, while the SVG does.
@trodi I'll try run your repo on my computer to see how it behaves.

@Xadren , if you are simply setting the templateUrl to a path to a PNG file, it won't work. templateUrl must point to an HTML (or SVG) file to work properly.

Pointing directly to an SVG will work because how SVG work, but if you have an image like a PNG, you'll have to embed it in an HTML file.

I've thought about adding a convenience option where you provide the image instead of the templateUrl and I wrap the image in an html file for you...

@trodi that makes sense. You've mentioned you thought about adding a convenience option for PNGs where you would wrap the image in an html file. That sounds like a good idea, but perhaps until that is implemented, there should be some info in the README that informs other users about the behaviour of PNGs vs SVGs - I think you mentioned doing something similar in one of your comments above.

@Xadren , I've updated the API docs API docs to further explain that an HTML file expected and to wrap a PNG.
Therefore I'll close this issue.
I mentioned the convenience option mainly to get feedback on whether if was worth doing / helpful. If you or others would like it, I'll happily add it.


"Caught handled exception, but response already started." when using sub apps

First check

[x] I added a very descriptive title to this issue.
[x] I used the GitHub search to find a similar issue and didn't find it.
[x] I searched the FastAPI documentation, with the integrated search.
[x] I already searched in Google "How to X in FastAPI" and didn't find any information.
[x] I already read and followed all the tutorial in the docs and didn't find an answer.
[x] I already checked if it is not related to FastAPI but to Pydantic.
[x] I already checked if it is not related to FastAPI but to Swagger UI.
[x] I already checked if it is not related to FastAPI but to ReDoc.
[ ] After submitting this, I commit to one of:

Read open issues with questions until I find 2 issues where I can help someone and add a comment to help there.
I already hit the "watch" button in this repository to receive notifications and I commit to help at least 2 people that ask questions in the future.
Implement a Pull Request for a confirmed bug.

Example
Here's a self-contained, minimal, reproducible, example with my use case:
https://github.com/mxab/fastapi-subapp-exceptions-show-case
When I throw an exception in a mounted sub app I get a strange error:

RuntimeError: Caught handled exception, but response already started.

I guess the question is if the exception_handler feature should be used like this, but the error is very confusing :)
Is there a better way?

class UnicornException(Exception):
    def __init__(self, name: str):
        self.name = name

app = FastAPI()

@app.get("/app")
def read_main():
    raise UnicornException(name="main")

foo = FastAPI()

@foo.get("/foo")
async def read_foo():
    raise UnicornException(name="foo")

app.mount("/foo", foo)

@app.exception_handler(UnicornException)
async def unicorn_exception_handler(request: Request, exc: UnicornException):
    return JSONResponse(status_code=418, content={"message": f"Oops! {exc.name}"},)

client = TestClient(app)

def test_read_main():
    response = client.get("/app")
    assert response.status_code == 418

# fails
def test_read_foo():
    response = client.get("/foo/foo")
    assert response.status_code == 418

Description

When I run the test
poetry install
poetry run pytest tests

The test test_read_foo which triggers the exception in the sub app fails
FAILED tests/test_app.py::test_read_foo - RuntimeError: Caught handled exception, but response already started.

Environment

OS: []: Windows WSL1 / ubuntu
FastAPI Version [e.g. 0.60.1]:

To know the FastAPI version use:

Python version:

3.6.10

I got the same problem.
I had to get rid of sub-app and it works fine.

Just use @foo.exception_handler(UnicornException)

So there is no way to capture the exception of sub-apps from the main app? I have several sub-apps but the error handling logic is the same for them, I don't think register for each app is convenient.

@hai-eb7  I had the same problem. and thanks for the thread. It seems sub-app exceptions are not handled by the main app.
What I did is register handlers to all sub-apps and it worked.

# Exception handlers
def add_exception_handlers(_app: FastAPI):
    @_app.exception_handler(ApiAuthException)
    async def api_auth_exception_handler(request: Request, exc: ApiAuthException):
        return await handler.api_auth_exception_handler(request, exc)

    @_app.exception_handler(ApiException)
    async def api_exception_handler(request: Request, exc: ApiException):
        return await handler.api_exception_handler(request, exc)

add_exception_handlers(main_app)
add_exception_handlers(sub_app)


Remove disposal of MemoryStream for email attachments sent using SendTokenizedBulkEmail

Resolves #3036
Summary
The using statement has been removed to avoid disposal of the MemoryStream prior to sending of email.

It is probably worth mentioning that this is not used anywhere in DNN.Platform or DNN.AdminExperience.  😉

I was requested as a reviewer on this, but I defer to @bdukes and @mitchelsellers opinions on this since I am no expert in this area... Ping me up if you need anything though.


Validate against Ogre 2.1 release Baldur!

See https://www.ogre3d.org/2020/04/26/ogre-2-1-baldur-released

It works "as is" just fine :-)


Update the namespaces for metric names in GitProvider receiver

Overview
We got feedback on metric namespaces from the most recent pull request that can be found here. We need to adjust the namespaces in metrics and contribute that upstream to OTEL contrib.
Definition of Done

code is adjusted to feedback
contributed and merged into our liatrio-otel-collector
contributed and merged into upstream otel-contrib
update the otel demo to reflect the metric updates (dashboard)
update documentation on adding more metrics and golden file generation

This is a contribution that needs to go to upstream immediately.

I created the PR yesterday.
Waiting on the review.

Adriel, will review today!


LOG.warning tenantIdColumn is deprecated

Bug description
I am encountering a warning when using the Tenancy package. The warning indicates that accessing a static trait property is deprecated. Below is the full log message:
LOG.warning: Accessing static trait property Stancl\Tenancy\Database\Concerns\BelongsToTenant::$tenantIdColumn is deprecated, it should only be accessed on a class using the trait in C:\laragon\www\dysrup-web\vendor\stancl\tenancy\src\Database\Concerns\BelongsToTenant.php on line 19

Actual Behavior
The following deprecation warning is logged:
LOG.warning: Accessing static trait property Stancl\Tenancy\Database\Concerns\BelongsToTenant::$tenantIdColumn is deprecated, it should only be accessed on a class using the trait in C:\laragon\www\dysrup-web\vendor\stancl\tenancy\src\Database\Concerns\BelongsToTenant.php on line 19

Steps to reproduce

Install the Tenancy package in a Laravel project.
Trigger any functionality that uses the BelongsToTenant trait.
Observe the warning in the logs.

Expected behavior
No deprecation warnings should be logged.
Laravel version
9.52.16
stancl/tenancy version
3.8.3
Environment
PHP version: 8.1
Operating System: Ubuntu 20.04

Duplicate of https://github.com/archtechx/tenancy/issues/854
Fixed in v4, can't be fixed in v3 without breaking changes.

Any hint on v4 release timeline? @stancl

Updates are posted on our Discord.


Create method for requesting ble permission

Extend IBluetoothLE with EnsurePermissionAsync() and maybe HasPermission (maybe also asnyc).

[ ] iOS
[ ] Android
[ ] Add Unauthorized feature to BLE state

Note that it does already exists the PermissionPlugin to handle all the logic related to permissions:
https://github.com/jamesmontemagno/PermissionsPlugin

@IlSocio Yes we know this. But we have to decide if we add a dependency on it or not and how :)


[feature] add e2e tests or continue-on-error for generic container

See comments from @ianlewis https://github.com/slsa-framework/slsa-github-generator/pull/1237#discussion_r1028713817

done, see https://github.com/slsa-framework/example-package/actions/workflows/e2e.container.schedule.main.continue-on-error.slsa3.yml. Closing.


[BEAM-6114] Handle Unsupported Lookup Joins in BeamSql

Added code in BeamSideInputLookupJoinRel which throws UnsupportedOperationException if "OUTER JOINS" are applied when the OUTER side is a Seekable Table.
Added Unit Tests for BeamSideInputLookupJoinRel
Modified and added Javadoc for Join Rels.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

R: @amaliujia

I was on a vocation and will take a look at this PR soon.

run SQL PostCommit


fix: use as library

Previously usage as a library (i.e. without the cli feature) was broken

@CrabNejonas friendly reminder to sign commits 😅


[Enhancement] Remember last viewed folder path

Currently, when you start up GameYob, it always shows you the root of the SD card no matter what.  So if you go into a folder named "Games" and load up a Game Boy game, then quit GameYob, and then launch GameYob again, it'll once again start you at the root of the SD card.
My suggestion then is a simple one - simply remember the last viewed folder and start there.  So using my above example, if you go into a folder named "Games" and load up a Game Boy game, then quit GameYob, and then launch GameYob again, it would instead start you right inside of the "Games" folder.
I don't know if this is a limitation of DSi homebrew or even a limitation of running DSi homebrew on a 2DS/3DS, but I can't help but notice that the likes of mGBA and Snes9X for 2DS/3DS both remember the last viewed folder path - heck, even open_agb_firm, which isn't even an emulator, remembers the last viewed folder path.

I will note here that the default path can be set in "gameyobds.ini", although having some way to select that without editing the INI would make sense in case future me decides to work on this in the future.


barSpacing variable gone

In the latest version of SwiftCharts, downloaded 27 August, you can no longer let SwiftCharts calculate barWidth by only passing barSpacing. The parameter barSpacing is gone, there is only a parameter barWidth.

Where's that? In ChartGroupedBarsLayer at least you can pass both.

This is in ChartStackedBarsLayer.swift. ChartStackedBarsLayer.swift has only barWidth, see below.
Kind regards, André Hartman
public init(xAxis: ChartAxis, yAxis: ChartAxis, innerFrame: CGRect, barModels: [ChartStackedBarModel], horizontal: Bool = false, barWidth: CGFloat, settings: ChartBarViewSettings, stackFrameSelectionViewUpdater: ChartViewSelector? = nil, tapHandler: ((ChartTappedBarStacked) -> Void)? = nil)

Dear Ivan,
ChartStackedBarsLayer only has barWidth:
init(horizontal: Bool, layer: ChartCoordsSpaceLayer, barWidth: CGFloat, viewGenerator: ChartStackedBarsLayer.ChartBarViewGenerator? = nil) {
self.stackedViewGenerator = viewGenerator
super.init(horizontal: horizontal, layer: layer, barWidth: barWidth, viewGenerator: nil)
}
Met vriendelijke groeten, Kind regards, Mes meilleures salutations,
André Hartman
Rodeweg 90
B-2920 Kalmthout
België
tel:		0032 3 666 0411
GSM:	0032 488 626 895

Op 27 aug. 2017, om 22:53 heeft Ivan Schütz<EMAIL_ADDRESS>het volgende geschreven:
Where's that? In ChartGroupedBarsLayer https://github.com/i-schuetz/SwiftCharts/blob/master/SwiftCharts/Layers/ChartGroupedBarsLayer.swift#L44 at least you can pass both.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/i-schuetz/SwiftCharts/issues/289#issuecomment-325224053, or mute the thread https://github.com/notifications/unsubscribe-auth/AAqAvFF0Ccdlvm38Wd9xX0m0tsacfgXtks5scddKgaJpZM4PDtK4.

Sorry for the long delay... and also because this is a non-answer: I honestly don't remember why it was removed and don't have time to investigate this. But you can determine the spacing indirectly by setting the width (and possibly the zoom level). Hope this doesn't cause a lot of trouble!


How do I compile and run serial_example in Ubuntu?

So far I've been using Visual Studio 2017 to compile and run my version of test_serial project (with serial_example.cc).
I would like to do the same on linux, so is there a Makefile or CMakeLists.txt file to compile and run an executable?

g++ serial_example.cc -lserial -L ../build/devel/lib -I../include -o test_serial
LD_LIBRARY_PATH=pwd/../build/devel/lib ./test_serial


Add a receipt so auth_cmd exec only runs when parameters change.

Possible solution/workaround for #678

Can we get this merged please?

Weird. The puppetlabs version is one behind this one.

Oops silly me. The puppetlabs repo is 1.1.0 and this one is 5.3.0
This one has not been updated since 2016 so it seems likely the puppetlabs repo has taken over.

Yeah.  I've made note to migrate my puppet-control.  It seems the next issue opened here would be to just ask that a notice be posted in the README and Puppet Forge directing people to the puppetlabs module as it looks like the continuation of this code base and puppetlabs is tagged it as "supported" in the forge.


Weird sawtooth cwnd in response to spurious congestion events

Hello! I am plotting the cwnd of tcp and curl+quiche over a long-running connection in the same simulation environment as I've previously described. The BDP of this connection is ~250 packets, which is the optimal cwnd. You can see that TCP grows to this cwnd, reduces it in response to congestion loss, then grows and repeats. However, when quic's cwnd reaches ~250 packets, it starts to exhibit this weird sawtooth behavior. The quic client oscillates between decreasing its cwnd based on a congestion_event and doing a rollback. This is unexpected behavior.
I think I've traced the issue to an incorrect interpretation of draft-ietf-tcpm-rfc8312bis-00 (as cited in the code):
"In cases where CUBIC reduces its congestion window in response to having detected packet loss via duplicate ACKs or timeouts, there is a possibility that the missing ACK would arrive after the congestion window reduction and a corresponding packet retransmission."
In these lines of code: https://github.com/cloudflare/quiche/blob/master/quiche/src/recovery/cubic.rs#L229-L249, a spurious congestion event is implemented as when not that many packets, in general, have been lost since the last cubic state was saved. But the RFC states that spurious congestion events are when we mark a packet as lost and start congestion recovery, but then that packet is ACKed. Does that seem right? I don't think I should be observing any spurious loss events in this simulation.
    // Detecting spurious congestion events.
    // <https://tools.ietf.org/id/draft-ietf-tcpm-rfc8312bis-00.html#section-4.9>
    //
    // When the recovery episode ends with recovering
    // a few packets (less than cwnd / mss * ROLLBACK_THRESHOLD_PERCENT(%)), it's
    // considered as spurious and restore to the previous state.
    if r.congestion_recovery_start_time.is_some() {
        let new_lost = r.lost_count - r.cubic_state.prior.lost_count;
        let rollback_threshold = (r.congestion_window / r.max_datagram_size) *
            ROLLBACK_THRESHOLD_PERCENT /
            100;
        let rollback_threshold = rollback_threshold.max(MIN_ROLLBACK_THRESHOLD);

        if new_lost < rollback_threshold {
            let did_rollback = rollback(r);

            if did_rollback {
                return;
            }
        }
    }

Actually I'm pretty sure the code here (https://github.com/cloudflare/quiche/blob/master/quiche/src/recovery/mod.rs#L535-L537) implements rollback as defined in draft-ietf-tcpm-rfc8312bis-00? Which is, it checks if an acked packet has previously been marked lost, and rolls back the cwnd accordingly.

And when I delete the "spurious congestion event" code copied in the original comment, the sawtooth behavior disappears and quic's behavior using cubic matches tcp's behavior using cubic.

It was implemented to get some perf improvement in the production network, such as mobile/wifi where spurious losses happen in the field. In your controlled environment its threshold may be too sensitive, so you may want to tune it or comment it and see (looks like you already did it). There are some differences other than that (e.g. linux tcp cubic implements hystart but quiche implements hystart++), I don't expect linux tcp cubic and quiche quic cubic behave exactly same.

To close the loop on this issue, I just meant to point out that if the goal was to implement the IETF draft referenced in the code, then I do not think that was achieved.
The draft defines a spurious congestion event as when a "missing ACK would arrive after the congestion window reduction and a corresponding packet retransmission", similar to another implementation in quiche I linked above, whereas the code here is just checking if the rate of packet loss has decreased since the last checkpoint. From when the code here was originally merged, this was the author's own detection algorithm, not following a specific standard.
If the goal was only to improve performance, sure. It significantly improves performance in my environment as well. But I was concerned this is due to the congestion control being overly aggressive and possibly unfair.


Just wondering

Do I need to put the .dll file into my bonelab mod or plugin folder? just curious. Also my fusion keeps telling me to allow access through to the networks but the prompt never pops up.

wild, same. shared problem I guess

LabFusion.dll goes in the mods folder, on quest you can ignore the LabFusionUpdater.dll. If the prompt never appears, manually go to your firewall and give FusionHelper permission (search "Allow an app through windows firewall" in your search bar).


Ability to Cancel a Future Scheduled Job

Hello,
We are planning to use Solid Queue to schedule tasks for future execution. However, we have some use cases where we might need to cancel enqueued jobs before they run.
While we could include logic within the job itself to check business conditions and decide whether to proceed, this approach would still consume unnecessary resources. Therefore, we would prefer the ability to cancel jobs before they execute.
Is there any plan to implement this feature? If so, what timeline would you anticipate for it?
Thanks in advance !

Hey @sesyilmaz, thanks for writing this question! I'm not sure I understand very well how this feature would work 🤔  As I see it, would the idea be that when the job is due for execution, Solid Queue detects it shouldn't run, and discards it? How would you specify that in a way that Solid Queue can know? I think that whatever the condition, it'd be something that consumes the same resources as if you included it in the job 🤔  I think I might be misunderstanding how you have imagined this feature to work.

Hello @rosa,
I apologize for the delayed response.
My question might not have been very clear. I wasn't looking for a way for Solid Queue to detect that it shouldn't run a job. Instead, I wanted to know how to delete, cancel or discard a job on demand if needed.
However, two of my colleagues had the chance to have a talk with you and mentioned this issue. Based on your suggestion, we will use the discard method you advised.
I will close this issue.
Thank you!

Ahhhh! I understand it now; thanks so much for clarifying and I'm sorry for the misunderstanding! Also, no worries at all about the delay 🙏


binlog ci use go1.18

master branch ci upgrade go version from go1.16 to go1.18

related to https://github.com/pingcap/tidb-binlog/pull/1142

releated to https://github.com/pingcap/tidb-binlog/pull/1142


Should I wait with merging https://github.com/packit/packit-service/pull/2588 then? :slightly_smiling_face:


chore: gate stable and add promotion action

The autoupdater pushed a partial update today, and the previous update broke the framework kmod (because it is now "upstream")
The update regression rate is too high for the current state of bazzite, for the vast majority of its users.
That being said, there are a lot of users that like to be a bit ahead.
Therefore, gate the stable builds behind an edge tag for power users, and add a small action to promote the current edge build to stable once it has been validated to work.
Just in time for F41.
The regression in this update was small: the Ally frame limiter stopped working and we were caught by surprise. It would have been worse, as with F41. The edge channel also allows us to update that channel to F41 earlier than Fedora.

With this setup the latest edge build is released (on Github) as stable https://github.com/ublue-os/bazzite/releases/tag/41.20241112 but the image has no stable tag yet.
I would also not recommend this setup as it will be confusing as to what the main branch is. There is no separation between edge and main this way. The setup that Bazzite had was in my opinion enough. I would recommend to use it as a merge tree though:
testing -> unstable -> stable
This goes only one way with the exception of hotfixes on stable which should also be merge in unstable and testing. So no pushes on main except hotfixes and merge commits. Edge is now just unstable with another name that adds complexity and confusion.

Thanks for the reply.

This got merged a tad bit too early. It should be called rolling.

I agree.

The problem it's designed to solve is that once an image is built it can't be rebuilt again

I understand.

Just because a ref got built in testing it won't be the same when it builds in stable due to dependency drift. So stable builds need to be tested before they land in the stable tag

I understand. Merging between branches doesn't necessarily mean building a new image. It could also be a promotion. I simply just prefer different git branches for rolling -> stable but git tags also works.
Thanks anyways!


iOS: App Scan GS1 Composite Code

This story is identical to the #101 only it covers the iOS Swift Scandit library.
PL user story: #93 (see full overview for additional information)
Description:
Changes to the App and decoding in order to be able to scan additional codes:

The ability to scan and decode a GS1 Composite Code

This code is used in Japan and contains the GTIN, Batch and Expiry Date

The Composite Code has the following Application Identifiers:
AI01 (GTIN)
AI10 (Batch Number)
AI17 (Expiry Date)

The rules for decoding could be:

The following a a sample code (refer also to #122 )

Acceptance criteria's:
positive tests

create a non-serialized batch. (refer to #100)
create a GS1 Composite code with a correct GTIN, Batch and Exp Date details, when scanned this should provide the ePI.

negative tests / error path testing

create a non-serialized batch.
create a GS1 Composite code with a correct GTIN, incorrect Batch and Exp Date details, when scanned this should not provide the ePI (refer to ADD REFERENCE)

Process scenario:

Japan used the GS1 Composite code for all prescription Medicines.
For non-prescription medicines Japan (like many other countries) uses a 13digit (typically) 1D barcode.
The attached document gives an overview of the different barcodes that are commonly used. This is for information purposes only.
Barcodes and Resolver v0.4.pdf

Additional information:
N/A

Testing: put the composite code in this manner with the \F separator (check the Evaluate escape sequences checkbox) https://barcode.tec-it.com/en/RSSLimitedCCA?data=05290931025615|10VC9695\F17210831


Support customized schema when output a report to a JSON file

Missing functionality

In the case of profiling a large data frame (many rows and columns),
a large (a few GBs) JSON report can be generated,
which is not convenient for further filtering based on the JSON report.
It would be great if there's a way to control the output schema off a JSON report,
so that some fields containing large data can be skipped to reduce the size of the JSON report.
There is no way to control the output schema of a JSON report.
Proposed feature

Support customizing the output schema. Allowing skipping some fields.
Alternatives considered

Additional context

Agreed, good suggestion. Could be a follow-up from #637


AKS clusters using Azure CNI should use large subnets

Rule request
Suggested rule change
Production AKS clusters should be deployed to a subnet of a minimum size to reduce IP exhaustion issues.
Subnet should be at least /23.
Applies to the following
The rule applies to the following:

Resource type: Microsoft.ContainerService/managedClusters

@BernieWhite I'd be happy to take this one on. Should be similar to the work I did before 😄.

@ArmaanMcleod Great Armaan. Reach out if you have any questions.

Thanks @BernieWhite. I guess I'll need to figure out how to get the subnet range, given only vnetSubnetID is exposed in the template. Will probably need to query this information in a separate function.
Do you know where I could create such a function, which will be exposed to the rule as well? I see a lot of functions in PSRule.Rules.Azure.psml that use Azure PowerShell, would this be the best place for it?
For testing, I could probably mock these functions to give set subnet range using Pester.
What do you think?

@ArmaanMcleod Yes currently there wouldn't be a reliable way to get the size of the subnet from within a template. Querying a REST API isn't going to work if the VNET/ subnet is not yet deployed. I would suggest that this rule wouldn't be applicable to templates.
The subnet associated to a node pool is already added as a nested resource here.
Would apply when you run Export-AzRuleData to dump a runtime configuration from Azure.
Previously, if used the -If { IsExport } as a rule pre-condition. See an examle of Azure.ACR.Usage.

@BernieWhite Ah that makes sense. Yeah limiting this rule to Export-AzRule sounds like a much better idea. I also considered the case where a subnet doesn't exist, but limiting it to just exports should solve this.
I'll let you know how I go, cheers for help mate 🙂.


Add mobilenet to benchmarks

Add mobilenet f32 and uint8 (randomly chosen between it and int8);
Expand setup script to build more;

Needed to update a dir or two post local-task renaming;
I haven't rerun this script from scratch (was updated as I went
along) as it builds quite a lot during but "it should work";

Add noxnn variant for mobilebert to check hypothesis;

Can you specify in the description that this is the manual comparison benchmarks, not our automatically populated benchmark dashboard?
Please run it from scratch before merging. YOu can do it over lunch :-)

Hehe, I knew you would ask :)

Looks good!


Move instead of borrow to avoid need to clone

This gets rids of a small TODO issue, and avoids copying three potentially large HashMaps.

Looks like delay in merge bot

Thanks for merging!


client/beefy: add some bounds on enqueued votes

When BEEFY voter runs behind rest of voters it will enqueue gossiped votes and process them as it catches up.
We should introduce some bounds on this queue to not grow forever if voter is stuck - we should instead drop votes once bounds are hit.

@acatangiu I will like to work on this

am working on it

am working on it

Oh... I already started @MrishoLukamba

Okey continue

Yes, the approach is good and you should put bounds on both pending_votes and pending_justifications.

You should also bound inner vec in pending_votes, use BoundedVec or some simple custom vec wrapper for BoundedBTreeMap<NumberFor<B>, BoundedVec<_, _>>.
Bounds sizes should be smth in the order of a few MBs for each map - just define some constants, document them and we can fine tune the value as a final detail.
For the inner votes Vec it would be great to have some way to configure bound based on authority_set size (basically make it runtime configurable instead of hardcoded const).
When bounds are reached:

votes can be dropped with a warn! log,
justifications can be dropped with a error! log (in the future we might even decide to stop the voter with "fatal" error for dropping justifications, but for now let's start with logging errors).

You should also bound inner vec in pending_votes, use BoundedVec or some simple custom vec wrapper for BoundedBTreeMap<NumberFor<B>, BoundedVec<_, _>>.
Bounds sizes should be smth in the order of a few MBs for each map - just define some constants, document them and we can fine tune the value as a final detail.
For the inner votes Vec it would be great to have some way to configure bound based on authority_set size (basically make it runtime configurable instead of hardcoded const).
When bounds are reached:

votes can be dropped with a warn! log,
justifications can be dropped with a error! log (in the future we might even decide to stop the voter with "fatal" error for dropping justifications, but for now let's start with logging errors).

Alright @acatangiu
Will open a PR soon enough


Does it support Resize the bounding box?

Once the bounding box is drawn, it is finalized. Any plan to support resizing?

+1 any support here outside of deleting and starting over?


message send error

Mail Dosen't send

bro what is command to send a new mail

is tis app running


Suggest moving the Issuer items currently under "Profile" to "Settings"

It seems the more obvious thing to put that item into Settings vs. Profile.  Suggested by @jleach .

Wondering if things related to the tenant should be in profile and anything app related in settings. I suppose the endorser stuff could be moved to configuration. I don't quite get why schema storage is in configuration but setting up the endorsers is in settings. So ya, consider moving endorser to configuration. 🤷


[SMALLFIX] Add @Nullable annotation

Add @nullable annotation for methods which may return null in /alluxio/core/server/common/src/main/java/alluxio/RpcUtils.java

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/2838/
Test PASSed.

@KSDeng Thanks for the contribution! It looks like there's an additional npm shrinkwrap file added by accident. Can you remove it? The @Nullable change looks good to me


NameError: name 'input_text' is not defined -> Tried to access input, but it was not imported. Perhaps you need from shiny.express import input?

Here’s a basic example of using a text output (reacting to changes in a text input):
from shiny.express import render, ui , input
ui.input_text("text", label="Enter some text")
@render.text
def text_out():
return f"Input text: {input.text()}"

Are you referring to the example on this page? https://shiny.posit.co/py/docs/ui-overview.html

yes.
On Thu, Aug 8, 2024 at 11:39 PM Winston Chang @.***>
wrote:

Are you referring to the example on this page?
https://shiny.posit.co/py/docs/ui-overview.html
—
Reply to this email directly, view it on GitHub
https://github.com/posit-dev/py-shiny-site/issues/199#issuecomment-2277037848,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AK6MJTRF6IT5JWGB2OOS55DZQQTVDAVCNFSM6AAAAABMHTPFJCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENZXGAZTOOBUHA
.
You are receiving this because you authored the thread.Message ID:
@.***>

Thanks, I've fixed it in e3c4d78f25675e824396759f8519400bae8581ec.


[r] Update mode semantics for implicit open

Addresses #1378:

Remove mode optarg on collection$get
Have opened children inherit parent's mode

Codecov Report
Patch coverage has no change and project coverage change: -11.53 :warning:

Comparison is base (18f0394) 65.24% compared to head (da60c73) 53.71%.

:exclamation: Current head da60c73 differs from pull request most recent head 6578437.  As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@             Coverage Diff             @@
##             main    #1383       +/-   ##
===========================================
- Coverage   65.24%   53.71%   -11.53%
===========================================
  Files          98       68       -30
  Lines        7938     5507     -2431
===========================================
- Hits         5179     2958     -2221
+ Misses       2759     2549      -210

Flag
Coverage Δ

python
?

r
53.71% <ø> (-0.05%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.
see 44 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


TF2 porting: Date encoder method docstring update, unit test refinement

Code Pull Requests
From what I can tell the underlying class parameters are surfaced.  I noticed a mis-match between parameters defined in __init__() method and docstring.  This PR updates docstring to match the __init()__ parameters.
I updated the test_experiment_date unit test to make use of parameterize decorator.

Assuming this PR is OK, per your earlier suggestion, I'll look at this as the next task:

I think the highest priority is to restore the functionality of the training loop, like validation, early stopping and the like, that we removed. Luckily now it will be much simplified because collecting metrics is much cleaner and simpler now.

looks good, thank you!

I noticed this PR was Closed instead of Merged.     This means the proposed docstring updates and change to test_experiment_date are not going to be incorporated.  What were the issues with the proposed changes.
Before I delete the branch on my side, I wanted to confirm that is intent.

Just wrong button

np...thank you.


The example "use key to control the preservation of elements and components" uses erroneous syntax

The example uses the wrong syntax:

To modify the People component to use the @key directive attribute with the people collection, update the  element to the following:

<Details @key="person"<EMAIL_ADDRESS>/>

It should be:

    <Details<EMAIL_ADDRESS>/>

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 81a32115-33f8-7d06-aaf0-b8ce918b5527
Version Independent ID: 31ad18df-1904-bbdd-cfed-8d0d0267c10e
Content: ASP.NET Core Razor components
Content Source: aspnetcore/blazor/components/index.md
Product: aspnet-core
Technology: aspnetcore-blazor
GitHub Login: @guardrex
Microsoft Alias: riande

Thanks @konstantinovaleksej ...There's a 😈 in the description ... a missing remark on this aspect ... because if not using a list (

), it works as described. If using a list, Yes!, it seems to require the @key on the list item tag (
).
Ping @SteveSandersonMS ... Your original example went like this ...

Consider the existing situation where you don't have this feature:

    @foreach (var flight in Flights)
    {
        <DetailsCard Flight="@flight" />
    }

... {ADDITIONAL REMARKS ON WHAT'S HAPPENING} ...
Fixing this with 'key'
The developer should be able to make their app more efficient (and sometimes, better behaving) using key like this:

    @foreach (var flight in Flights)
    {
        <DetailsCard key="@flight" Flight="@flight" />
    }

Cross-reference: https://github.com/dotnet/AspNetCore.Docs/issues/12755#issuecomment-500803582
BUT ...
The doc example doesn't work ...

    @foreach (var person in people)
    {

    }

As @konstantinovaleksej points out, the @key must be on the list item ...

    @foreach (var person in people)
    {

    }

This is also the case if the doc example is refactored to pass the Person instead of merely passing the Person.Data property (that would make the doc example exactly match your example).
If the ordered list is dropped (🔪 the

) ...
@foreach (var person in people)
{
    <Details @key="person"<EMAIL_ADDRESS>/>

}

... it ✨ Just Works!™ ✨ with @key on the Details component.
Also if passing-the-Person and the ordered list is dropped ...
@foreach (var person in people)
{

}

... it also ✨ Just Works!™ ✨
@SteveSandersonMS ... We need to flesh out what's going on with the list and the position of the @key directive. Why is it required on
 in the former case with a list but can be used directly on the Details component in the latter case without a list?

I've made a quick patch 🏃 to the topic's example to remove the ordered list. It just goes with ...
@foreach (var person in people)
{
    <Details<EMAIL_ADDRESS>/>
}

... now, which is fine until I have a better understanding on why the

 breaks the example if not keyed on
.

We need to flesh out what's going on with the list and the position of the @key directive. Why is it required on  in the former case with a list but can be used directly on the Details component in the latter case without a list?

@key is not globally scoped (otherwise you'd get random clashes all over the place). It's only scoped to its own siblings within its parent. So if you have:

...

...

... then the first and second keys are compared against each other. But if you have:

...

...

... then the two keys are completely unrelated and have no influence on each other.
Hopefully that explains why this achieves something useful:
@foreach (var person in people)
{
    <Details @key="person"<EMAIL_ADDRESS>/>
}

... and so does this:
@foreach (var person in people)
{

        <Details<EMAIL_ADDRESS>/>

}

... but this does not:
@foreach (var person in people)
{

        <Details @key="person"<EMAIL_ADDRESS>/>

}


[SUGGESTION] Add a feature to redirect ROBLOX to use the UWP version of it.

becus yes yes yes (explained by the title)
the paragraph below is a generated using chatgpt on why you should add this feature

Adding UWP redirection of ROBLOX can greatly enhance the user experience, increase user engagement and retention, differentiate an app from competitors, and future-proof an app. With so many benefits to adding UWP redirection, it's a smart business decision for anyone looking to improve their ROBLOX app and stand out in a crowded market.

What? Why don't you just install the UWP version then?

What? Why don't you just install and use the UWP version then?

i just click play on a link instead of searching the frick out of it and figuring out where is the game if im using UWP

I can't really do anything about that, and it would defeat the entire purpose of Bloxstrap as it does not apply to the UWP version. Why are you using the UWP version anyway? What's wrong with just using the normal Windows version?


swagger::ByteArray and string:binary.

Hi guys,
I am not enirely sure if it's an issue. So, I would like to get your opinion.
Open API spec says that binary strings are not base64-encoded.
https://swagger.io/specification/
string | byte | base64 encoded characters
string | binary | any sequence of octets
But rust-server codegen uses swagger::ByteArray for both, binary and byte formats.
https://github.com/swagger-api/swagger-codegen/blob/master/modules/swagger-codegen/src/main/java/io/swagger/codegen/languages/RustServerCodegen.java#L120-L121
As far as I understand swagger::ByteArray is always base64-encoded. Am I missing something here?

Thanks for raising!  I think there's a bug, but I'm not quite sure what the right behavior is...
On the Rust API, I don't think a developer should care whether this data will be serialized as base64 or binary, hence both map to swagger::ByteArray.
I suspect the bug here is that, when serialized to JSON, we serialize as base64 whether the specification specifies byte or binary.  However, I don't think JSON defines a way of passing raw bytes (e.g. " would need to be escaped in some way), so I'm not sure it's really meaningful to specify binary for a field in a JSON body.  What do you think?
Binary fields that are included directly as part of the HTTP body (e.g. via a multipart/* encoding) should be encoded correctly (although worth checking that they actually are).
BTW, all active Rust development takes place on the openapi-generator fork rather than the swagger-codegen project you refer to.  See the Q&A for reasons.

Just noticed https://github.com/OpenAPITools/openapi-generator/issues/3189, which is related.

Suggested fix to use Vec<u8> instead makes sense to me.  I guess it's swagger-rs issue.
Also, it's worth mentioning that JSON is not the only serialization format: e.g. CBOR is binary one.

Good point about CBOR... but I don't think openapi-generator for Rust supports that at all!

Since it uses serde for serialization it's actually straightforward to extend that.


Structured configs : allow MISSING as a value

make this work:
@dataclass
class Config(DictConfig):
    db1: DBConfig = MISSING

This is now supported as of #167, as well as many other use cases with interpolation, missing, optional, None and combinations of those, for all supported OmegaConf node types.


Allow sending and receiving raw protobuf structs

I found more things I couldn't do with the custom message types, such as replying to messages and mentioning people.
Last time you said you were going to ditch the custom message types, so here's a pull request that adds the option to use raw protobuf structs.
I also exported the download and upload functions so that downloading and uploading stuff without the custom message types would be possible.

Thanks! I think it's a good way to handle every kind of message for now. I am not totally sure how and if we should continue on the dispatching pattern we got at the moment or if we should ditch them completely.


[Towards MicroProfile 4.0] Blog update

@rstjames,
Let's adjust the future media pushes related to blog on MP 4.0: https://twitter.com/MicroProfileIO/status/1267484566731796481?s=20
Can this be completed before next week's media reminder? Thanks
Design:

(design wording) MicroProfile 4.0 Release is delayed due to the MicroProfile Working Group requirement.

Media description:
#MicroProfileRelease 4.0 is delayed due to the @MicroProfileIO Working Group Requirement.
#MicroProfilers💛 working on the APIs should continue their work towards Milestone 1. Read more HERE: https://bit.ly/36NV21n
#OpenSource #OSS

@aeiras
all set till end of July.

perfect Ryan, thank you!


Using overlayWith before a rotate call throws an error

I'm working on a project where I need to extract a piece of an image and then mask the extracted image with a polygon shape and then rotate it. I am using an SVG string buffer as the mask.
When I run this line of code with rotate() I get an error:
"Overlay image must have same dimensions or smaller"
sharp(imageData)
    .extract({left: left, top: top, width: width, height: height})
    .overlayWith(mask, {cutout: true})
    .flip(isMirrored)
    .rotate(rotation)
    .png()
    .toFile(filePath);

When I run it without the rotate call, it runs without error. I am not sure if this is an actual bug or if I may be doing something wrong. I am assuming that the overlayWith is happening after the rotation and that may be what is causing the error.
I also want to thank you for creating this library. It is brilliant!

Hello, your guess is correct, the overlay is applied at the end. You'll probably need to split this into two pipelines, something like:
sharp(imageData)
  .extract({left: left, top: top, width: width, height: height})
  .overlayWith(mask, { cutout: true })
  .raw()
  .toBuffer({ resolveWithObject: true })
  .then(({ data, info }) => sharp(data, { raw: info })
    .flip(isMirrored)
    .rotate(rotation)
    .png()
    .toFile(filePath)
  );

Ah thanks! I will do it that way.


Namespace EventSystem not found

Describe the bug
I have been trying to get started with the MixedRealityToolkit-Unity and am noticing a bug in the setup process. After importing the MixedRealityToolkit 90+ compiler errors pop up. Most of them say "type or namespace 'Evensystems' could not be found", but it appears for other namespaces aswell like: BaseEventData, IEventSystemHandler, RaycastResult and a couple others. I am using Unity 2019.3.0a3 and have setup a project with buildsettings for the Hololens (v1)
To reproduce
Steps to reproduce the behavior:

Create a new Unity Project.
Click on Assets -> Import Package
Import the MixedRealityToolkit.
See the errors pop up in the console.

Expected behavior
The MixedRealityToolkit is loaded in Assets. A dropdown menu for the toolkit should appear (among other setting changes).
Screenshots
Your Setup (please complete the following information)

Unity Version [2019.3.0a3]
MRTK Version [ v2.0.0-RC1-Refresh ]

Target Platform (please complete the following information)

HoloLens

Unity Version [2019.3.0a3]

Our officially supported version is currently Unity 2018.3/.4. It's possible there are some breaking changes in Unity we'll need to account for when we add support for 2019.3, but we can't guarantee the MRTK will work in an alpha release of Unity.

Ah, Thank you! The package is indeed not compatible with Unity 2019.3.x (yet), but after Installing the 2018.3.x version of Unity all problems were resolved.

Thanks for the answer ! I'll be using Unity 2018.3.x for the project, thanks for your time !

Does not work for 2019.2 as well. Would be good to keep this issue open, and maybe update the README.MD to indicate the range of Unity version that it would work.

Closing this one out - we're clear in our getting started about using 2018.4.x, and we can update this when we've done the 2019.3 reconciliation


Left Nav Bar Multi-Level Menu Collapse Issue

Steps to regenerate Issue

Click on Menu Toggle (menu_toggle) Nav needs to be 'nav-sm'
Click on Multilevel Menu
Click on 2nd Level One menu item
The Whole Menu collapses before opening the level two (Problem)

Thank you @satbir90 for pointing this out, we are already aware of this issue  and we will solve it as soon as possible.


MS: Master Thesis Dev

Note MS: This is just a temporal branch for all the changes I make during my master thesis. This branch should not be merged.

PR can be found here: #944


[Documentation Suggestion] better menu doc to assist new users

As suggested by chrisib in discord:

having a "here's how to use the menu" section in https://github.com/Allen-Synthesis/EuroPi/blob/main/software/programming_instructions.md would be nice? they specifically call out the contrib package and have a section dedicated to saving menu.py as main.py, but there's no explanation of what buttons and knobs to use to do anything.
similarly, the menu doesn't seem to be documented at all on the main page of https://github.com/Allen-Synthesis/EuroPi/tree/main/software/contrib

https://github.com/Allen-Synthesis/EuroPi/pull/219

This can probably be closed now since the relevant changes have been merged.


"end_to_end_test.sh local" fails with "AndroidLint failed to run"

The linter binary should be at /android-sdk-linux/tools/lint, but it's disappearing during the Docker build:

(Oops, submitted by accident somehow)
Here's the error that's causing the dir to disappear:
$ docker run -i -t e38c093793c8
root@5bf2ee09e291:/# android update sdk --no-ui --filter tools
...
Installing Archives:
  Preparing to install archives
  Downloading Android SDK Tools, revision 24.1.2
  Installing Android SDK Tools, revision 24.1.2
  Failed to rename directory /android-sdk-linux/tools to /android-sdk-linux/temp/ToolPackage.old01.
  Failed to create directory /android-sdk-linux/tools
  Done. Nothing was installed.
root@5bf2ee09e291:/# ls -l /android-sdk-linux/tools
ls -l /android-sdk-linux/tools
total 0

A relevant SO thread: http://stackoverflow.com/questions/8839255/failed-to-rename-directory-tools-to-temp-toolpackage-old01-in-c-android-sdk-win
None of the top answers helped, but just unzipping the tools ZIP myself seems to work.

Here's the issue in the Android SDK bug tracker:
https://code.google.com/p/android/issues/detail?id=4410

We're now seeing similar failures on Jenkins:
https://jenkins-dot-shipshapecode.appspot.com/job/shipshape_end-to-end_test/5/
This seems to require a slightly different (and incompatible) workaround...
Inside the Java dev container, we get
# docker exec android_lint_0 ls /android-sdk-linux/tools/
tools
# docker exec android_lint_0 ls /android-sdk-linux/tools/tools/
NOTICE.txt
android
ant
apps
...

The culprit seems to be this line in the Dockerfile:
RUN unzip /android-sdk-linux/temp/tools_*-linux.zip -d /android-sdk-linux/tools

Somehow this behaves differently on the Jenkins slave than it does on Travis.

 The problem here isn't Travis, it's the local end_to_end_test.sh script.
Travis is passing with the fix: https://travis-ci.org/google/shipshape/builds/60447010
The local script fails with a different error:
WARNING: Analyzer AndroidLint failed to run: The triggered category "AndroidLint" could not be found at the locations [localhost:10005 localhost:10006]

This is happening because the end-to-end test is starting a shipping container without any analyzer services (i.e. not AndroidLint container) to run the first couple of tests (JSHint etc.). Then a second run is started where shipshape is called with an analyzer service as a flag. However, now a shipping container is running (not killed at end of last run due to the default of the stay_up flag being true) and it has not been started with the analyzer service. To make the shipping container aware of a new analyzer service we need to restart it if a new service is given when shipshape is started.

There is now a check which restarts the shipping container if it does not have a link to needed analyzer containers (like AndroidLint). Closing this issue.


Updated the README.md to use correct script name

The instructions provided for running the parser refer to the main script by the wrong filename.  I have updated it from bittorrent-parser.py, which does not exist in the project, to parse-torrent.py.

Thanks aagallag


[C++][ctest]Test failed within solution configuration Release|x64

Describe the bug, including details regarding any error messages, version, and platform.
MSVC team uses Arrow as part of RWC testing to help optimize compilers. Recently, when we were testing, it seems that the test was failing on several items.Could you plz take a look?Thanks.
The following tests FAILED:
72-parquet-internals-test (Failed)
73-parquet-reader-test (Failed)
75-parquet-arrow-test (Failed)
Steps to repro

Open VS2022 x64 command prompt.
cd /d C:\gitP\apache\arrow\cpp\build_amd64
set ARROW_TIMEZONE_DATABASE=C:\gitP\apache\arrow\tzdata
ctest -C Release --output-on-failure -E "arrow-array-test" 2>&1

Expect
100% tests passed.
Actual result
[  FAILED  ] 10 tests, listed below:
[  FAILED  ] TestArrowReadWrite.FuzzReader
[  FAILED  ] TestArrowReaderAdHoc.Int96BadMemoryAccess
[  FAILED  ] TestArrowReaderAdHoc.CorruptedSchema
[  FAILED  ] TestArrowReaderAdHoc.ReadFloat16Files
[  FAILED  ] TestFromParquetSchema.CorruptMetadata
[  FAILED  ] ReadDecimals/TestArrowReaderAdHocSparkAndHvr.ReadDecimals/0, where GetParam() = ("int32_decimal.parquet", (ptr = 000001FC6AAE0720, value = decimal128(4, 2)))
[  FAILED  ] ReadDecimals/TestArrowReaderAdHocSparkAndHvr.ReadDecimals/1, where GetParam() = ("int64_decimal.parquet", (ptr = 000001FC6AAE08E0, value = decimal128(10, 2)))
[  FAILED  ] ReadDecimals/TestArrowReaderAdHocSparkAndHvr.ReadDecimals/2, where GetParam() = ("fixed_length_decimal.parquet", (ptr = 000001FC6AAE1600, value = decimal128(25, 2)))
[  FAILED  ] ReadDecimals/TestArrowReaderAdHocSparkAndHvr.ReadDecimals/3, where GetParam() = ("fixed_length_decimal_legacy.parquet", (ptr = 000001FC6AAE1130, value = decimal128(13, 2)))
[  FAILED  ] ReadDecimals/TestArrowReaderAdHocSparkAndHvr.ReadDecimals/4, where GetParam() = ("byte_array_decimal.parquet", (ptr = 000001FC6AAE0AA0, value = decimal128(4, 2)))

P.S. To see the detail info,I have attached the build log and test log as attachments.If you need more information, please leave a message under this issue
Build (4).log
Test (1).log
Test.log.4.log
Component(s)
C++, Continuous Integration

unknown file: error: C++ exception with description "Please point the PARQUET_TEST_DATA environment variable to the test data directory" thrown in the test body.

Could you also set PARQUET_TEST_DATA environment variable?
See also: https://arrow.apache.org/docs/developers/cpp/development.html#apache-parquet-development

unknown file: error: C++ exception with description "Please point the PARQUET_TEST_DATA environment variable to the test data directory" thrown in the test body.

Could you also set PARQUET_TEST_DATA environment variable?
See also: https://arrow.apache.org/docs/developers/cpp/development.html#apache-parquet-development

Sorry, we did not set the environment variables. We will try to set the parameters according to the content in the link and try to rebuild and test.

Is that In addition, I would like to ask whether the parquet test is necessary. If we set -DARROW_PARQUET to OFF when executing cmake, will the parquet test be skipped? Because after discussion, we believe this is an open source project issue rather than a compiler issue. @kou

Yes. Parquet tests must be skipped.

Noted,thanks for your help!

Is that In addition, I would like to ask whether the parquet test is necessary. If we set -DARROW_PARQUET to OFF when executing cmake, will the parquet test be skipped?

In addition to @kou 's answer, while you can definitely disable some Arrow components, Parquet is a very important and widely-used component (and its performance is quite important as well), so it would probably interesting for you to keep it updated?


Added https redirect example for PHP on NGINX/AL2

Issue #, if available:
Description of changes:
New Amazon Linux 2 PHP platforms use NGINX as the web server. No existing HTTPS redir examples for this platform branch.
Ebextensions example added for enabling HTTPS redirection for all HTTP requests, unless they come from the ELB healthchecker. Note that this does not require a rewrite of the default EB NGINX .conf file.
Feedback welcome!
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@JohnZProd Please resubmit under the community directory (https://github.com/awsdocs/elastic-beanstalk-samples/tree/master/configuration-files/community-provided/security-configuration) and I'm happy to merge.

Thanks @dankhen moved to community directory with a brief summary


Segmentationu error

Standard error:
Using antsBrainExtraction with the following arguments:
  image dimension         = 3
  anatomical image        = /data/sub-TAURESTMCIm024S4674/anat/sub-TAURESTMCIm024S4674_T1w.nii.gz
  extraction template     = /niworkflows_data/ants_oasis_template_ras/T_template0.nii.gz
  extraction reg. mask    = /niworkflows_data/ants_oasis_template_ras/T_template0_BrainCerebellumRegistrationMask.nii.gz
  extraction prior        = /root/src/fmriprep/work/fmriprep_wf/single_subject_TAURESTMCIm024S4674_wf/anat_preproc_wf/skullstrip_ants_wf/t1_skull_strip/T_template0_BrainCerebellumProbabilityMask.nii.gz
  output prefix           = highres001_
  output image suffix     = nii.gz

N4 parameters (pre brain extraction):
  convergence             = [50x50x50x50,0.0000001]
  shrink factor           = 4
  B-spline parameters     = [200]

Atropos parameters (extraction):
   convergence            = [3,0.0]
   likelihood             = Gaussian
   initialization         = kmeans[3]
   mrf                    = [0.1,1x1x1]
   use clock random seed  = 1

/usr/lib/ants/antsBrainExtraction.sh: line 117:   849 Segmentation fault      (core dumped) $cmd
Return code: 1
I. always get this error from different subjects in my study. How can I avoid that?
thank you,
Tragus

Looks like a memory error similar to what you reported last time https://github.com/poldracklab/fmriprep/issues/961. Maybe docker limits got reset?

Hi @tragus1, any progress troubleshooting this?

Please reopen if this problem comes back.


Deleting RKE template does not reliably delete revisions and leaves orphans

What kind of request is this (question/bug/enhancement/feature request):
Bug
Steps to reproduce (least amount of steps as possible):

Create 10 RKE templates, test1, test2, test3, ..., test10. Can leave all settings as default.
Delete the RKE template (note, template, not the revision).

Result:
1 of the 10 RKE templates I removed left an orphaned rke template revision:

Ran on kubectl console in local cluster:
> kubectl get clustertemplates -A
No resources found

> kubectl get clustertemplaterevisions -n cattle-global-data ctr-2fzn5 -o yaml
apiVersion: management.cattle.io/v3
kind: ClusterTemplateRevision
metadata:
  annotations:
    field.cattle.io/creatorId: u-emk3r7goqi
  creationTimestamp: "2020-04-29T22:22:05Z"
  generateName: ctr-
  generation: 3
  labels:
    io.cattle.field/clusterTemplateId: ct-9mbzx
  name: ctr-2fzn5
  namespace: cattle-global-data
  ownerReferences:
  - apiVersion: management.cattle.io/v3
    kind: ClusterTemplate
    name: ct-9mbzx
    uid: 84c8326a-7afb-4a28-b52b-3262affbedf3
  resourceVersion: "10529982"
  selfLink: /apis/management.cattle.io/v3/namespaces/cattle-global-data/clustertemplaterevisions/ctr-2fzn5
  uid: 84449a06-233d-4470-97e5-a3270faca5fb
spec:
  clusterConfig:
    agentImageOverride: ""
    desiredAgentImage: ""
    desiredAuthImage: ""
    dockerRootDir: /var/lib/docker
    enableClusterAlerting: false
    enableClusterMonitoring: false
    enableNetworkPolicy: false
    localClusterAuthEndpoint:
      enabled: true
    rancherKubernetesEngineConfig:
      addonJobTimeout: 30
      authentication:
        strategy: x509
      authorization: {}
      bastionHost: {}
      cloudProvider: {}
      dns:
        nodelocal:
          ipAddress: ""
      ignoreDockerVersion: true
      ingress:
        provider: nginx
      kubernetesVersion: v1.17.4-rancher1-3
      monitoring:
        provider: metrics-server
        replicas: 1
      network:
        options:
          flannel_backend_type: vxlan
        plugin: canal
      restore: {}
      services:
        etcd:
          backupConfig:
            enabled: true
            intervalHours: 12
            retention: 6
            s3BackupConfig: null
          creation: 12h
          extraArgs:
            election-timeout: "5000"
            heartbeat-interval: "500"
          retention: 72h
          snapshot: false
        kubeApi:
          serviceNodePortRange: 30000-32767
        kubeController: {}
        kubelet: {}
        kubeproxy: {}
        scheduler: {}
      sshAgentAuth: false
      systemImages: {}
      upgradeStrategy:
        maxUnavailableControlplane: "1"
        maxUnavailableWorker: 10%
        nodeDrainInput:
          gracePeriod: -1
          ignoreDaemonSets: true
          timeout: 120
    scheduledClusterScan: {}
    windowsPreferedCluster: false
  clusterTemplateName: cattle-global-data:ct-9mbzx
  displayName: v1
  enabled: true
  questions:
  - default: v1.17.4-rancher1-3
    type: string
    variable: rancherKubernetesEngineConfig.kubernetesVersion

Other details that may be helpful:
Appears the more revisions you have in your RKE template, the more likely this is to happen.
Environment information

Rancher version (rancher/rancher/rancher/server image tag or shown bottom left in the UI):
v2.4.2
Installation option (single install/HA):
HA

Cluster information

Cluster type (Hosted/Infrastructure Provider/Custom/Imported):
Machine type (cloud/VM/metal) and specifications (CPU/memory):
Kubernetes version (use kubectl version):

> kubectl version
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.8", GitCommit:"ec6eb119b81be488b030e849b9e64fda4caaf33c", GitTreeState:"clean", BuildDate:"2020-03-12T21:00:06Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.4+k3s1", GitCommit:"3eee8ac3a1cf0a216c8a660571329d4bda3bdf77", GitTreeState:"clean", BuildDate:"2020-03-25T16:13:25Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"}

Docker version (use docker version):
NA, containerd

Reproduced the issue in Rancher v2.4.3 on a HA setup
Steps:
Using a python test script, created 200 RKE templates with each template having 10 revisions in them. Deleted the templates
After all templates were deleted, few revisions were left around.
> kubectl get clustertemplates -A
No resources found
> kubectl get clustertemplaterevisions -n cattle-global-data
NAME        AGE
ctr-2k7gn   26m
ctr-4wsts   21m
ctr-6qhnt   19m
ctr-78b6d   24m
ctr-7rpz4   21m
ctr-8nht9   18m
ctr-jb2mw   24m
ctr-jbjdf   28m
ctr-k6z4j   23m
ctr-kf2qv   25m
ctr-kkd6k   25m
ctr-nj6mp   24m
ctr-qtv9n   28m
ctr-qxqhq   26m
ctr-r4rj8   26m
ctr-rhblv   19m
ctr-zjh9p   28m
>

 kubectl get clustertemplaterevisions -n cattle-global-data ctr-2k7gn -o yaml
apiVersion: management.cattle.io/v3
kind: ClusterTemplateRevision
metadata:
  annotations:
    field.cattle.io/creatorId: user-wxkz5
  creationTimestamp: "2020-05-08T21:24:53Z"
  generateName: ctr-
  generation: 1
  labels:
    io.cattle.field/clusterTemplateId: ct-hlg7r
  name: ctr-2k7gn
  namespace: cattle-global-data
  ownerReferences:
  - apiVersion: management.cattle.io/v3
    kind: ClusterTemplate
    name: ct-hlg7r
    uid: 1acc1fc8-bd7b-41d4-969f-007cbfd76314
  resourceVersion: "55967"
  selfLink: /apis/management.cattle.io/v3/namespaces/cattle-global-data/clustertemplaterevisions/ctr-2k7gn
  uid: 65ffc857-97fa-4173-89dc-3ab2fd83eee7
spec:
  clusterConfig:
    dockerRootDir: /var/lib/docker123
    enableClusterAlerting: false
    enableClusterMonitoring: false
    enableNetworkPolicy: false
    localClusterAuthEndpoint:
      enabled: true
    rancherKubernetesEngineConfig:
      addonJobTimeout: 30
      authentication:
        strategy: x509
      ignoreDockerVersion: true
      ingress:
        provider: nginx
      kubernetesVersion: v1.15.11-rancher1-3
      monitoring:
        provider: metrics-server
        replicas: 1
      network:
        mtu: 0
        options:
          flannel_backend_type: vxlan
        plugin: canal
      services:
        etcd:
          backupConfig:
            enabled: true
            intervalHours: 12
            retention: 6
            s3BackupConfig:
              accessKey: AKIAJ7ITR7F2MWQZZDRA
              bucketName: test-auto-s3
              endpoint: s3.amazonaws.com
              secretKey: rYn/8WfMbYRwk+SM64h6sAtRHbHkjpY9fYIB/g8Q
            safeTimestamp: false
          creation: 12h
          extraArgs:
            election-timeout: "5000"
            heartbeat-interval: "500"
          gid: 0
          retention: 72h
          snapshot: false
          uid: 0
        kubeApi:
          alwaysPullImages: false
          podSecurityPolicy: false
          serviceNodePortRange: 30000-32767
      sshAgentAuth: false
    windowsPreferedCluster: false
  clusterTemplateName: cattle-global-data:ct-hlg7r
  displayName: revision1-53426
  enabled: true

Clicking on the revision takes us to the below link

This seems to be happening only for HA setups. And it is very hard to reproduce this issue; @soumyalj can you update how longer it took for you to actually hit this state?

@prachidamle : It tool me 3 attempts to reproduce the bug on a HA setup. Used the python test to reproduce as mentioned above.

@soumyalj Also what was your dataset? How many templates/revisions reproduced it?

This seems to be a k8s garbage collection issue related to cleanup of resources in HA setups based on owner references set.
The v3.ClusterTemplate resource is the owner of v3.ClusterTemplateRevisions and both are present in the same namespace - "cattle-global-data"
The v3.ClusterTemplateRevision resource has the owner reference as set here: https://github.com/rancher/rancher/blob/master/pkg/api/store/clustertemplate/store.go#L171
        var ownerReferencesSlice []map[string]interface{}
	ownerReference := map[string]interface{}{
		managementv3.OwnerReferenceFieldKind:       "ClusterTemplate",
		managementv3.OwnerReferenceFieldAPIVersion: "management.cattle.io/v3",
		managementv3.OwnerReferenceFieldName:       templateName,
		managementv3.OwnerReferenceFieldUID:        template.UUID,
	}

Thus each v3.ClusterTemplateRevision points to its parent v3.ClusterTemplate.
When user deletes a ClusterTemplate, we specify the delete option metav1.DeletePropagationBackground here https://github.com/rancher/norman/blob/release/v2.4/store/proxy/proxy_store.go#L530
Background policy which makes Kubernetes delete the owner object immediately and the garbage collector then deletes the dependents
https://kubernetes.io/docs/concepts/workloads/controllers/garbage-collection/#background-cascading-deletion
But in this case it looks like rarely Kubernetes GC does not cleanup some of the child ClusterTemplateRevision objects. Those which remain and orphaned since the parent ClusterTemplate is deleted, also they do not have the deletionTimestamp set.
Seems like some race condition with HA setups causing GC to not cleanup all child resources.

@prachi: Created 200 templates with 10 revisions in each.

Further updates:
Tested kubectl delete on an HA setup having 200 ClusterTemplates and 2000 ClusterTemplateRevisions and this issue reproduced!
> kubectl get clustertemplates -A
No resources found
> kubectl get clustertemplaterevisions -A
NAMESPACE            NAME        AGE
cattle-global-data   ctr-2mgzd   179m
cattle-global-data   ctr-7hd7n   177m
cattle-global-data   ctr-bms8s   3h3m
cattle-global-data   ctr-svg2d   176m
cattle-global-data   ctr-w52zv   3h5m
cattle-global-data   ctr-zzhwl   176m

Thus this issue does not seem to be related to rancher code...

The root cause of this issue is invalid OwnerReferences used by RoleBindings created pointing to a ClusterTemplate.
The OwnerReferences were using wrong "kind" and "apiVersion" for these resources.
Example for ClusterTemplate:
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  creationTimestamp: "2020-06-04T21:06:26Z"
  labels:
    cattle.io/creator: norman
  name: ct-nlcgc-ct-a
  namespace: cattle-global-data
  ownerReferences:
  - apiVersion: management.cattle.io
    kind: clustertemplates
    name: ct-nlcgc
    uid: 0eb0f6ac-6e91-4a04-b9cf-8693bb702f36
  resourceVersion: "25988"
  selfLink: /apis/rbac.authorization.k8s.io/v1/namespaces/cattle-global-data/rolebindings/ct-nlcgc-ct-a
  uid: 7f7c1263-9e01-4859-961e-099f532c0a65
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: ct-nlcgc-ct-a
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: User
  name: user-gjrll

The ownerreference here is wrong. The apiVersion is wrong (it is missing the version v3) and the kind is set to the resource name (clustertemplates) instead of kind (ClusterTemplate)
This caused the issue where after deleting a ClusterTemplate, the dependent ClusterTemplateRevisions were not deleted and orphaned because the Kubernetes GC was stuck in a loop trying to delete the dependent Role/Rolebinding that were using invalid OwnerReferences. So GC never finished cleaning up the bindings or the ClusterTemplateRevisions either leading to the orphaned data.
This also occurs when the Kubernetes GC hits a specific race condition hence could only be reproduced sometimes with large data.

Upgrade impact:

There will not be any upgrade step for existing clusters.

We wont correct the rolebindings on existing clusters. Also, bindings themselved don't seem to create issue as such. Also no other feature seen impacted so far except ClusterTemplateRevisions; since they dont have another dependent resource like the ClusterTemplateRevisions that will be stay orphaned.

For ClusterTemplates this affects deletion of the dependent ClusterTemplateRevisions causing them to never get cleaned up and then cause issues with Rancher UI listing templates

If customers have orphaned revisions, they should be manually deleted using kubectl

This PR fixes the OwnerReferences of Role and Rolebindings created for ClusterTemplate, ClusterTemplateRevision, GlobalDNS, MulticlusterApp and NodeTemplate resources.
The OwnerReferences were using wrong kind and apiVersion for these resources.

Validated on v2.4-head.
Steps:

Using a python test script, created 200 RKE templates with each template having 40 revisions in them. Deleted the templates
After all templates were deleted, no revisions were left around.

Validated that
Cluster templates feature is functional
Cluster templates can be shared across users and the rbac use cases are functional.

Validated the following on a upgraded setup from 2.4.5 to 2.4-head (HA) - commit id: ``
Global DNS Provider

- Users -   User1, User2, User3 - std user
- As an admin,   deploy a Global DNS Provider with ember access to User1 - Owner and User2 -   read Only
- Login as   User1. Verify User1 is able to list/edit/delete the Global DNS Provider
- Login as   User2. Verify User1 is able to list the Global DNS Provider. User should NOT be able to edit the GDP.
- Login as   User3. User3 should not be able to list/edit the Global DNS provider   created

Global DNS Entry

- Users -   User1, User3 - std user
- As an admin,   deploy a Global DNS Entry with ember access to User1 - Owner and User2 - read   Only
- Login as   User1. Verify User1 is able to list/edit/delete the Global DNS Entry
- Login as   User3. User3 should not be able to list/edit the Global DNS provider created

MCAPP

- Users - User1, User2, User3
- As an Admin   Deploy a MCAPP, on cluster C1, in projects p1 and p2
- While   dpeloying add users and give them the following member access in the app
- User1 -   Owner, User2 - Member and User3 - Read Only
- Login as   User1. Verify the User is able to list/edit/delete the MCAPP
- Login as   User2. Verify the User is able to list/edit/delete the MCAPP
- Login as   User3. Verify the User is able to list the MCAPP. But the user should NOT be   able to edit/delete mcapp

Node template

- As a Std User1, create a node template - temp1
- As Std  User2, user should not be able to list/edit/delete node template - temp1
- As admin,  user should be able to    list/edit/delete node template - temp1
- Admin should   be able to use temp1 to create a cluster

Cloud Credentials

- As a Std User1, create a cloud credential - cred1
- As Std   User2, user should NOT be able to list/edit/delete cloud credential - cred1
- As admin,   user should be able to    list/edit/delete cloud credential - cred1
- Admin should   be able to use cred1 to create a cluster using a node template


Set to Go 1.22.0

Setting to the latest z-stream is incompatible
with downstream build systems. Setting to the
lowest possible to be safe.

[x] I have taken backward compatibility into consideration.

/cc @xiangjingli


Ow fork 7.3 ntests

Decreased number of test runs for fitting in the github limits

Wrong base branch