[AAP-9555] Refactor parameters for aws_cloudtrail

Add unit test
Resolves AAP-9555: [COLLECTION] Unify the param list for aws source plugins
Follow-up #79

@abikouo  please review


minerva-ag: Modify fw version shell command

Summary:

Modify fw version shell command

Test Plan:

Build code: PASS

Log:
uart:~$ get_fw_version vr
comp_id |              sensor_name               |version |remain
2       |CPU_P0V85_PVDD                          |7da9ef23|0023
3       |CPU_P0V75_PVDD_CH_N                     |9c08d327|0024
4       |CPU_P0V75_PVDD_CH_S                     |9c08d327|0024
5       |CPU_P0V75_TRVDD_ZONEA                   |5267fe32|0024
6       |CPU_P0V75_TRVDD_ZONEB                   |6a8cebce|0024
7       |CPU_P1V1_VDDC_HBM0_2_4                  |87e1f0ea|0026
8       |CPU_P0V9_TRVDD_ZONEA                    |ca3cb12a|0013
9       |CPU_P0V9_TRVDD_ZONEB                    |94efeec0|0024
a       |CPU_P1V1_VDDC_HBM1_3_5                  |52c3262a|0020
b       |CPU_P0V8_VDDA_PCIE                      |170bf827|0024

@Victor-Jhong we would need to resolve these conflicts. Can you rebase to main?

The changes have been pushed by other PR, I'll close this PR.


Implement APCA Contrast

APCA (Accessible Perceptual Contrast Algorithm) is a new contrast algorithm. It currently targets replacing the WCAG 2.1 color contrast, or at least that is the intent. It has not done so yet, and there is no official WCAG 3 contrast.
The repo is here: https://github.com/Myndex/apca-w3.
The algorithm is pretty straightforward to implement and we have done so locally:
>>> from coloraide import Color
>>> text = Color('red')
>>> background = Color('blue')
>>> text.contrast(background, method='apca')
-20.313299765604636

We'd love to include this, but at this time, the license is so oppressive, that I do not think I could dare include it in an MIT project. The restrictions seem too constrained for me to do so.
I understand that they want to control the implementations to not leave a bad impression about the algorithm as they wish to gain good traction in regard to their claims. They also seem to be seeking some patents in regard to the algorithm. It's very confusing territory.  They seem to want to make it publicly available, at least through browsers, but want to have extremely strict control. They've licensed it to the WCAG, but restrictions dictate it can only be used for the web. Our library can be used in whatever as there are no such restrictions for our current license in that regard, not for personal or commercial.
Some minor quibbles are related to their extremely strict requirement that if you use it, you must be exact in your implementation and must be up to date. I've never seen such requirements in a license before. I don't so much have a problem with ensuring the algorithm is accurate per se, but it seems they are trying to protect the APCA, or at least the claim of "compliance".
I do hope that if/when it becomes officially WCAG 3, whatever that iteration looks like, it will have a reasonable license attached to it at that time. That would allow distribution in our package much easier.

I was looking into this more. If I understand correctly, what we've implemented is covered under the W3 license. It "should" be safe to include with attribution. The one thing that I think still troubles me is the following restrictions applied on top of the W3 license.
DISCLAIMER AND LIMITATIONS OF USE:
apca-w3 is an embodiment of certain supra-threshold contrast prediction technologies and it is licensed to the W3 on a limited basis for use in WCAG accessibility guidelines for web content only. apca-w3 may be used for predicting contrast for web content used for that specific purpose without royalty.

Said license excludes other use cases not related to web content. Prohibited uses include medical, clinical evaluation, human safety related, aerospace, transportation, military applications, and uses which are not specific to web-based content presented on self-illuminated displays or devices.

The ability for them to audit isn't a problem as everything is public. Keeping up to date isn't that hard, if I'm aware, I'll update. But the above is pretty troublesome.
It specifically calls out a number of industries that can't use it. Our project does not restrict usage. Even more troubling and uses which are not specific to web-based content presented on self-illuminated displays or devices. I absolutely have no intention of restricting people to only use the tool in the way I tell them to use it. Indicating proper intention of use is one thing, but this sort of restriction poisons the project.
I assume when this is officially part of the W3 spec that such restrictions will no longer be in place for anything actively used by the W3.
It really is a shame that there isn't a way to properly utilize this, yet.

Sadly, DeltaPhiStar, created by the same person, is completely out of the question as it is released under AGPL and we cannot include it here. https://github.com/Myndex/deltaphistar

Here is a playground with it working. https://facelessuser.github.io/coloraide/playground/?source=https%3A%2F%2Fgist.githubusercontent.com%2Ffacelessuser%2Fc1615dae555fdfe481b27ac99f1d9354%2Fraw%2F907257cd91bb697fea3473c120662ad4d8f108e8%2Fapca.py
Obviously, it is restricted by the license: https://gist.github.com/facelessuser/c1615dae555fdfe481b27ac99f1d9354#file-license-md. This is also why we will not merge this in at this time, but I don't want to lose the work.

@Myndex forgive my non-lawyer brain. I am not well-versed in AGPL. Is AGPL different than GPL in the sense that it doesn't require me to essentially restrict my project to the terms of AGPL?

...Is AGPL different than GPL in the sense that it doesn't require me to essentially restrict my project to the terms of AGPL?

Hi @facelessuser -- at the moment the slightly restrictive licenses are in place but we are providing exceptions in writing on a pretty regular basis. We're not trying to stifle development/ adoption but we are trying to limit how fast these emerging technologies are being adopted, partly because there's still a lot of unsettled issues regarding emerging standards and so forth.
The rapid interest, popularity, and adoption took everybody by surprise, and the W3C & AGWG don't see WCAG 3 becoming a recommendation for a couple years.
That said I recognize the interest in a functional contrast method to help readability, and reasonable guidelines — so we're developing APCA DeltaPhiStar and so forth, as separate independent projects, but we are being careful about how it's going to be adopted and implemented at this time.
If you like, send an email to<EMAIL_ADDRESS>

at the moment the slightly restrictive licenses are in place but we are providing exceptions in writing on a pretty regular basis. We're not trying to stifle development/ adoption but we are trying to limit how fast these emerging technologies are being adopted, partly because there are still a lot of unsettled issues regarding emerging standards and so forth.

@Myndex I completely understand, and this does make sense. I did read around at various places, so I did kind of come to the conclusion that APCA approach is still under research, etc. Don't take my grumblings as criticism, more me just realizing and working through that at this time I can experiment with it, but probably not something I can distribute publicly, or better stated, can only make available as a separate entity.
DeltaPhiStar looks interesting as well. My understanding was that if I include AGPL in my code, it will affect how people can use the MIT project. I could certainly be wrong, but if this is so, I cannot adopt the code unless I also adopt the AGPL license.
Regardless, your work in this area is very appreciated. I look forward to when one or more of these are more accessible.

Assuming that I'm wrong, and AGPL can live with MIT, if we are concerned with additional restrictions, we could implement it in https://github.com/facelessuser/coloraide-extras which is used to house experimental plugins. This would keep it out of the main distribution, and if/when it is safer to mainline it, we could migrate it back here.

Yep, totally understand. Thanks for taking the time to comment!

Implemented a Delta Phi Star gist that can be found here: https://facelessuser.github.io/coloraide/playground/?source=https%3A%2F%2Fgist.githubusercontent.com%2Ffacelessuser%2F49f11edb4e9d9301f0a8140bf5087d84%2Fraw%2Fc88a626edeb34f6df500e271a145ec6a8ed8cf76%2Fdelta_phi_star.py#
Gist contains license: https://gist.github.com/facelessuser/49f11edb4e9d9301f0a8140bf5087d84
This will at least give me a starting point once licensing is more agreeable or we create a package that can be released under the license (maybe under the extras package, maybe not). The gist at least gives us the ability to experiment with the contrast approach.


Which license is applied for this repository?

Hi,
I'd like to use this nice device and software provided here.
I wonder there is an appropriate license for this project.
If there is, please could you state the license in files? (Or I'll send a pull request if needed)
Thanks in advance!

Thanks to point it out. It's the Apache license. Just added a license file.

@xiongyihui Thanks for very quick response! Got it! :+1:


Responsiveness issue at /programs

Description
there is extra white space in mobile devices in dark mode
Expected Behavior
there should not be extra white space
Screenshots

Enviroment:

OS: Mac Linux Windows
Browser: Chrome Safari Firefox
Version:
Device: Desktop Mobile

Contributor Guides and Handbook
The meshery.io website uses Jekyll and GitHub Pages. Site content is found under the master branch.

📚 See the Contributing to Meshery.io Website section of the readme.md and other contributing instructions, too.
🎨 Wireframes and designs for Meshery UI in Figma
🙋🏾🙋🏼 Questions: Discussion Forum and Community Slack

please assign me this issue , i want to work on this issue

@Rajivkumawat3,  Thank you for looking into this issue, the responsiveness needs to be fixed. Since there are already two contributors working alongside to update the programs page (issue #1200), this will be taken care of by them too.

Noted, @thisiskaransgit ✅

ok sir

Can you assign this issue to me and explain on what needs to be done ? @samyakshah3008

@itsajay1029 I also wanted to work on it but as they said that two contributors is already working on this

Can anyone assign me this issue?

@itsajay1029 I also wanted to work on it but as they said that two contributors is already working on this

Ohh, ok. No problem

@samyakshah3008 would you like to give an update here?

I could work on this. You could assign it to me.


System.IO.Compression.Brotli: .NET Standard 1.4 -> 1.1

@Vedin It would be nice in the System.IO.Compression.Brotli library to downgrade .NET Standard version from 1.4 to 1.1. In general, class libraries which target a lower .NET Platform Standard version, can be loaded by the largest number of existing platforms. Key libraries are still targeted on .NET Standard 1.1  (e.g., System.IO.Compression).
Problems with the compilation will not be.

@Taritsyn do you want to contribute a PR?

Here is my microscopic PR.


Vickel: Watch ibb.co

Vickel requests the watch of the watch_keyword ibb\.co. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

ibb\.co has been seen in 58 true positives, 512 false positives, and 12 NAAs.

Rejected by Jeff Schaller in Charcoal HQ. Reason: 'at author's request '


Update interfaces.lib.inc

Quick fix ( maybe ) for GUA addresses in wrong order on widgets and Interface Info.

@marjohn56 isn't that the sort that's already there?

The issue here is we already established we can't trust the ordering of the data, so reordering it to fit our needs will just introduce bugs elsewhere that relies on a certain ordering already short of the spots that we just fixed. :)

I think I see what you mean, but if that's the issue, it's likely easier to capture the order when parsing and sort in one pass using link-local, capture order next. now it only guarantees link local is bottom. If you want a proposal, just let me know.

@AdSchellevis I wasn't arguing for it, I was arguing against changing this at all actually. the side effect is unpredictable.

@fichtner I know, we partly trust on ordering at the moment indeed, but only to detect if link-local is the only address if I'm not mistaken. Just saying that if we need to know the original position in ifconfig, we might as well add it as an attribute (and use it in sorting).  I don't think ifconfig itself knows the concept of primary, but I might be mistaken there.

That's indeed the issue, the primary is removed and readded, but ifconfig decides to make a VIP alias the primary. It might be that this is dhcp6c's fault for adding an alias, but it's unreliable to fix that (if indeed the case) because other services might do it differently or more or less correct depending on the situation.

I totally agree, primary in this context only means first existing (which doesn't help if you're looking for an entry that's gone ;) ).

That's indeed the issue, the primary is removed and readded, but ifconfig decides to make a VIP alias the primary. It might be that this is dhcp6c's fault for adding an alias, but it's unreliable to fix that (if indeed the case) because other services might do it differently or more or less correct depending on the situation.

dhcp6c does not add the alias, the alias is there either before dhcp6c adds the address or its left there when dhcp6c updates and removes the GUA before adding the new one. That's why un my original patch I removed the aliases before dhcp6c assigned the GUA, then put them back.

Alias here means just an additional address, not the primary one like you assign statically. If dhcp6c adds the primary IP as an alias (ifconfig alias notation) as well ifconfig is confused because there is no notion of a primary IP anywhere so the ordering is just random as we are seeing.

Maybe to illustrate: IPv4 DHCP doesn't have this odd behaviour it seems and the primary IP assigned is not flushed out by additional aliases.

If you manually add a v6 alias, which is the same as add, then remove the GUA, then put back the GUA, you'll find the GUA is at the bottom of the listed addresses when you do ifconfig, so it's just FreeBSD is agnostic about the order of addresses on the interfaces. Do the same thing with an IPv4 address and the same thing happens, add a new address, remove the old address, put back the old address  and it's bottom of the list.

Interestingly they have the same problem over the road, and have had for several years.
https://redmine.pfsense.org/issues/5999

d3b5d9ede21b64 might just do the trick for the interface stats


Update Nest and MSS freq, mcbist chiplet ID's, turn on Async mode

I would like @austinwrx to confirm that we want async mode as default before this is turned on.
As well, we need SBE updates turned on in witherspoon for this to work - I believe that is also in the pipeline.

Closing this as I think system owners should make the changes in serverwiz2 - if its decided hand editing is the way to go then we can reopen.

I can make the changes later. At this time, I don't want to enable ASYNC mode before we demonstrate that HB SBE update works correctly.


[help wanted] Can I use filteredPosOp from abstract type array in order to get specific implementation field

Hi there,
I'm so thankful about your work. This library is very useful.
Thank you so much.
I need some help.
I think that is my fault, but I can't do this:
MyRootClass::list.colProperty.filteredPosOp("some_alias") / SomeChildClassType::specificChildInstanceField
Is it possible ?
version: 3.9.2

Yes you can. Look at this unit test https://github.com/Litote/kmongo/blob/master/kmongo-property/src/test/kotlin/org/litote/kmongo/property/KPropertyPathTest.kt#L80
HTH


Implement Authorization Rules in Blog App using CanCanCan

Overview
This pull request introduces authorization rules to the Blog App using the CanCanCan gem, in line with the project's learning objectives. The focus is on limiting access to specific resources within the application based on user roles.
Implemented Features

Integrated the CanCanCan gem for managing authorization.
Added a role column to the users table through a migration.
Implemented authorization rules allowing only users with admin roles or owners of posts/comments to delete them.
Enhanced views to include "Delete" buttons, visible only to authorized users.

Changes Made

Installed and configured CanCanCan in the application.
Updated the users table schema to include a role column.
Defined ability rules in ability.rb to handle post and comment deletion based on user roles.
Modified post and comment views to conditionally display the "Delete" button.

Compliance with Requirements

Adhered to Ruby best practices as recommended by Microverse.
Ensured no linter errors.
Followed the specified Gitflow for the project.
Documented all changes professionally, as per Microverse guidelines.

Testing

Conducted manual testing to verify that the authorization rules work as intended.
Ensured that only authorized users can view and use the "Delete" functionality for posts and comments.

Request for Code Review
We request a code review for the implementation of authorization rules in the Blog App using CanCanCan. Your feedback and suggestions would be greatly appreciated to ensure the project aligns with best practices and meets all specified requirements. Thank you for your guidance and support in enhancing the app's functionality.

Hello Reviewer!

I'd like to explain how users are set up with roles in our application. By default, every new user is created with a 'member' role. If we need to configure any user as an admin, we can do so by executing a few commands in the terminal. Here's the process:

Open the terminal and enter the Rails console:
rails console

Find the user you want to update (e.g., User with ID 1):
user = User.find(1)

Update the user's role to 'admin':
user.update(role: 'admin')

Verify that the role has been updated:
user.role

Exit the console:
quit

This method allows us to manage user roles directly from the console, providing flexibility for administrative purposes.


Application wide search does not work?

Using thinking-sphinx 3.1.3
I have several models, here i am taking example of only two, Book and Author.
When i do Thinking ThinkingSphinx.search('heinlein') it returns only the matching books and movies but not matching author "Robert A. Heinlein" which is in my DB, even when i do ThinkingSphinx.search('Robert A. Heinlein') it does not return the author.
And there is also error when specify the class name
irb(main):012:0> ThinkingSphinx.search('Heinlein', :classes => [Author])
(0.4ms)  SELECT DISTINCT artist_type
FROM artists
Sphinx  Retrying query "SELECT * FROM artist_core WHERE MATCH('Heinlein @sphinx_internal_class_name (Author)') AND sphinx_deleted = 0 LIMIT 0, 20; SHOW META" after error: no enabled local indexes to search - SELECT * FROM artist_core WHERE MATCH('Heinlein @sphinx_internal_class_name (Author)') AND sphinx_deleted = 0 LIMIT 0, 20; SHOW META
Sphinx  Retrying query "SELECT * FROM artist_core WHERE MATCH('Heinlein @sphinx_internal_class_name (Author)') AND sphinx_deleted = 0 LIMIT 0, 20; SHOW META" after error: no enabled local indexes to search - SELECT * FROM artist_core WHERE MATCH('Heinlein @sphinx_internal_class_name (Author)') AND sphinx_deleted = 0 LIMIT 0, 20; SHOW META
Sphinx Query (6.4ms)  SELECT * FROM artist_core WHERE MATCH('Heinlein @sphinx_internal_class_name (Author)') AND sphinx_deleted = 0 LIMIT 0, 20
ThinkingSphinx::SphinxError: no enabled local indexes to search - SELECT * FROM artist_core WHERE MATCH('Heinlein @sphinx_internal_class_name (Author)') AND sphinx_deleted = 0 LIMIT 0, 20; SHOW META

Is Author a subclass of Artist? And do you have an index defined on either of those models? Both?

Yes Author is a subclass of Artist and i have defined index on Artist, as below.
ThinkingSphinx::Index.define :artist, :with => :active_record do
indexes title#, :sortable => true
indexes ntitle
indexes search_keywords
set_property :enable_star => true
set_property :min_infix_len => 1
set_property :morphology => 'stem_en'
attributes
has youtoo_popularity, created_at, updated_at
end
Apart from Author, Artist also other subclasses.

Okay. It looks like the Rails app is picking up the index without any issues, but Sphinx isn't aware of it. Have you run rake ts:rebuild since adding that index definition?

Yes, the record is very old, and i always run rake ts:rebuild

Does searching on Author directly work, as opposed to a global search? What about searching on Artist?
And do you have the output from a full indexing run on hand? I'm curious as to whether there are any errors processing the artist_core index (which would explain why Sphinx doesn't have it available for searches).

Not sure how i can i run it for Artist, but when i do ThinkingSphinx.search('Heinlein', :classes => [Artist]) it returns error as stated above.
FYI: I also a method Artist#search

Sorry, yes, I should have been more clear - I meant Artist.search('Heinlein') and Author.search('Heinlein'). If neither of those work, then it sounds like your index files either have been loaded by Sphinx and then corrupted, or Sphinx hasn't loaded them for some reason. Does running rake ts:index help at all? And if not, can you run rake ts:rebuild and let me know what the output is?

Thanks pat i have fixed it, the issue was the field "search_keywords" was not available in DB while artist_index.rb had "indexes search_keywords", so i removed this and things are working find.
Thank you so much for helping out :)

No worries, good to know it's working again :)


Tcx and gpx ecmascript 5 support

Making the tcx and gpx compatibel for ecmascript 5 will be great.

Hi there,
You meant the library and not the files right ?

No I mean the files tcx, gpx and xlmHelper(forgot to mention it). Im making
an running app in tizen for old wearables but the old wearables doesn't
support ecmascript 6. Making the script files compatible will be great.
Thank you
Op do 11 okt. 2018 02:10 schreef Paul du Pavillon<EMAIL_ADDRESS>
:

Hi there,
You meant the library and not the files right ?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/pdupavillon/nikeplus-client/issues/5#issuecomment-428773914,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AgKXnfyvnBrpPXiYwdqwRSMtEVjxZvK2ks5ujoxogaJpZM4XWiL8
.

I've created new files with es5 support but github didn't show the new files yet?
Ps: is it possible to add a heartrate extension to the gpx file?
Thanks

You need to create a pull request with your changes

Hi Paul,
I found a way how to get aunthenticity token for nikeplus.
Check the attachment you will find the endpoints for using it.
Cheers
Houssain
Op vr 26 okt. 2018 03:15 schreef Paul du Pavillon<EMAIL_ADDRESS>
You need to create a pull request with your changes
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/pdupavillon/nikeplus-client/issues/5#issuecomment-433254842,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AgKXnWwfei6WdW_Kzw-Kvgf8SnxcrY5xks5uomIwgaJpZM4XWiL8
.

Nike post URL: https://s3.nikecdn.com/login?appVersion=543&experienceVersion=441&uxid=com.nike.sport.running.droid.2.9&locale=es_ES&backendEnvironment=identity&browser=Google Inc.&os=undefined&mobile=false&native=true&visit=1&visitor=" + Guid.NewGuid().ToString();
REFERER URL:   https://s3.nikecdn.com/unite/mobile.html?adobe_mc=TS%3D1515493478|MCMID%3D19101753396136820572602757237369923454|MCORGID%3DF0935E09512D2C270A490D4D%2540AdobeOrg&androidSDKVersion=2.8.1&uxid=com.nike.sport.running.droid.2.9&locale=es_ES&backendEnvironment=identity&view=login&clientId=WLr1eIG5JSNNcBJM3npVa6L76MK8OBTt&facebookAppId=84697719333&wechatAppId=wxde7d0246cfaf32f7
Host = "s3.nikecdn.com";
JSON OBJECT: Dictionary<string, string> data = new Dictionary<string, string>();
data.Add("username", "email");
data.Add("password", "password");
data.Add("client_id", "WLr1eIG5JSNNcBJM3npVa6L76MK8OBTt");
data.Add("ux_id", "com.nike.sport.running.ios");
data.Add("grant_type", "password");

@CodinnQ,
I'm just curious about which tool do you use to test the api call?
Because I'm only getting an access denied with postman.

Hi I tested it via c# httpclient with cookies, I included all the headers
and I get the json response with auth. token.
Im only gettin timeout when using xmlhttpwebrquest with tizen
Op wo 20 feb. 2019 01:07 schreef Paul du Pavillon<EMAIL_ADDRESS>
:

@CodinnQ https://github.com/CodinnQ,
I'm just curious about which tool do you use to test the api call?
Because I'm only getting an access denied with postman.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pdupavillon/nikeplus-client/issues/5#issuecomment-465363780,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AgKXnaryIpx7NwyWl5NzYeXRJv44ZgKAks5vPJGsgaJpZM4XWiL8
.


Forward local port

How can I forward a local port to a remote port?
I understand that I can do a remote forward with channel_forward_listen. But I couldn't find an equivalent rust method for ssh_channel_open_forward  for direct port forwarding.

+1 on this, I am looking for a similar method. @JoyceBabu did you find any solutions to this?

Looking through libssh2 itself, it doesn't look like there is an exact match for libssh's ssh_channel_open_forward. I think channel_forward_listen (and the extended variant) is the closest that currently exists.
I'm going to close this issue for now as I don't think this a problem to be solved at the Rust level, but the underlying C library. Please ask on the libssh2 issue tracker or the mailing list. If it turns out that I was missing a function, please let me know and I'll reopen this issue and we can work on adding any missing methods.

FWIW, I think the idea is that you use https://docs.rs/ssh2/0.9.1/ssh2/struct.Session.html#method.channel_direct_tcpip to tell the ssh server to create a channel that connects to the target host.  Then your application uses read/write on the channel to send data to that remote host.

@wez, thanks! Thats exactly what I did
I wrote this bit of code to implement local port forwarding using this library. I want to essentially create an analog of the ssh -L command (ssh -NTf -L <local_port>:localhost:<remote_port> <remote_ip>). The problem that I am running into is that if the request has a JSON body (or any other payload), it doesn't forward that body to the remote IP. Do you or anyone else have any pointers on learning more about how to handle this?
I have attached a gist with my code below.
https://gist.github.com/mihirsamdarshi/9035edf5b7279de8e11969e2527ec549
Thanks!

@mihirsamdarshi I did not go ahead with the project.

To forward a local port, you need to first set up a local listener, accept loop, and each time you accept locally, you call Session::channel_direct_tcpip to create a new remote connection, and then your app needs to read/write the local socket and proxy the data into write/read calls on the remote channel.

Hey @wez, so I ended up writing the code you suggested (below), but I was running into a couple problems that maybe you could help me with. The below code reads and forwards the entire request from the host port to the remote port, but doesn't seem to read back the entire response from the remote port. I seem to consistently only get 17 bytes read from the response.
I suspect that this has something to do with the code being blocking, since I started off using the async-ssh2-rs-lite crate, which kind of works, but its not nearly as seamless as just using ssh -L.  There are delays when receiving complete responses that I can't seem to figure out how to start debugging.
I was looking through the wezterm source and noticed that you hadn't implemented port forwarding either, so maybe you're not the right person to ask, but I am new to Rust/TCP/libssh2, and any help pointing me where to start dealing with this problem would be really useful.
const BUFFER_SIZE: usize = 128

fn read_buf_bytes(
    full_req_len: &mut usize,
    full_req_buf: &mut Vec<u8>,
    reader_buf_len: usize,
    mut reader_buf: Vec<u8>,
) -> bool {
    // Added these lines for verification of reading requests correctly
    if reader_buf_len == 0 {
        // Added these lines for verification of reading requests correctly
        println!("No bytes read from response");
        false
    } else {
        *full_req_len += reader_buf_len;
        // we need not read more data in case we have read less data than buffer size
        if reader_buf_len < BUFFER_SIZE {
            // let us only append the data how much we have read rather than complete
            // existing buffer data as n is less than buffer size
            full_req_buf.append(&mut reader_buf[..reader_buf_len].to_vec()); // convert slice into vec
            false
        } else {
            // append complete buffer vec data into request_buffer vec as n == buffer_size
            full_req_buf.append(&mut reader_buf);
            true
        }
    }
}

/// Read the stream data and return stream data & its length.
fn read_stream<R: Read>(mut stream: R) -> (Vec<u8>, usize) {
    let mut request_buffer = vec![];
    // let us loop & try to read the whole request data
    let mut request_len = 0usize;
    loop {
        let mut buffer = vec![0; BUFFER_SIZE];
        // println!("Reading stream data");
        match stream.read(&mut buffer) {
            Ok(n) => {
                if !read_buf_bytes(&mut request_len, &mut request_buffer, n, buffer) {
                    break;
                }
            }
            Err(e) => {
                println!("Error in reading request data: {:?}", e);
                break;
            }
        }
    }

    (request_buffer, request_len)
}

/// Read the stream data and return stream data & its length.
fn read_channel<R: Read>(channel: &mut R) -> (Vec<u8>, usize) {
    let mut response_buffer = vec![];
    // let us loop & try to read the whole request data
    let mut response_len = 0usize;
    loop {
        let mut buffer = vec![0; BUFFER_SIZE];
        // println!("Reading stream data");
        let future_stream = channel.read(&mut buffer);
        thread::sleep(Duration::from_millis(10));

        match future_stream {
            Ok(n) => {
                if !read_buf_bytes(&mut response_len, &mut response_buffer, n, buffer) {
                    break;
                }
            }
            Err(e) => {
                println!("Error in reading response data: {:?}", e);
                break;
            }
        }
    }

    (response_buffer, response_len)
}

fn handle_req<R: Runtime>(session: Arc<Session>, mut stream: TcpStream, window: tauri::Window<R>) {
    if let Ok(channel) = session.channel_direct_tcpip("localhost", REMOTE_WEB_CLIENT_PORT, None) {
        let mut channel = Box::new(channel);
        // read the user-facing TCPStream
        let (request, req_bytes) = read_stream(&mut stream);

        println!(
            "REQUEST ({} BYTES): {}",
            req_bytes,
            String::from_utf8_lossy(&request[..])
        );
        // send the incoming request over the channel to the remote localhost and port
        match channel.write_all(&request[..req_bytes]) {
            Ok(_) => (),
            Err(e) => window
                .emit(
                    "backend_error",
                    format!("Failed to forward request, error: {}", e),
                )
                .unwrap(),
        };
        channel.flush().unwrap();

        // read the response from the channel to the remote server
        let (response, res_bytes) = read_channel(&mut channel);

        // then forward the response to the user-facing TCPStream
        match stream.write_all(&response[..res_bytes]) {
            Ok(_) => (),
            Err(e) => window
                .emit(
                    "backend_error",
                    format!("Failed to write response, error: {}", e),
                )
                .unwrap(),
        };
        stream.flush().unwrap();
        println!("SENT {} BYTES AS RESPONSE\n", res_bytes);
        channel.close().expect("Failed to close channel");
    } else {
        window
            .emit("backend_error", "Failed to open channel".to_string())
            .unwrap();
    };
}

/// Searches for open local ports in the given range. This function is necessary
/// because it may not always be possible to open a single static port on the
/// user's computer, for example if they are developing applications or running
/// Docker containers.
fn get_listener_in_port_range(port_range: Range<u16>) -> Result<TcpListener, String> {
    for port in port_range.clone() {
        if let Ok(listener) = TcpListener::bind(("<IP_ADDRESS>", port)) {
            return Ok(listener);
        } else {
            continue;
        };
    }
    Err(format!(
        "No ports in range {} - {} available",
        &port_range.start, &port_range.end
    ))
}

async fn listen_on_forwarded_port<R: Runtime>(
    ssh_session: Arc<Session>,
    sender: Sender<Result<SocketAddr, String>>,
    should_exit: Arc<AtomicBool>,
    window: tauri::Window<R>,
) -> std::io::Result<()> {
    match get_listener_in_port_range(1000..2000) {
        Ok(listener) => {
            // if the listener is successfully created, then we can start listening on the
            // port, then get the local address
            if let Ok(address) = listener.local_addr() {
                sender.send(Ok(address)).unwrap();
            } else {
                // send the local address to the main thread
                sender
                    .send(Err(
                        "Could not retrieve address from TCPListener".to_string()
                    ))
                    .unwrap();
            }

            // loop over incoming TCPStreams (requests)
            for stream in listener.incoming() {
                let cloned_session = Arc::clone(&ssh_session);
                let cloned_window = window.clone();
                // check that the shared AtomicBool does not say to exit the TCPStream
                if should_exit.load(Ordering::SeqCst) {
                    println!("Received close connection signal");
                    break;
                }

                match stream {
                    Ok(stream) => {
                        thread::spawn(move || handle_req(cloned_session, stream, cloned_window));
                    }
                    Err(e) => panic!("encountered error: {}", e),
                }
            }
        }
        Err(e) => sender.send(Err(e)).unwrap(),
    }

    println!("TCP Listener stopped");

    Ok(())
}

Is there any definitive answer as to how we're supposed to do this? Even with Channel::channel_direct_tcpip, I have issues understanding what I could do with that next.
My use-case is to access a database that's only locally-accessible through SSH. So first you log-in via SSH, then you log in the local database.
It's fairly common and most database clients allow a SSH tunnel before authenticating (for security reasons), for example Beekeeper Studio but I'm sure a lot of them have this feature.
After a quick search I see that pigeon-rs simply does it by spawning a ssh process. I guess this will have to do for now 😄

@mihirsamdarshi Where you ever able to get this figured out. I've been struggling with the exact same problem in Tauri.

@devinMcArthur not really fully, but take a look at my testing repo that I made with a bunch of libraries. I would appreciate any help/PRs, if you have any ideas.

@mihirsamdarshi Were you ever able to get this figured out. I've been struggling with the exact same problem in Tauri.

@devinMcArthur not really fully, but take a look at my testing repo that I made with a bunch of libraries. I would appreciate any help/PRs, if you have any ideas.

I've had to solve a similar problem in my app, it may not be exactly what you need but hopefully you can adapt the main io::copy threads to your exact application. I haven't done any benchmarks, but performance seems similar to the standard "sss -L" command.
use std::{sync::Arc, net::TcpListener};
use std::io::{self};
use log::{ error, warn, debug};
use ssh2::Session;

pub fn create_tunnel(host_server:String,host_port:u16, remote_server:String,remote_port:u16,session:Arc<Session>) -> Result<(),String>{
    match TcpListener::bind((host_server.as_str(), host_port)) {
        Err(err) => return Err(err.to_string()),
        Ok(listener) => {
            debug!("Succesfully bound {}:{} on host machine",host_server.as_str(),host_port);
            match session.channel_direct_tcpip(remote_server.as_str(), remote_port, None) {
                Err(err) => return Err(err.to_string()),
                Ok(channel) => {
                    debug!("Succesfully bound {}:{} on remote machine",host_server.as_str(),host_port);
                    session.set_timeout(10000);//increase if you're timing out a lot
                    session.set_blocking(false);//very important, otherwise reading from channel will block whil waiting for data that may not arrive
                    let mut stream_id=0;

                    tokio::task::spawn_blocking(move || {
                        'listener_loop: loop{
                            match listener.accept(){
                                Err(err) => {
                                    error!("failed to accept connection, reason {:?}",err);
                                    break 'listener_loop;
                                },
                                Ok((stream,socket)) => {
                                    debug!("New TCP stream from socket {:}",socket);

                                    let mut reader_stream=stream;
                                    let mut  writer_stream=reader_stream.try_clone().unwrap();

                                    let  mut writer_channel={channel.stream(stream_id)};//open two streams on the same channel, so we can read and write seperatly
                                    let  mut reader_channel={channel.stream(stream_id)};
                                    stream_id+=1;//new TCP streams bound to new channel streams

                                    //pipe stream output into channel
                                    tokio::task::spawn_blocking(move ||{
                                        loop{
                                            match io::copy(&mut reader_stream, &mut writer_channel){
                                                Ok(_) => (),
                                                Err(err) => {
                                                    error!("Failed to write to channel, reason: {:?}",err);
                                                    break
                                                },
                                            }
                                        }
                                    });

                                    //pipe channel output into stream
                                    tokio::task::spawn_blocking(move ||{
                                        loop{
                                            match io::copy(&mut reader_channel, &mut writer_stream){
                                                Ok(_) => (),
                                                Err(err) => {
                                                    error!("Failed to read from channel, reason: {:?}",err);
                                                    break
                                                },
                                            }
                                        }
                                    });
                                },
                            }
                        }
                        //not accepting any more connections
                        warn!("Tunnel on {:}:{:}  to {:}:{:} closed!",host_server.as_str(),host_port,remote_server.as_str(),remote_port)
                    });
                },
            }
        },
    };

    Ok(())
}


Add confirmation of keyboard-initiated transaction change cancelation

Explanation of Bugfix/Feature/Modification:
When using YNAB in keyboard-driven way (e.g., using <Tab> to switch between rows/buttons and running "Add another split" action) there is a good chance that one extra/few <Tab> hit can move active element selection past "Add another split"/before "Save" button and land on "Cancel".  Hitting an <Enter> key in such situation make whole transaction changeset to be discarded.
This feature makes a JS window.confirm (Ok/Cancel) prompt appear in situations when "Cancel" button in transaction row is being pressed by keyboard hit (i.e., <Enter>), thus allowing user to re-confirm "cancelation event" and possibly guard against an unsaved data loss.

makes sense and looks good to me. Thanks!


policycoreutils: build with gcc6

Motivation for this change
Current version 2.4 doesn't build w/gcc7.
This fix is 18.03-only. On master, an update of all SELinux tools to v2.7 is being prepared (#36978)
/cc ZHF #36453
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Fits CONTRIBUTING.md.

Success on x86_64-linux (full log)
Attempted: policycoreutils
Partial log (click to expand)
/nix/store/s86xgwqvpb9a1qn51hzadw4g18qhci2w-policycoreutils-2.4/bin/sepolgen-ifgen: interpreter directive changed from " /usr/bin/python -Es" to "/nix/store/7b2z0vfbs9539ga4pxx5gmli47rz5y3n-python-2.7.14/bin/python -Es"
/nix/store/s86xgwqvpb9a1qn51hzadw4g18qhci2w-policycoreutils-2.4/bin/audit2allow: interpreter directive changed from " /usr/bin/python -Es" to "/nix/store/7b2z0vfbs9539ga4pxx5gmli47rz5y3n-python-2.7.14/bin/python -Es"
/nix/store/s86xgwqvpb9a1qn51hzadw4g18qhci2w-policycoreutils-2.4/bin/chcat: interpreter directive changed from " /usr/bin/python -Es" to "/nix/store/7b2z0vfbs9539ga4pxx5gmli47rz5y3n-python-2.7.14/bin/python -Es"
/nix/store/s86xgwqvpb9a1qn51hzadw4g18qhci2w-policycoreutils-2.4/bin/sandbox: interpreter directive changed from " /usr/bin/python -Es" to "/nix/store/7b2z0vfbs9539ga4pxx5gmli47rz5y3n-python-2.7.14/bin/python -Es"
/nix/store/s86xgwqvpb9a1qn51hzadw4g18qhci2w-policycoreutils-2.4/bin/sepolicy: interpreter directive changed from " /usr/bin/python -Es" to "/nix/store/7b2z0vfbs9539ga4pxx5gmli47rz5y3n-python-2.7.14/bin/python -Es"
/nix/store/s86xgwqvpb9a1qn51hzadw4g18qhci2w-policycoreutils-2.4/lib/python2.7/site-packages/seobject.py: interpreter directive changed from " /usr/bin/python -Es" to "/nix/store/7b2z0vfbs9539ga4pxx5gmli47rz5y3n-python-2.7.14/bin/python -Es"
/nix/store/s86xgwqvpb9a1qn51hzadw4g18qhci2w-policycoreutils-2.4/sbin/semanage: interpreter directive changed from " /usr/bin/python -Es" to "/nix/store/7b2z0vfbs9539ga4pxx5gmli47rz5y3n-python-2.7.14/bin/python -Es"
/nix/store/s86xgwqvpb9a1qn51hzadw4g18qhci2w-policycoreutils-2.4/sbin/fixfiles: interpreter directive changed from "/bin/bash" to "/nix/store/zqh3l3lyw32q1ayb15bnvg9f24j5v2p0-bash-4.4-p12/bin/bash"
checking for references to /tmp/nix-build-policycoreutils-2.4.drv-0 in /nix/store/s86xgwqvpb9a1qn51hzadw4g18qhci2w-policycoreutils-2.4...
moving /nix/store/s86xgwqvpb9a1qn51hzadw4g18qhci2w-policycoreutils-2.4/sbin/* to /nix/store/s86xgwqvpb9a1qn51hzadw4g18qhci2w-policycoreutils-2.4/bin

Success on aarch64-linux (full log)
Attempted: policycoreutils
Partial log (click to expand)
/nix/store/jyhrsg3ldbqzcy0jmhqg0vsw4584zv73-policycoreutils-2.4/bin/system-config-selinux: interpreter directive changed from "/bin/sh" to "/nix/store/lnws52qanbi5vz8qg24v4ym57nznfndg-bash-4.4-p12/bin/sh"
/nix/store/jyhrsg3ldbqzcy0jmhqg0vsw4584zv73-policycoreutils-2.4/lib/python2.7/site-packages/seobject.py: interpreter directive changed from " /usr/bin/python -Es" to "/nix/store/5crgsfgy8n7v50hrihx4mhxdqa01nrmw-python-2.7.14/bin/python -Es"
/nix/store/jyhrsg3ldbqzcy0jmhqg0vsw4584zv73-policycoreutils-2.4/share/sandbox/start: interpreter directive changed from " /usr/bin/python -Es" to "/nix/store/5crgsfgy8n7v50hrihx4mhxdqa01nrmw-python-2.7.14/bin/python -Es"
/nix/store/jyhrsg3ldbqzcy0jmhqg0vsw4584zv73-policycoreutils-2.4/share/sandbox/sandboxX.sh: interpreter directive changed from "/bin/bash" to "/nix/store/lnws52qanbi5vz8qg24v4ym57nznfndg-bash-4.4-p12/bin/bash"
/nix/store/jyhrsg3ldbqzcy0jmhqg0vsw4584zv73-policycoreutils-2.4/share/system-config-selinux/polgengui.py: interpreter directive changed from "/usr/bin/python -Es" to "/nix/store/5crgsfgy8n7v50hrihx4mhxdqa01nrmw-python-2.7.14/bin/python -Es"
/nix/store/jyhrsg3ldbqzcy0jmhqg0vsw4584zv73-policycoreutils-2.4/share/system-config-selinux/system-config-selinux.py: interpreter directive changed from "/usr/bin/python -Es" to "/nix/store/5crgsfgy8n7v50hrihx4mhxdqa01nrmw-python-2.7.14/bin/python -Es"
/nix/store/jyhrsg3ldbqzcy0jmhqg0vsw4584zv73-policycoreutils-2.4/share/system-config-selinux/selinux_server.py: interpreter directive changed from "/usr/bin/python" to "/nix/store/5crgsfgy8n7v50hrihx4mhxdqa01nrmw-python-2.7.14/bin/python"
checking for references to /build in /nix/store/jyhrsg3ldbqzcy0jmhqg0vsw4584zv73-policycoreutils-2.4...
moving /nix/store/jyhrsg3ldbqzcy0jmhqg0vsw4584zv73-policycoreutils-2.4/sbin/* to /nix/store/jyhrsg3ldbqzcy0jmhqg0vsw4584zv73-policycoreutils-2.4/bin
/nix/store/jyhrsg3ldbqzcy0jmhqg0vsw4584zv73-policycoreutils-2.4

See 0c52ab986b.


Add appveyor CI config

https://github.com/metanorma/metanorma/issues/41

Ruby 2.3 dropped because https://github.com/metanorma/isobib/issues/39


Suppress future warning affecting Dataflow notebook

Please add a meaningful description for your change here
A future warning raised in Dataflow notebook affecting layout. However, we cannot easily change to data.map as long as we still support pandas dataframe < 2.1.0

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

JavaJarExpansionServiceTest failing but no clue how this could related to this PR


Nova syntax schema does not recognise subsyntax element

Nova’s syntax definitions allows for a subsyntax element instead of a subscopes one in Start-End scopes. The XML extension does flag these as invalid, however.

Running Nova 7.3 on macOS 11.5.2 with XML extension version 0.1.1

Hey, thanks for the report. I've released 0.2.0 which should fix this. Feel free to reopen this issue if something isn't correct!

Just updated and can confirm the subsyntax linting error is gone. Sweet, and thanks!


[Tooltip] Fix Arrow placement overlap

Closes #20582
It seems that solution is found but no one is submitting PR's so I went ahead and did it myself :)
Normalizes arrow margins to 1em as per @oliviertassinari recommendations

[x] I have followed (at least) the PR section of the contributing guide.

@esseswann Awesome, thank you!


SSLProtocolException after upgrading to Android 10 (and dovedot <IP_ADDRESS>?)

After upgrading an imap server to <IP_ADDRESS> from older dovecot version (actually debian upgrade), my mother's phone's k9 stopped working (some samsung phone). I didn't deep-dive into it but I remember that it was something related to encryption.
Not I upgraded my own phone (fairphone 3) from andoid 9 to 10. Same problem. Mails are not being synchronized any more. There is no prominent error message, but on the folder list I can read "SSLProtocolException: SSL h"
On server side, it says:
Sep 10 22:30:08 [servername] dovecot: imap-login: Disconnected (no auth attempts in 0 secs): user=<>, rip=[ip], lip=[ip], TLS handshaking: SSL_accept() failed: error:14094417:SSL routines:ssl3_read_bytes:sslv3 alert illegal parameter: SSL alert number 47, session=<[...]>
With verbose SSL logging:
Sep 10 22:44:01 [server] dovecot: imap-login: Debug: SSL: where=0x10, ret=1: before SSL initialization
Sep 10 22:44:01 [server] dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: before SSL initialization
Sep 10 22:44:01 [server] dovecot: imap-login: Debug: SSL: where=0x2002, ret=-1: before SSL initialization
Sep 10 22:44:01 [server] dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: before SSL initialization
Sep 10 22:44:01 [server] dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3/TLS read client hello
Sep 10 22:44:01 [server] dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3/TLS write server hello
Sep 10 22:44:01 [server] dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3/TLS write change cipher spec
Sep 10 22:44:01 [server] dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: TLSv1.3 write encrypted extensions
Sep 10 22:44:01 [server] dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3/TLS write certificate
Sep 10 22:44:01 [server] dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: TLSv1.3 write server certificate verify
Sep 10 22:44:01 [server] dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3/TLS write finished
Sep 10 22:44:01 [server] dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: TLSv1.3 early data
Sep 10 22:44:01 [server] dovecot: imap-login: Debug: SSL: where=0x2002, ret=-1: TLSv1.3 early data
Sep 10 22:44:01 [server] dovecot: imap-login: Debug: SSL: where=0x2002, ret=-1: TLSv1.3 early data
Sep 10 22:44:01 [server] dovecot: imap-login: Debug: SSL: where=0x2002, ret=-1: TLSv1.3 early data
Sep 10 22:44:01 [server] dovecot: imap-login: Debug: SSL: where=0x2002, ret=-1: TLSv1.3 early data
Sep 10 22:44:01 [server] dovecot: imap-login: Debug: SSL alert: where=0x4004, ret=559: fatal illegal parameter
Sep 10 22:44:01 [server] dovecot: imap-login: Debug: SSL: where=0x2002, ret=-1: error
Sep 10 22:44:01 [server] dovecot: imap-login: Debug: SSL error: SSL_accept() failed: error:14094417:SSL routines:ssl3_read_bytes:sslv3 alert illegal parameter: SSL alert number 47
Sep 10 22:44:01 [server] dovecot: imap-login: Disconnected (no auth attempts in 0 secs): user=<>, rip=[ip], lip=[ip], TLS handshaking: SSL_accept() failed: error:14094417:SSL routines:ssl3_read_bytes:sslv3 alert illegal parameter: SSL alert number 47
The issue imo only happens in k9 on android 10, not in k9 on earlier android and not on any non-android mail cliend (never tested another client on android 10 though). Seems like something changed in android 10.

K-9 Mail uses the platform's TLS stack. Your server might be using ciphers and/or TLS versions not supported by your Android device(s). See https://developer.android.com/reference/javax/net/ssl/SSLEngine
In the future please use the forum for support requests.


How to embed this way?

Sorry for my ignorance, but I did not understand how I incorporate so that "Click to open" does not appear.
I wish it were like this: http://prntscr.com/ket6mt

@Walter-Correa Hi,
README describes how to do it:

Publish unzipped folders
Publish zipped EPUB files

The point is appending ?book=xxx query to your BiB/i URI such as https://your.web.site/somewhere/bib/i/?book=BOOKNAME.


translation result does not make sense ???

i tried this model on its data , the result of translation was not accurate , why? "the prediction score were negative - what does that mean??-"
and what if i tried this model on a data of a complex language type like Arabic for example
which i had tried it the result were catastrophic ... keeps give me one word as a translation for all sentences
what is the explanation of this case and how can i handle this problem and make it work on my data ....???

The prediction score is the negative log likelihood of the sentence.
On how much data did you train the model and how did you prepare them?

I trained model its working super in training data but for test data it is giving same output for everything what to do and i also train till 100000 steps but no improvements


Two datapoints with errorbars

Two datapoints with errorbars does not plot with the mpl backend:
hv.ErrorBars(([0,1],[1,2],[0.1,0.2]))

Works
hv.ErrorBars(([0,1,1],[1,2,2],[0.1,0.2,0.2]))

Definitely a bug. Thanks for reporting this!

Fixed in PR referenced above. Closing.


Concat multiply videos.

In example we have a possibility to trim videos.
But what about concat two videos just one after another?
Is it possible right now?

LiTr doesn't support joining multiple videos yet. This is a complicated task that requires a lot of other (also not-so-simple) prerequisite functionality to be implemented - such as resampling/muxing audio, changing video frame rate, etc. I don't think it will be available anytime soon.


bump mermaid version

Hi, as of 10.6.0, mermaid has a cool xychart that seems really useful. I would try doing this in a PR, but I'm guessing upgrading a vendored library is something that should really only be done by you the maintainer, since the diff will probably be impossible to review.
Thanks for this awesome plugin :)

Hi @znd4 fixed in latest version #35


Add tests for streams in route handler responses

This patch adds two tests cases for sending a stream to micro.send from within a route handler. Tried to reproduce https://github.com/pedronauck/micro-router/issues/32.

Coverage remained the same at 100.0% when pulling c8d3876440425ab89257f9d278f7d5afdf3d0e58 on johanbrook:fix/streams into 59f66f274f9e11eed6dea042b8d89a5a0452ab16 on pedronauck:master.

thanks @johanbrook :v:


COMPAT: Add a numpy<2 pin to install requirements for 2.0.x

See https://github.com/shapely/shapely/issues/1972 for the details.
Current Shapely wheels for 2.0.x (and for an upcoming 2.0.3 release) are not compatible with future numpy 2.0 packages (if you install shapely now in an environment with nightly numpy, it fails to import), therefore it is recommended to add this upper pin now for releases.
We will keep the more flexible pin on the development branch, assuming by the time we release shapely 2.1, numpy 2.0 will be out and we can build shapely 2.0 to be compatible with all numpy versions.

It seems there is a bit of work needed for maint-2.0 to work with GEOS main, I'll investigate ....

Did some additional fixes in https://github.com/shapely/shapely/pull/1977, will update this PR now

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.3%) to 87.943%

Totals

Change from base Build<PHONE_NUMBER>:
0.3%

Covered Lines:
2582

Relevant Lines:
2936

💛  - Coveralls


chore(caching): Add ability to write env vars to .env file [EXT-2679]

Writes to .env file even if it does't exist
Adds .env to .gitignore if possible

@shikaan there's an unrelated test that files because it checks logs. Not sure why that's even happening but checking that one out.

@kdamball  What's the purpose of this PR if you do not store the env variables anymore? From what I can see, this PR would only add two dependencies and a js-file that's never used.

@andipaetzold there's a follow up PR for implementing both as we didn't have upload ready.


[joy-ui][docs] The overlay property of the Skeleton component doesn't seem to have any effect

Related page
https://mui.com/joy-ui/api/skeleton/
Kind of issue
Other
Issue description
Hi! I'm writing a React component library that styles Joy UI components the Tailwind CSS way.
I recently implemented a Skeleton component, and while writing the API documentation and reviewing it again, I found that the overlay property has no effect on the style of the Skeleton.
The API documentation says that when overlay is true, the position of the Skeleton becomes absolute, but it seems that the style is currently applied by setting the variant property to overlay.
Did they miss removing the overlay property during the Joy UI patching process?
Context
No response
Search keywords: skeleton, overlay

Thanks for reporting the issue! Joy UI development is temporarily on hold as the maintainers focus on the next two major releases of Material UI


Fix NFT external link field bug

What does this PR do and why?
Fixes #1789
Screenshots or screen recordings
See 'Website' field:

When hovering 'LINK TO NFT' (see link in bottom-left):

Acceptance checklist

[ ] I have evaluated the Approval Guidelines for this PR.
[ ] I have tested this PR in all supported browsers.
[ ] I have tested this PR in dark mode and light mode (if applicable).

Hey @johnnyd-eth nice PR! This is awesome for the creation, but won't fix existing links. I think we need to intervene in NftPreview and check if it is a fully resolvable URL, otherwise add on http/s://


Temporary run Create PR to Main workflow for any branch

:tada: This PR is included in version 1.2.3-test.1 :tada:
The release is available on:

v1.2.3-test.1
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.2.3 :tada:
The release is available on:

v1.2.3
GitHub release

Your semantic-release bot :package::rocket:


Seed Package Page, Create new scheduler type package is not available

create new schedule package
select group scheduler
Select Object, click + Add object
Drop down appear but scheduler not available in list

Required above image package name in drop down

in second phase


SxS dll loading fails in host process containers running on containerd 1.7

Describe the bug
My team runs a ruby script within a hpc (host process container). This works on containerd 1.6, but after upgrading to containerd 1.7 ruby fails to start with the below error message
Program 'ruby.exe' failed to run: The application has failed to start because its side-by-side configuration is
incorrect. Please see the application event log or use the command-line sxstrace.exe tool for more detailAt
C:\hpc\opt\hostlogswindows\scripts\powershell\main.ps1:63 char:5
    +     & $rubypath ./opt/hostlogswindows/scripts/ruby/tomlparser-hostlog ...
    +     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.
At C:\hpc\opt\hostlogswindows\scripts\powershell\main.ps1:63 char:5
    +     & $rubypath ./opt/hostlogswindows/scripts/ruby/tomlparser-hostlog ...
    +     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ResourceUnavailable: (:) [], ApplicationFailedException
    + FullyQualifiedErrorId : NativeCommandFailed

The sxstrace.exe tool provides this additional info, which indicates that the manifest isn't found:
Begin Activation Context Generation.
Input Parameter:
        Flags = 0
        ProcessorArchitecture = AMD64
        CultureFallBacks = en-US;en
        ManifestPath = C:\hpc\ruby31\bin\ruby.exe
        AssemblyDirectory = C:\hpc\ruby31\bin\
        Application Config File =
-----------------
INFO: Parsing Manifest File C:\hpc\ruby31\bin\ruby.exe.
        INFO: Manifest Definition Identity is (null).
        INFO: Reference: ruby_builtin_dlls,type="win32",version="<IP_ADDRESS>"
INFO: Resolving reference ruby_builtin_dlls,type="win32",version="<IP_ADDRESS>".
        INFO: Resolving reference for ProcessorArchitecture ruby_builtin_dlls,type="win32",version="<IP_ADDRESS>".
                INFO: Resolving reference for culture Neutral.
                        INFO: Applying Binding Policy.
                                INFO: No binding policy redirect found.
                        INFO: Begin assembly probing.
                                INFO: Did not find the assembly in WinSxS.
                                INFO: Attempt to probe manifest at C:\hpc\ruby31\bin\ruby_builtin_dlls.DLL.
                                INFO: Attempt to probe manifest at C:\hpc\ruby31\bin\ruby_builtin_dlls.MANIFEST.
                                INFO: Attempt to probe manifest at C:\hpc\ruby31\bin\ruby_builtin_dlls\ruby_builtin_
dlls.DLL.
                                INFO: Attempt to probe manifest at C:\hpc\ruby31\bin\ruby_builtin_dlls\ruby_builtin_
dlls.MANIFEST.
                                INFO: Did not find manifest for culture Neutral.
                        INFO: End assembly probing.
        ERROR: Cannot resolve reference ruby_builtin_dlls,type="win32",version="<IP_ADDRESS>".
ERROR: Activation Context generation failed.
End Activation Context Generation.

However, the manifest does exist at the expected location (C:\hpc\ruby31\bin\ruby_builtin_dlls\ruby_builtin_dlls.manifest) with the below content
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
    <assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
      <assemblyIdentity type="win32" name="ruby_builtin_dlls" version="<IP_ADDRESS>"></assemblyIdentity>

      <file name="libffi-8.dll"/><file name="libgmp-10.dll"/><file name="libwinpthread-1.dll"/><file name="libyaml-0-2.dll"/>
      <filename="zlib1.dll"/><file name="libcrypto-1_1-x64.dll"/><file name="libgcc_s_seh-1.dll"/><file name="libssl-1_1-x64.dll"/>
    </assembly>

We are able to run ruby by copying the binaries outside of C:\hpc, but can’t run it inside C:\hpc. (In other words, we can run it inside the container if the container copies the files onto the host filesystem, but we lose the benefits of filesystem isolation)
Since this works both in containerd 1.6 hpc and in containerd 1.7 outside of the c:\hpc directory, my guess is it's likely related to the new bind mount used for the C:\hpc directory.
To Reproduce
I've created a lightweight container which we've been able to use to reproduce this
Dockerfile
FROM mcr.microsoft.com/windows/servercore:ltsc2019

# Install chocolatey
ENV chocolateyVersion 1.4.0
RUN powershell -Command "Set-ExecutionPolicy Bypass -Scope Process -Force; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))"
# Install ruby
RUN choco install -y ruby --version <IP_ADDRESS> --params "'/InstallDir:C:\ruby31'"

COPY main.ps1 /main.ps1

ENTRYPOINT ["powershell", "C:\\hpc\\main.ps1"]

main.ps1
& ./ruby31/bin/ruby.exe --version

while($true){
    Start-Sleep 3600
}

And we've deployed it to our kubernetes cluster with this yaml, replacing <IMAGE> with the built container image from our container registry
apiVersion: apps/v1
kind: DaemonSet
metadata:
 name: ruby-test
 labels:
  app: ruby-test
spec:
 selector:
  matchLabels:
    app: ruby-test
 template:
  metadata:
    labels:
      app: ruby-test
  spec:
    securityContext:
      windowsOptions:
        hostProcess: true
        runAsUserName: "NT AUTHORITY\\SYSTEM"
    hostNetwork: true
    containers:
     - name: ruby-test
       image: <IMAGE>
       imagePullPolicy: Always
       workingDir: /hpc
    nodeSelector:
      kubernetes.io/os: windows

Expected behavior
Expected behavior is to be able to run executables with side-by-side dlls inside a host process container.
When running an exe within a container and a working directory inside the container filesystem (c:\hpc), we'd expect the SxS dll loader to be able to find manifests and load dlls within the container filesystem
Configuration:

Edition: Windows Server 2019
Base Image being used: Windows Server Core 2019
Container engine: containerd
Container Engine version 1.7

Additional context

We are able to run ruby by copying the binaries outside of C:\hpc, but can’t run it inside C:\hpc. (In other words, we can run it inside the container if the container copies the files onto the host filesystem, but we lose the benefits of filesystem isolation)
Since this works both in containerd 1.6 hpc and in containerd 1.7 outside of the c:\hpc directory, my guess is it's likely related to the new bind mount used for the C:\hpc directory.

Thanks for the really detailed and easy to repo steps.  The current work around is to copy this to the host and run it.
This does appear to be due to the bind mount feature. It seems like something is executing from the host context which wouldn't see that binding: https://github.com/microsoft/hcsshim/blob/e7b0eab484b277ab1a30a282b7232744a34e6624/internal/jobcontainers/jobcontainer.go#L334-L341

Closing because workaround seems low effort and acceptable.

Closing because workaround seems low effort and acceptable.

@fady-azmy-msft I think that this bug may cause user's downtime if the user upgrade their cluster to use containerd 1.7 but hit this issue.

ACK. Engineering is looking into this issue, but I don't have any timelines on this

Just FYI, in https://github.com/Azure/AKS/issues/3885, AbelHu commented on this as "may not be fixed (by design)". (However, this might just mean "at this point".)

Just FYI, in https://github.com/Azure/AKS/issues/3885, @AbelHu  commented on this as "may not be fixed (by design)". (However, this might just mean "at this point".)


chore: bump babel core

:tada: This PR is included in version 1.477.3 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Runix binaries get killed on MacOS ventura (code signing issue?)

Testing on an ARM MacOS Ventura 13.6:
$ nix-build
trace: [niv]: providing nixpkgs from tarball
trace: [niv]: providing fenix from tarball
trace: real cf: /nix/store/8vp0klf3xqx6wpfiay7lk9nri52dajb2-apple-framework-CoreFoundation-11.0.0
trace: [niv]: providing fetlock from tarball
trace: replace-dependency.nix: Derivation /nix/store/md319rcz201xzjps08s4ajyv362ifwr6-rust_runix-0.1.0 does not depend on /nix/store/8vp0klf3xqx6wpfiay7lk9nri52dajb2-apple-framework-CoreFoundation-11.0.0
/nix/store/zrn06flwaaqy79jkxvgxh04s0wqkinx0-rust_runix-0.1.0

$ ./result/bin/runix
Error: Not enough arguments (expecting at least one)

So the nix binary works, however a rewritten binary does not work. After a failed bootstrap I can run it manually:
$ /tmp/runix-bootstrap/store/zrn06flwaaqy79jkxvgxh04s0wqkinx0-rust_runix-0.1.0/bin/runix
fish: Job 1, '/tmp/runix-bootstrap/store/zrn0…' terminated by signal SIGKILL (Forced quit)

The console app at this time (filtering for SIGN) shows:
ignoring detached code signature on 'runix' with cdhash '87d787947219efa34d5b7ffd160e8701f0ad54be' because it is invalid, or not a simple adhoc signature.
CODE SIGNING: process 22806[runix]: rejecting invalid page at address 0x100d74000 from offset 0x0 in file "/private/tmp/runix-bootstrap/store/zrn06flwaaqy79jkxvgxh04s0wqkinx0-rust_runix-0.1.0/bin/runix" (cs_mtime:1.0 == mtime:1.0) (signed:1 validated:1 tainted:1 nx:0 wpmapped:0 dirty:0 depth:0)

I can sign the binary with an ad-hoc signature, and then it fails due to an unsigned shared lib:
$ codesign -vvvv -s - -f runix-bootstrap
runix-bootstrap: replacing existing signature
runix-bootstrap: signed Mach-O thin (arm64) [runix-bootstrap-dcd0090a72c81d7ab39d4ba0177f5ea2f9a0aeba]

$ ./runix-bootstrap
fish: Job 1, './runix-bootstrap' terminated by signal SIGKILL (Forced quit)

CODE SIGNING: process 22869[runix-bootstrap]: rejecting invalid page at address 0x10513c000 from offset 0x0 in file "/private/tmp/runix-bootstrap/store/gddpydckjch1hh376hc6l3i6jva4k0sp-libiconv-50/lib/libiconv.dylib" (cs_mtime:1.0 == mtime:1.0) (signed:1 validated:1 tainted:1 nx:0 wpmapped:0 dirty:0 depth:0)

So I think if I signed all shared libraries, it ought to work.
The fix
Runix does modify executables, which is going to corrupt signatures. I guess previous versions of MacOS didn't care. I think the fix will mean that runix has to run codesign on all appropriate files after rewriting them.
The good(?) news is codesign seems to silently succeed when you give it a non-binary file. So I think we could just throw every file at codesign if it's executable.

Looks like this is fixed in d792889 🎉


added endpoints and tests

I added all of the endpoints that were brought up here.

Awesome thanks!


Self-closing tags are replaced by opening and closing tags by Element append(String html)

I am uplifting Jsoup version from 1.11.3 to 1.15.3 in my project. I am seeing that there is a difference in the way self closing tags are handled by public Element append(String html) (https://github.com/jhy/jsoup/blob/master/src/main/java/org/jsoup/nodes/Element.java#L732)
With 1.11.3 version of jsoup I see that when we have
 tag and i call the append(String html) function it replace the
 with

But with 1.15.3 version of jsoup I see that when we have
 tag and i call the append(String html) function it replace the
 with

I see that when i call append(String html) it uses HTMLTreeBuilder, So I think append(String html) uses HTML parser to parse the html String provided as input.
I want the append(String html) to return
 tag as
  when I uplift the jsoup version to 1.15.3. How can this be achieved? is this a issue with the latest version?
Please can someone give me suggestions how I can handle this case in my code. I need to use the append(String html) function to append a sub section within a section.
For example
If my subsection is
 this is
 text
 when I call append(String html)  to append it to the
 tag it should return me

 this is
 text

 ( Note: this is the behavior in 1.11.3 version of jsoup, but not in 1.15.3)
Thank you in advance....

Can you provide actual test code that shows the behavior (code, what you get, vs what you want). I'm not clear on what parser (HTML or XML), what you're appending, what serializer you're using, etc.

Hi @jhy
I was trying to write a simple test to re-create this scenario. I was not able to exactly re-create this case but found a different behavior when I call append(String html)
Input string =

 this is a sample text

HTML string output  from jsoup xml parser:

 this is a sample text

HTML string after append is called:

 this is a sample text

Test code:
    public void TestAppend(){
        String inputHTML ="

 this is a sample text
";

        Document xmlDoc = Jsoup.parse(inputHTML, Parser.xmlParser());
        Document container = Document.createShell("");
        int nodesCount = xmlDoc.childNodeSize();
        for (int i = 0; i < nodesCount; i++) {
            container.body().appendChild(xmlDoc.childNode(0));
        }
        xmlDoc = container;
        xmlDoc.outputSettings().syntax(Document.OutputSettings.Syntax.xml);
        xmlDoc.outputSettings().escapeMode(Entities.EscapeMode.xhtml);
        System.out.println("HTML string output  from jsoup xml parser:\n"+xmlDoc.toString());

        Element sectionElement = new Element(Tag.valueOf("span"), "");
        sectionElement.empty();
        sectionElement.append(xmlDoc.outerHtml());
        System.out.println("HTML string after append is called: \n"+sectionElement.toString());
    }

If we see the output from the xmlParser we can see
 is retained as it is.
But as soon as we call sectionElement.append(xmlDoc.outerHtml());  ,
 gets converted to

In our code we call append(String html) multiple times to create subsections. So eventually
 gets converted to

How can I ensure that when I call sectionElement.append(xmlDoc.outerHtml());  does not convert
 to
 ?
Is there a reason why append(String html) converts
 to
 when it encounters
 ?

I see that when i call append(String html) it uses HTMLTreeBuilder, So I think append(String html) uses HTML parser to parse the html String provided as input.

No, it uses the same parser as the parser that produced the Element you are appending do. See:
https://github.com/jhy/jsoup/blob/da23af85f39df9f0df732029ed26c34764811009/src/main/java/org/jsoup/nodes/Element.java#L732-L737
In your example code, you are creating a new Element outside of a configured parser, and so that defaults to using the HTML parser:

Element sectionElement = new Element(Tag.valueOf("div"), "");

And then you are re-parsing the XML doc as HTML (in the context of the HTML Element sectionElement). Hence, the self-closing br tag is emitted as
.
Also, you might like to use the wrap(html) method to wrap a div or other content around existing content. It works more efficiently and simply than serialising and re-parsing as you're doing now.
Here's an example:
String xml = "

Text
";
Document doc = Jsoup.parse(xml, Parser.xmlParser());
Element p = doc.selectFirst("p");
Element div = p.wrap("div");
p.append("
");
System.out.println("XML:\n" + doc.outerHtml());

Produces:
XML:

Text


Support OpenTracing in Baker

Include opentracing in Baker, to trace incoming requests and Client Interactions.

Input for the design session on this:
a. A span maps to an execution of an interaction?
b. When an interaction is retried, a new span is created?
c. How does the processId fit in the specification? Use the parentId as processId?

This can be build using #119 as an extension to baker

Closing, this can be build as an extension using the event listening API.
Or, if that is not enough, we need to create a way to pass a context.
That is another issue.


FileNotFoundError: [Errno 2] No such file or directory: 'imagenet_pre_trained_model'

Hi @ZHANGDONG-NJUST
during the segmentation process, the error reported here:
Mode: train
Device:
    0: GeForce RTX 2080 Ti
    1: GeForce RTX 2080 Ti
Dataset: VOCAug
    # data: 1464
    Split: train
    Root: /disk/PASCAL_VOC_2012/VOCdevkit/VOC2012
Model: DeepLabV2_ResNet101_MSC
Traceback (most recent call last):
  File "main.py", line 477, in <module>
    main()
  File "/disk/tia/anaconda3/envs/tia36/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/disk/tia/anaconda3/envs/tia36/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/disk/tia/anaconda3/envs/tia36/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/disk/tia/anaconda3/envs/tia36/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/disk/tia/anaconda3/envs/tia36/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "main.py", line 129, in train
    state_dict = torch.load(CONFIG.MODEL.INIT_MODEL)
  File "/disk/tia/anaconda3/envs/tia36/lib/python3.6/site-packages/torch/serialization.py", line 584, in load
    with _open_file_like(f, 'rb') as opened_file:
  File "/disk/tia/anaconda3/envs/tia36/lib/python3.6/site-packages/torch/serialization.py", line 234, in _open_file_like
    return _open_file(name_or_buffer, mode)
  File "/disk/tia/anaconda3/envs/tia36/lib/python3.6/site-packages/torch/serialization.py", line 215, in __init__
    super(_open_file, self).__init__(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: 'imagenet_pre_trained_model'

Could you please help me to solve this problem?
By the way, does the ins_seg in /CONTA-master/pseudo_mask/result/ is the generated pseudo mask?

Thanks for your report. You can download an ImageNet pre-trained ResNet model and put it here, this problem will be solved.

@ZHANGDONG-NJUST Sorry to bother you, may i ask is the initial weight of the segmentation network only pre-trained on ImageNet or also fine tune on COCO, Thanks a lot.

@hiddlephila   only pre-trained on ImageNet not COCO


join more than two tables

I frequently need to join more than two tables, I can join two at a time with petl currently but it would be very convenient to have a single function call to join any number of tables.

Have been able to live without this, will re-open if someone else requests it.


Update scalafmt-core to 3.2.2 in series/0.23

Updates org.scalameta:scalafmt-core from 3.2.1 to 3.2.2.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Files still referring to the old version number
The following files still refer to the old version number (3.2.1).
You might want to review and update them manually.
core/shared/src/main/scala/org/http4s/Uri.scala
tests/shared/src/test/scala/org/http4s/parser/ForwardedHeaderSpec.scala
website/src/hugo/content/changelog.md

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scalameta", artifactId = "scalafmt-core" } ]

labels: library-update, early-semver-patch, semver-spec-patch, old-version-remains, commit-count:n:2

Closing this, better to merge the changes forward from 0.22


POST to Wellcome storage service with pointer to S3 file

The first problem encountered when trying to do this is that by default Archivematica generates AIPs in 7z/bzip2 format, which is not supported by the Wellcome Storage Service.
Two possible approaches:

Adapt the Storage Service to accept .7z files (or another format natively supported by Archivematica)
Adapt Archivematica to support zip files which the Storage Service back end (using java.util.zip.ZipFile) is able to accept.

Archivematica has the following options for AIP compression:

7zip with bzip2
7zip with lzma
7zip without compression
Uncompressed
Parallel bzip2

To add support for other compression methods would require a data migration to add the option to the workflow, and code in the compress_aip method to invoke the compression command. gzip is already available in the MCP Client container where the compression would run, but zip is not.

I think we'd prefer to make Archivematica compatible is possible. What does "Uncompressed" do?

It means no command is run during that step - so it remains as a directory.

Tasks:

[ ] Support compressing AIPs with zip/deflate
[ ] Add unzip command to container so atool can decompress during verify AIP step
[ ] Add missing fields to bag-info.txt: external-identifier, source-organisation, bagging-date

We need to determine which of the missing fields we need to add to the bag info file. The bag spec doesn't require any of them, but the storage service does.
As far as I can tell, the bagit-java tool currently used by the archivematica pipeline to generate AIPs, doesn't appear to support specifying these fields, although it does support supplying an externally generated bag-info file.
bagit-python does, and could be used as a replacement although may require a little manipulation of directories to get it to generate in the same directory structure as the original tool.
If we need to add any extra fields to the bag info file (regardless of which ones they are) this will need to be addressed.

AM issue for external identifiers: https://github.com/archivematica/Issues/issues/492

AM issue for zip/definate: https://github.com/archivematica/Issues/issues/493

AM PR for zip/deflate: https://github.com/artefactual/archivematica/pull/1346

The completion of this task is blocked on a couple of issues in the storage service:

making Source-Organization optional (https://github.com/wellcometrust/platform/pull/3427)
supporting the bag existing inside a top level directory within the zip file. (https://github.com/wellcometrust/platform/pull/3434)

I modified an AM-generated zip to prevent the storage service erroring on these two issues and posted it to see if any more issues came up, and it went through. Once those are fixed, it should be possible to upload a bag from the AM workflow into the storage service and then continue with https://github.com/wellcometrust/platform/issues/3386

AM PR for external identifiers: https://github.com/artefactual/archivematica/pull/1347


fix(tabs): prevent tabs from scrolling

Description (required)
Prevent tabs from scrolling during navigation

Related issues & labels (optional)

Closes #5678
Suggested label: bug

Hi @yasnbouz , thank you for making a contribution to Astro Docs!  🙌
I can see that you tackled an open issue, but I should warn you that there was no discussion around that issue, so we haven't had a chance to decide whether this is in fact an issue that we want or intend to fix.  😄  That just means this contribution will have to wait as we figure out what we in fact want to do about that opened issues, so you'll have to hang on a little longer!
It's always safe to stick to the "help wanted" labels when working on issues. Sometimes people open what they feel are issues, but for various reasons are not something we decide or intend to change. We have also just recently migrated our docs to Starlight, so we have a lot of review planned as we work out some of the new internals, and it's possible that this contribution may not end up fitting into our larger planned work. That doesn't mean it was a bad solution to the problem, just that this is not something we have yet requested assistance with, and it may not end up being compatible with other site maintenance we are implementing.
So, please hold on for a while longer as this is not something we had planned to look at, and are trying to incorporate reviewing this along with our other maintenance!


shotcontroller tests

The shotcontroller is very useful but there are some things that break easily and are very crucial. This is mainly the autogeneva. When a robot is not able to use the geneva, or when we don't want it specifically then we need to make sure the positioning and aiming is still correct.

fixed in #471


Improve Class Structure of UI

Changes

Change class structure of UI to Improve cohesion and reduce coupling

Related Issues
Closes #70

Codecov Report

Merging #71 (bbe56b1) into develop (b92b502) will increase coverage by 4.12%.
The diff coverage is 0.00%.

@@              Coverage Diff              @@
##             develop      #71      +/-   ##
=============================================
+ Coverage      59.36%   63.48%   +4.12%
- Complexity       340      364      +24
=============================================
  Files             76       75       -1
  Lines           1351     1331      -20
  Branches         134      126       -8
=============================================
+ Hits             802      845      +43
+ Misses           487      442      -45
+ Partials          62       44      -18

Impacted Files
Coverage Δ

src/main/java/seedu/ibook/ui/CommandBox.java
0.00% <0.00%> (ø)

src/main/java/seedu/ibook/ui/MainWindow.java
0.00% <0.00%> (ø)

src/main/java/seedu/ibook/ui/MenuToolbar.java
0.00% <0.00%> (ø)

src/main/java/seedu/ibook/ui/ResultWindow.java
0.00% <0.00%> (ø)

src/main/java/seedu/ibook/ui/UiComponent.java
0.00% <0.00%> (ø)

src/main/java/seedu/ibook/ui/popup/Popup.java
0.00% <0.00%> (ø)

src/main/java/seedu/ibook/ui/popup/PopupAdd.java
0.00% <0.00%> (ø)

...rc/main/java/seedu/ibook/ui/popup/PopupDelete.java
0.00% <0.00%> (ø)

...rc/main/java/seedu/ibook/ui/popup/PopupUpdate.java
0.00% <0.00%> (ø)

...rc/main/java/seedu/ibook/ui/table/ProductCard.java
0.00% <0.00%> (ø)

... and 20 more

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more


Update Gene Expression cell count in tool tip

Currently the cell count “cell count” column values don’t always match the tooltip table cell count values. The Cell Count column queries the cube directly, where we suspect the tool tip back calculates the number. This ticket encapsulates the work to update the tool tip so the values match

Per Alec, we now have cell count directly in API response, so use that for tooltip too!
https://czi-sci.slack.com/archives/C0247APK621/p1663337171342269


Error: No Manifest when trying to install plug-in on Big Sur

Hello,
On my MBP Pro 16 (2019), running OSX Big Sur, I've installed Aerial using the DMG then double clicking the application.
On the icon in the menu bar I can see a red dot.
When I drop down the menu, it says : "Screen-Saver : Plug-in not installed! [INSTALL NOW]"
When I click on "INSTALL NOW", I get an error "No Manifest, please report"...
How can I fix that ?
Thanks a lot.

closing this, please feel free to reopen if you haven't fixed your issue


Why not use BEIR?

Hey guys, awesome work. Simple question, why did you not just use BEIR (& possibly extend it with training datasets)?

Hi @Muennighoff,
Thanks for your question. This is very an interesting one!
This was because, the focus was different at that time: This paper and its evaluation code are dedicated to sentence-level similarity tasks, which are totally different from the setting of queries and documents in the IR tasks from BeIR.
Actually, one can refer to my latest work "GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval". We also tried TSDAE on BeIR and we find it works well for domain-adaptation usage (cf. Table 1), but it works poorly in the purely unsupervised setting (cf. Table 9 in the Appendix).

I see, thanks for your fast reply

i guess it depends on the dataset - Quora & ArguAna from BEIR for example contain duplicates across queries and documents and hence have similar avg lens for both, so maybe closer to sentence-level similarity
you're BioASQ score of 70.7 for BM25 in GPL Table1 seems a bit off compared to the scores in the BEIR paper

Also if you don't mind me asking here:
Is the MLM column in Table 9, using pre-trained DistilBERT MEAN pooling to encode without any further training? Or is it MLM done on MS-Marco?
Just for feedback, it's a bit confusing having "MS MARCO" as a model in the table, which is a dataset in most other papers in this niche.

Yes, I would say Quora is a sentence-level task (although we did not use it in the TSDAE paper, since the language is quite general). And we made the difference more clear between these two papers, TSDAE and GPL.
Two main differences that influence the result: (1) We downsampled the corpus to 1M from 15M for efficient training&evaluation; (2) we concatenated the titles and bodies to form a document (in the GPL paper) instead of using multiple fields (in the BeIR paper), since I think this way is more consistent among both sparse and dense retrieval (actually this does not change the score too much, e.g. <1.0 difference).

Also if you don't mind me asking here:
Is the MLM column in Table 9, using pre-trained DistilBERT MEAN pooling to encode without any further training? Or is it MLM done on MS-Marco?
Just for feedback, it's a bit confusing having "MS MARCO" as a model in the table, which is a dataset in most other papers in this niche.

The setting is: Do MLM training on the target corpus (e.g. FiQA) and then directly do the evaluation (yeah, mean-pooling) on the target dataset without any other training.
Thanks for your comments! Yes, we were also quite unsure about how to name the model. We thought about using "MarginMSE", but that is also ambiguous since the new method GPL also involves MarginMSE. We will consider other options if possible.

Also if you don't mind me asking here:
Is the MLM column in Table 9, using pre-trained DistilBERT MEAN pooling to encode without any further training? Or is it MLM done on MS-Marco?
Just for feedback, it's a bit confusing having "MS MARCO" as a model in the table, which is a dataset in most other papers in this niche.

The setting is: Do MLM training on the target corpus (e.g. FiQA) and then directly do the evaluation (yeah, with mean-pooling) on the target dataset without any other training. For other unsupervised/pre-training methods, the settings are the same.
Thanks for your comments! Yes, we were also quite unsure about how to name the model. We thought about using "MarginMSE", but that is also ambiguous since the new method GPL also involves MarginMSE. We will consider other options if possible.

Makes sense, thanks a lot!


Repo sync for protected CLA branch

The pull request is created from master637399391517537198 to master to fix git push error for protected CLA branch

@huypub : Thanks for your contribution! The author(s) have been notified to review your proposed change.