[ffigen] Fix global test flake

global_test and global_native_test were flaky. I'm still not sure why, but I had a hunch that compiling all the native_objc_test dylibs into a single dylib might fix it, and it did. Each test used to have its own dylib, but when they were loaded they all ended up in the same namespace anyway, so it's better to make one big objc_test.dylib. That's the main change in this PR.
Other changes:

To fix some duplicate symbol errors in the native wrapListenerBlock functions, I'm now prepending the ffigen module name to them.
To fix other duplicate symbol errors in the functions defined in native_objc_test/util.h, I've removed that file from all the places it was being #included, made it a .m file, and I'm compiling it into objc_test.dylib as its own file.
For some reason that meant I couldn't use bool anymore, so I switched to the old C style int booleans.
failed_to_load_test.dart used to compile failed_to_load_test.m into a dylib just like all the other tests, but then just didn't load the dylib. Since failed_to_load_test.m is compiled into objc_test.dylib, I changed it so that ClassThatWillFailToLoad just isn't given a definition. This triggers the same behavior that test needs.
inherited_instancetype_test and is_instance_config both define a BaseClass and a ChildClass, so I had to rename one of them.

Fixes https://github.com/dart-lang/native/issues/1435

coverage: 90.347% (+0.04%) from 90.307%
when pulling 048e6ffb56f0eea5f6f0a8264baf1af6cc0ba214 on globalflake
into cc34333924fcc0099ec5becf3778a121bd312254 on main.


nullability

Everything is nullable
Issue #699 highlights a great example where fields can return a value of null based on filters the query may impose on that field.
This holds true even when isRequired is set on that field.
... We need to document this.

@jesstelford this one is a little un-clear to me. Is it possible to add some more information so that I or someone outside the core team could action this issue?

I think @jesstelford means.. if we add where arguments to all field types then, depending on the query, any field can have a null value.
I'd agree that, usually, everything is nullable but for different reasons -- enabling the isRequired option doesn't say anything about existing data. It just means you can't specify a null value for creates and updates.

Great clarification @molomby, thanks 👍
@MadeByMike we need docs somewhere that point out all queries for all fields have the potential to return null, and users should handle that case correctly. Additionally, we should document that adding isRequired only has an impact on the validation done during a create or update mutation, but makes no guarantees about the value returned from the API.


Roll engine b59c86493159..3a9c22afde67 (1 commits)

https://github.com/flutter/engine/compare/b59c86493159...3a9c22afde67
git log b59c86493159..3a9c22afde67 --no-merges --oneline
3a9c22afd Allow GetRectsForRange to provide more detailed/nuanced metrics through RectStyle enum. (flutter/engine#6335)
The AutoRoll server is located here: https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md
If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

@rmistry @liyuqian : New roll has been created by https://autoroll.skia.org/r/flutter-engine-flutter-autoroll

Trybots failed. These were the failed builds: https://flutter-dashboard.appspot.com/build.html

Commit queue failed; closing this roll.


Invoking an admin service

Describe the issue:
Invoking admin services with session cookies gives HTTP 401 error
How to reproduce:
log in to the Web application using WSO2 Identity Server and copy yours cookie

insert cookies in the call to the web service
 SSLContext sslContext = SSLContext.getInstance("TLS");
    sslContext.init(null, new TrustManager[] {new TrustAllTrustManager()}, null);
    String serviceEndPoint = wso2url +"/services/RemoteUserStoreManagerService";
    adminStub = new RemoteUserStoreManagerServiceStub(serviceEndPoint);
    ServiceClient client = adminStub._getServiceClient();
    Options option = client.getOptions();
    option.setProperty(HTTPConstants.COOKIE_STRING, "route=1600762628.789.330.799271; Path=/; Secure=null; HttpOnly=null;JSESSIONID=59FBBDE6F3FF5847C1591CDD2B2F2446; Path=/; Secure=null; HttpOnly=null");
    option.setProperty(HTTPConstants.CUSTOM_PROTOCOL_HANDLER,new Protocol("https", (ProtocolSocketFactory) new SSLProtocolSocketFactory(sslContext), 443));
    option.setManageSession(true);

Expected behavior:
The user list
Login Successful route=1600783574.582.332.933103; Path=/; Secure=null; HttpOnly=null;JSESSIONID=B3B3B7B213FD1A89D72DFE9FA5598A7C; Path=/; Secure=null; HttpOnly=null Listing user names of Carbon server......  User Name : admin
Environment information:

Product Version:  IS 5.9.0

Hi @maghibus ,
These cookies are only for front-end session. So, you won't be able to use those to authenticate to the (backend) admin services directly.
To get a cookie which you can use to invoke an admin service, you can use the 'AuthenticationAdmin' SOAP service first with the correct credentials and get the cookies in that response (given that you are correctly authenticated) to invoke the other admin services.

Closing as answered. Please feel free to re-open if you have any further details on this.


Remove npm audit warnings and tslint deprecated rule warning

What:

This update fixes:
-npm audit warnings
-tslint deprecated rule warning
-adds a new contributor
-adds Google firestore package (to use auth package)

Issue number: N/A

Codecov Report

Merging #485 into master will decrease coverage by 0.18%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #485      +/-   ##
==========================================
- Coverage    72.1%   71.92%   -0.19%
==========================================
  Files          65       65
  Lines         552      552
  Branches       34       34
==========================================
- Hits          398      397       -1
- Misses        144      145       +1
  Partials       10       10

Impacted Files
Coverage Δ

...tarter/src/app/core/animations/route.animations.ts
80% <0%> (-10%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e2c8e90...6ac4c69. Read the comment docs.

Wondering if the codecov/project test failed because of different dependencies, as there were no code changes there?
Any ideas?


Bug/Discussion: Cursor-Position in collaborative coding

When using this I've stumbled upon an issue. I'm not sure if my suggested fix is actually always a good idea, or how easy it would be to solve, so this should probably be discussed.
If multiple persons work in different areas of a file, this can lead to conflicts, when the person that is closer to the beginning of the file inserts/deletes newlines, or with both in a single line just when it enters anything.
Because this will not change the position of the cursor of the other person in the file, meaning its cursor will move in relation to the context the person was working on, since that context is indirectly moved by the other person.
This can be fairly disrupting, if one was currently writing something, it might be spread over multiple lines or be... interlaced with the past characters.
The fix would be to move the cursor accordingly, depending on its current position. If its on the same line and past the other cursor, it has to be moved forward, if its simply below, it has to be moved down, if its above or generally before the other cursor, it doesn't get influenced.

Perhaps a better way to reason about it is to place each cursor in relation to the cursor before, and just the first cursor in relation to the file-start.
That way the state would not be manipulated, and it would cut down on required logic and potential errors. Would be a potentially bigger change from the status quo though. Maybe it interacts better or worse with network delays etc, I don't know the way it works currently, also I'm tired.

Because this will not change the position of the cursor of the other person in the file, meaning its cursor will move in relation to the context the person was working on, since that context is indirectly moved by the other person.

Thanks for the bug report. Unfortunately, I don't understand what you mean by this. Insertions of newlines are considered in the collaborative editing algorithm used by Rustpad, and they should not affect other users' changes. Could you clarify the issue, and either provide a minimal reproducible example or show a video of what you mean?

Sorry for the slow response. I've been incredibly busy.
This is embarrassing. I am unable to recreate this issue. It behaves exactly like I suggested as an improvement.
I could've sworn that it behaved erratic like that in one of my sessions, but me being me I did not validate it before going through this report process.
Sorry to take up your time and thanks for this well designed App!


Inactive highlight ranges are incorrect in the presence of #elif

In the following code:
#define BAR

#if defined(FOO)
int x1;
int x2;
#elif defined(BAR)
int y1;
int y2;
#else
int z1;
int z2;
#endif

the highlighting meta.disabled should not apply to the int y1; or int y2; lines.
It currently does apply to the int y1; line.

The preprocessor is reporting (via PPCallbacks::SourceRangeSkipped()) a skipped source range with start {line = 3, character = 0} and end {line = 7, character = 0}. The end of the range is the beginning of the int y1; line. However, clangd does not care about the character positions, it applies the highlighting to the whole of line 7.

Proposed fix


[Share-NG] Emtpy sharedWithMe response when received project drive as share

Describe the bug
Share project space drive root using /drives/{driveID}/root/invite . Gets empty share with me list
Similar issues in past

#8215
#8027

Steps to reproduce

create space newspace
share space with /drives/{driveID}/root/invite endpoint
list share with me

Expected behavior
The response should list the project space newspace similar to shared resource of project space
Actual behavior
{"value":[]}

Setup
Please describe how you started the server and provide a list of relevant environment variables or configuration files.

OCIS_XXX=5.1.0-prealpha+f1deebad63
OCIS_YYY=somevalue
PROXY_XXX=somevalue

Additional context
Add any other context about the problem here.

I think this is the expected behavior. The sharedWithMe endpoint is only supposed to list normal "File" and "Folder" shares. The issues you linked above are about Folder-(or File-)Shares created from within a space, not about permissions granted on a space itself.
To know which spaces a user as access to we have /me/drives/


aktualizr: bump to latest and accommodate hmi_stub.

Sprint release candidate.

Ok I'm gonna merge this, to fix image builds with latest aktualizr. We can still bump aktualizr again later


More NoUnexpectedThunks instances

This is really just one commit; the first one is https://github.com/input-output-hk/cardano-ledger/pull/646 .

bors r+

bors r+


Bootstrap 5 Sass-variable $tooltip-margin not clear or not working

When setting the V5 Sass-variable $tooltip-margin to anything but it's default 0 there seems to be no discernable effect on actual tooltips. Forcing the value by adding !important results in an offset in both x and y directions, which would be of little practical use (though noted that !important could have overriden other definitions down the line in this case).
If this behavior was replaced by the JS offset option, which is how I interpreted it, this variable should probably be removed. But feel free to close / remove this issue if this is simply lack of understanding of it's intended use.
See this Pen for simulated use of the variable.
Confirmed in Chrome, Edge, Firefox on Windows 10.

@prietveld  you are very right on this. It is a left over that will be fixed soon by #34627


add AVX512 implementation of bit unpack

What type of PR is this：

[ ] BugFix
[ ] Feature
[x] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Accelerate bit unpack with AVX512 intrinsic, get about 3~8X speedup in the microbenchmark

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 2 committers have signed the CLA.:white_check_mark: wanweiqiangintel:x: aaaaaaaaYou have signed the CLA already but the status is still pending? Let us recheck it.


security/tinc: replaces insecure xml parser

https://docs.python.org/3/library/xml.html#xml-vulnerabilities

OPNsense doesn't ship with lxml and no user provided xml is being processed here....


stop being nice

noticed that Rails/FilePath was the only cop not following the "Prefer" / "Use" convention :trollface:

This looks good. Thank you!


Support aws-sdk version3

Hi,
Could we bump the dependency to aws-sdk or there are some breaking changes preventing this?
Thank you for your work on this gem by the way!

@fernandocarletti Also with aws-sdk v3, I think we could only have dependency for the ec2 gem, since they split the library 👍

Hello @allaire. I don't think we will have any issue updating it. If you can work on this right now, let me know then I merge your PR ASAP. Otherwise, I'll check this during this week. Maybe it would be good to just bump a minor version, just in case.

@fernandocarletti I just tested it and was able to correctly call cap staging aws:ec2:instances using aws-sdk v3

Awesome! Just send the PR so I can make the release. Thank you!

On it
On Mon, Mar 26, 2018 at 4:11 PM, Fernando Carlétti<EMAIL_ADDRESS>
wrote:

Awesome! Just send the PR so I can make the release. Thank you!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/fernandocarletti/capistrano-aws/issues/1#issuecomment-376295934,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AASlTWELqtmr0mxb-Znp57HV5s8BOcUSks5tiUt_gaJpZM4S70QE
.

--
Mathieu Allaire
http://www.mathieuallaire.ca


add collection landscape of the heart

{
  "name": "BitRabbit",
  "supply": "4800",
  "slug": "BitRabbit",
  "description": "BitRabbit’s token economy design mainly includes two parts: BTR token and BitRabbit NFT Build a strong community: The community is the key to the success of BitRabbit NFT and BTR tokens. By building a positive community, attract more users and investors, and provide a good user experience and support.",
  "twitter_link": "https://twitter.com/BTR_BitRabbit",
  "discord_link": "https://discord.gg/w8mH3fMaxw",
  "website_link": "https://www.bitrabbit.xyz/"
}


movewindow silent option

Description
Add a movewindowsilent option, which moves a window to another monitor but keeps the current one in focus. (Similar to how movetoworkspace and movetoworkspacesilent behave in relation to each other)

#5597


ADBDEV-1938-orca-only

Here are some reminders before you submit the pull request

[ ] Add tests for the change
[ ] Document changes
[ ] Communicate in the mailing list if needed
[ ] Pass make installcheck
[ ] Review a PR in return to support the community

Allure report https://allure-ee.adsw.io/launch/55396

Allure report https://allure-ee.adsw.io/launch/55468

Failed job Regression tests with ORCA on x86_64: https://gitlab.adsw.io/arenadata/github_mirroring/gpdb/-/jobs/664816


Approval Needed for Next Step

Please comment 'approve' on this issue to proceed with the third workflow.

approved

approved

approved


Add paid_updates

Greetings,
I have noticed that some software on the repositories are paid_updates.
It would be interesting to add an element inside the casks :

paid_updates true | false

Then it would be possible via

brew cask outdated

to know if the update of a specific cask would break a previous licensed version.
Best regards
@Rbcafe

The main repository always has the latest version of a cask. Legacy versions can be added to caskroom/versions
- Include the latest minor version of legacy versions of commercial and freemium software.
- Legacy versions of commercial and freemium software are restricted to a maximum of five casks.


feat: add social urls and treasury address to space profile

Summary
Towards: https://github.com/snapshot-labs/sx-ui/issues/409
Towards: https://github.com/snapshot-labs/sx-ui/issues/413
This PR adds social urls and treasury address to space profile editor.
Those won't be displayed yet as those values are not indexed.
Test plan

Create space/edit space.
Check metadata contents.

Screenshots

Will draft for now, will add full flow (with displaying actual values) here, will be mergable once sx-api and sx-subgraphs PRs are merged.


[FR]Provide 1 or 2 optional config values for blind positions with actual window clearness to enable correct "sun on floor length"

Is your feature request related to a problem? Please describe.
When using "length on the floor" config option, respectively setSunDataFloorLength, calculation is based on window top and window bottom position. However, with normal blinds, typically neither window position close to 0% nor 100% directly impact the length of the sun on the floor, because:

Close to 100%, at first small slots are visible, before the blind actually starts to rise from the ground
When starting to rise from the ground, there are typically some more cm that are covered by the window frame (i.e., still no impact on the sun length on the floor)

-> As a consequence, e.g., 20% of the overall scale doesn't have an impact on the sun on the floor at all.
Therefore, calculation is wrong, and sun on the floor length does not remain constant throughout the day

Same is true for close to max blind openings, where the window frames again causes that no greater sun length on the floor can be reached

Describe the solution you'd like
Provide 1 or better 2 optional config values for the positions, where the blind actually starts to influence the sun on the floor length
1st config value for "sun on floor length min" position, e.g., as num value >= closed position (min)
2nd config value for "sun on floor length max" position, e.g., as num value <= open position (max)
Example:
closed position (min) = 0
open position (max) = 1
sun on floor length min = 0.25
sun on floor length max = 0.9
-> Accordingly, sun on floor length calculation would be based on the 2 optional values, given they are configured
Describe alternatives you've considered
You could do it externally to the blind node, e.g., by utilizing "range node", or even do the % blind position calculation to reach the sun on floor length completely externally to the node. However, IMHO, since the functionality for blind control based on sun on floor length is officially supported by the blind node, it should also provide means to do it correctly.
Additional context
Add any other context or screenshots about the feature request here.

When I think about it, you can achieve this very easily by differentiating closed position (min) and open position (max).
The calculation is much simpler than you might think. The lengthOnTheFloor value is only used to calculate where the sun falls in the window, using the top and bottom position as a limit and as a maximum and minimum for the shutter position:

So instead having floor length min and a floor length max you could change the window bottom and top position and  define a different blind open and closed position for the sun position calculation
When there covered by the window frame, then as the window bottom position in the node-settings a value should be taken as the position from when the sun calculation should be done. So the value of window bottom should be "real window bottom" + "window frame". The same for the window top which should be "real window top" - "window frame".
If I implement the desired "sun on floor length min" and "sun on floor length max", I would internally calculate only an offset for the lower fender position and the upper window position. and use this offsets for the check if the sun is below the window bottom or above the window top:

Perhaps instead of  "sun on floor length min" and "sun on floor length max"  it would be better to be able to define this offset for "window below" and "window above"?

Thank you for your comprehensive answer.
I think the issue is with the following calculation:
node.level.current = nodepos PrcToAbs_(node, (height - wBottom) / wTop - wBottom));
and the function PrcToAbs https://github.com/rdmtc/node-red-contrib-sun-position/blob/a50a6f588f425e94f1cf6b80741c41eef997f75c/nodes/blind-control.js#L34-L35

Thank you for your comprehensive answer.
I think the issue is with the following calculation:
node.level.current = nodepos PrcToAbs_(node, (height - wBottom) / wTop - wBottom));
and the function PrcToAbs
https://github.com/rdmtc/node-red-contrib-sun-position/blob/a50a6f588f425e94f1cf6b80741c41eef997f75c/nodes/blind-control.js#L34-L35
The percentage is calculated on and applied to the total range of the blind, i.e., levelTop, levelBottom.
Example:
wTop = 180; // as you proposed the real free space below the frame
wBottom = 20; // as you proposed the real free space below the frame
altitude=22; //22 degree = 0.38 rad
floorLength=200;
levelTop = 1;
levelBottom = 0;

This means:
height = tan(0.38)*200 = 80.81
(height - wBottom) / wTop - wBottom) = (80.81 - 20) / (180 - 20) = 38.01%
(node.nodeData.levelTop - node.nodeData.levelBottom) * levelPercent) + node.nodeData.levelBottom =
((1 - 0) * 38.01%) + 0 = 0.3801
This does not lead to a correct positioning of the blind, because
a) the calculation is not based on the actual window free space
b) due to the following nature of a classical blind

Level
Description

100%
Fully open: blind completely in the roller shutter box

90%
blind came out of the roller shutter box and is at the level of the window frame, i.e., starts to influences the sun that falls into the window

30%
blind already touches the window frame at the bottom, i.e., any further blind lowering does not influence how much sun falls into the window (besides a complete close that closes the slits between the slats

0%
Fully closed: blind completely down and slits are closed too

-> 38% is close to not letting any sun anymore in and definitely leads not to a sun on floor of 200.

This depends heavily on the used blind. What is required here is not two values for the length on the ground, better will be an offset for open/closed blinds.


make DefaultsKey.defaultValue publicly readable

Should make 3rd party extensions like #165 easier, and implementations of register(defaults:) possible. Can precede #189

Rebase? Sure, done :)

thanks @DivineDominion!


Deploy compiled data files to public location

A few people have suggested places where the data reports could be stored for later use.  This is a master issue compiling each of those locations.
Possibilities

Manual deployment to GitHub releases (issue #3)
Manual deployment to npm (issue #4)
storing the data in GitHub (possible, but not something we've considered as we don't want people sending pull requests against raw data.) (issue #456)
public S3 bucket

Hi, a public S3 bucket would also be great - easy to get at the data for non-technical users.

In Li, we're storing the generated reports in a public S3 bucket.  We need to make them available via a CDN / human friendly  URL, but that's where they are and will be.  Closing, as we're not updating this project.  Cheers! z


Overhaul CookieJar: more efficient, more secure, more docs.

This commit completely rewrites CookieJar so that it is more efficient while being simpler and more correct overall. At a high-level, a CookieJar now returns borrows to Cookie structures whenever possible. CookieJar and children jars (now PrivateJar and SignedJar) are distinct types. The "encrypted" child jar has been replaced by PrivateJar which uses authenticated encryption. All crypto is done via ring.
A detailed list of changes is below:

The "permanent" child jar has been removed.
A make_permanent method has been added to Cookie.
A permanent method has been added to CookieBuilder.
A named constructor has been added to Cookie.
CookieJar no longer has a generic lifetime parameter.
A CookieJar no longer contains RefCells.
CookieJar::{add_original, add, remove, clear} take &mut self.
CookieJar::new does not take in a key parameter.
CookieJar::find returns a borrow to a Cookie.
CookieJar::remove takes in a Cookie.
CookieJar::clear has been deprecated.
A CookieJar tracks its modifications more robustly, passing a new suite of unit tests.
Computing the delta is zero-cost operation.
Cookie iteration is significantly simplified.
The signed method has been removed from CookieJar.
A new Signed trait was added with a signed(&key) method.
The Signed trait is implemented for CookieJar.
SignedJar cookies sign via ring's HMAC-SHA256 implementation.
The encrypted method has been removed from CookieJar.
A new Private trait was added with a private(&key) method.
The Private trait is implemented for CookieJar.
PrivateJar cookies use AEAD (AES-256-GCM) from ring.
The openssl dependency was dropped.
A ring dependency was introduced.
All items are extensively documented.

Fixes #35.
Fixes #50.
Fixes #64.
Closes #68.
Fixes #75.
Finally, I'd like to, once again, request that I be given maintainer access to the repository/crate. I'd really like to continue improving and maintaining this crate in the future.

Also cc @briansmith (usage of ring here), although to me at least it all looked great! (and ergonomic to use)

I wish we could migrate to std::time, but unless I'm missing something, it doesn't have a concept of absolute calendar time such as Tm. I've tried working with chrono before and unfortunately am not a fan. :(
On the base64 front: I'm totally down for using something more focused, but what? I'd guess the base64 crate would be the thing to use. Do you think it's worth it? `rustc-serialize is a fairly lightweight dependency, even though it includes much more functionality than we're using.

FWIW, I released ring 0.7 with a new API for the HMAC key length recommendation; see https://briansmith.org/rustdoc/ring/hmac/fn.recommended_key_len.html. Your choice is also correct, but if you use recommended_key_len(HMAC_DIGEST_ALG) instead, then you don't need to have a separate KEY_LEN constant.

Thanks so much for the review @briansmith!

I wish we could migrate to std::time, but unless I'm missing something, it doesn't have a concept of absolute calendar time such as Tm.

Yeah that's true, all we'd get is offsets from SystemTime and such. I'd love to drop time as a dep and leave heavy lifting (if necessary) to other crates, but the time may not yet be ripe to do so.

On the base64 front: I'm totally down for using something more focused, but what? I'd guess the base64 crate would be the thing to use. Do you think it's worth it? rustc-serialize is a fairly lightweight dependency, even though it includes much more functionality than we're using.

Oh just kinda a random thought, I'd love to burn rustc-serialize to the ground but it's a private dep so we can change this at any time :)

On the base64 front: I'm totally down for using something more focused, but what? I'd guess the base64 crate would be the thing to use. Do you think it's worth it? rustc-serialize is a fairly lightweight dependency, even though it includes much more functionality than we're using.

In #78 I suggest using Z85 encoding instead of Base64 encoding. It's more space-efficient and just as compatible as Base64-encoding, AFAICT. Also there are two crates that implement Z85 already.

@briansmith oh oops sorry! I was away all weekend and been digging backwards through email so I ended up seeing that later :). In any case, sounds like a great idea to me!

Ok so API-wise I think we're all set here modulo the key length question. I'd like to not have a restriction that the keys passed in are a precise length because that seems (a) difficult to change over time and (b) surprising in terms of trying to match it.
It sounds like hashing the input is a bad idea, but could we perhaps say that you need to pass in at least X bytes of data and we use slices within that for various keys? That way we can also provide a sample command for generating the keys to pass in perhaps.

I'd like to not have a restriction that the keys passed in are a precise length because that seems (a) difficult to change over time and (b) surprising in terms of trying to match it.

Keep in mind that if/when the algorithms change, the keys need to change too. For example, let's say you switch AES-256-GCM to ChaCha20-Poly1305. Even though the key size is the same, it isn't safe to use the same key (or, rather, we assume it is unsafe and nobody has proven otherwise, nor will anybody ever do so).
Thus, if you want to have some flexibility then you need to specify, at least, a minimum key length, e.g. 256 bits. Then you can use HKDF, (see ring::hkdf::extract_and_expand) to derive the keys you need. The info parameter needs to be something that unambiguously identifies the algorithm, usage, and context, e.g. "HTTP-cookie-encrypt-and-auth;AES-256-GCM;v1.0;shared-across-origin:"+origin for the AEAD key and "HTTP-cookie-auth-only;HMAC-SHA2-256;v1.0;shared-across-origin: " + origin for the HMAC key, where origin is the origin of the cookie.
Then whenever you change anything related to the crypto in cookie-rs, you need to change the those labels. Also note that with this scheme, different origins will always have different keys. For this latter reason alone, it may be a good idea to move to the derived-key scheme.
However, still, I recommend that instead of specifying a minimum key length, you instead specify a fixed key length. In general we (crypto people) don't like variables, especially regarding key lengths. (We now generally regard the way HMAC's key was specified as a mistake.)

@alexcrichton I'm not sure I understand what you mean by "sample command for generating the keys to pass in". Do you mean like a shell command? If so, how would that be any different if we had a fixed-key length? And if you mean some example Rust code, then you should consider that most of these keys will be fixed. That is, they won't be generated at runtime. This is because (especially in production) you generally want to accept cookies that were signed the last time you started your application or by other instances of that application. As a result, you need one, fixed key.
In general, I'm in agreement with @briansmith here, as evidenced by my choice of a fixed-key length to begin with.

@briansmith that sounds like a great idea! (sorry I'm pretty unfamiliar with hkdf). I'd be totally fine though requiring a fixed key length if we can use that to derive further keys (even if we change underlying algorithms)
@SergioBenitez eh I basically just mean as a new users I see this key restriction or these random constants pass in and I'm not sure how to do so myself. Just some docs for "here's how to create your own key then pass it in".

@alexcrichton I think allowing variable sized keys or extracting keys of different lengths from one key is something that should be built on-top of cookie but not done by cookie itself. I think requiring a fixed-size key is the safest thing we can do. What's more, I don't see the 32-byte key size changing any time soon. The only algorithm I can see us switching to anywhere in the vicinity of the near future is AES-256-GCM-SIV which, if I'm not mistaken, also requires 32-byte keys.
As far as how to get these random constants...the safest way (and the way Rocket will do this) is to encode the key in a text-safe format (base64 in Rocket) and then get the byte representation of the key. It's trivial to generate a safe 32-byte random key in this way: openssl rand -base64 32.

don't see the 32-byte key size changing any time soon. The only algorithm I can see us switching to anywhere in the vicinity of the near future is AES-256-GCM-SIV which, if I'm not mistaken, also requires 32-byte keys.

But, you shouldn't use the same key for different algorithms. So if there's any possibility that the algorithm will change (and I think there is), it is better to derive the keys from the master key.

As far as how to get these random constants...the safest way (and the way Rocket will do this) is to encode the key in a text-safe format (base64 in Rocket) and then get the byte representation of the key. It's trivial to generate a safe 32-byte random key in this way: openssl rand -base64 32.

Yes, and this is why variable-length inputs are bad, because some callers are likely to pass in the base64-encoded key bytes, instead of decoding the base64 first. Using base64-encoded bytes as a key, without decoding them, is bad.

But, you shouldn't use the same key for different algorithms. So if there's any possibility that the algorithm will change (and I think there is), it is better to derive the keys from the master key.

How do we defend against this with hkdf? Use a different salt each time we change the algorithm?
Running hkdf isn't free. Again, I think if a library on-top of cookie wants to use hkdf, then so be it, but it's not something I think cookie should handle.

Yes, and this is why variable-length inputs are bad, because some callers are likely to pass in the base64-encoded key bytes, instead of decoding the base64 first. Using base64-encoded bytes as a key, without decoding them, is bad.

Indeed.

Running hkdf isn't free. Again, I think if a library on-top of cookie wants to use hkdf, then so be it, but it's not something I think cookie should handle.

Yes, I explained in https://github.com/alexcrichton/cookie-rs/pull/76#issuecomment-281863128.
I don't want to get too hung up in providing advice here, so I won't debate it further.

@SergioBenitez lets use hkdf. "This probably won't change soon" sounds like "when this changes it'll be too difficult to change when we really need to". Let's cover our bases and start out conservatively? If this really has a perf impact or something like that we can reevaluate.

Okay. Now using HKDF via a Key struct. I quite like this approach.
This should be all!

👍

@SergioBenitez do you want to do the release on this one?

Released! :)


Should datamodule's hparams be saved inside checkpoint

🚀 Feature
LDM supports save_hyperparameters method just like LM.
https://github.com/PyTorchLightning/pytorch-lightning/blob/1ef34c5ffd20a42c687682798eb7db9b1dbcdb05/pytorch_lightning/core/datamodule.py#L26
but during checkpointing, only model.hparams are saved.
https://github.com/PyTorchLightning/pytorch-lightning/blob/1ef34c5ffd20a42c687682798eb7db9b1dbcdb05/pytorch_lightning/trainer/connectors/checkpoint_connector.py#L365-L374

Motivation

A user can have data specific hparams, like batch_size, image_size for transforms etc...
There can be cases where these hparams are required again, for eg in case of transfer learning/fine-tuning and user might be able to reload the hparams used while pre-training, without remembering what hparams were used during init, if they are saved inside the checkpoint.
Moreover, if we support this, we can possibly introduce LDM.load_from_checkpoint method just like we have in LM.
Pitch

2 possible ways we can save LDM hparams:

Inject the LDM hparams inside model hparams, just like we do with logger.log_hyperparams
https://github.com/PyTorchLightning/pytorch-lightning/blob/1ef34c5ffd20a42c687682798eb7db9b1dbcdb05/pytorch_lightning/trainer/trainer.py#L1273
Create separate keys for each i.e. change hyper_parameters to model_hyper_parameters and add another datamodule_hyper_parameters.

Alternatives

One can use on_save_checkpoint/on_load_checkpoint to reload the hparams, but this is slightly inconsistent with what we have within LM.
Additional context

If you enjoy Lightning, check out our other projects! ⚡

Metrics: Machine learning metrics for distributed, scalable PyTorch applications.

Lite: enables pure PyTorch users to scale their existing code on any kind of device while retaining full control over their own loops and optimization logic.

Flash: The fastest way to get a Lightning baseline! A collection of tasks for fast prototyping, baselining, fine-tuning, and solving problems with deep learning.

Bolts: Pretrained SOTA Deep Learning models, callbacks, and more for research and production with PyTorch Lightning and PyTorch.

Lightning Transformers: Flexible interface for high-performance research using SOTA Transformers leveraging Pytorch Lightning, Transformers, and Hydra.

cc @borda @carmocca @awaelchli @ananthsub @ninginthecloud @jjenniferdai @rohitgr7

Another alternative: use the LightningCLI and forget about hparams in your classes! 🎉


DEV: FsR DeleteFromStore Failed with full template (default delete template in WATT)

DeleteFromStore is OK with simplified template, but failed with full template(which is default delete template in WATT

<fluidsReports xmlns=“http://www.witsml.org/schemas/1series” xmlns:xsi=“http://www.w3.org/2001/XMLSchema-instance” xsi:schemaLocation=“http://www.witsml.org/schemas/1series  ../xsd_schemas/obj_fluidsReport.xsd” version=“<IP_ADDRESS>">
<fluidsReport uidWell=“W-0820” uidWellbore=“WB-0820" uid=“FR-0820”>
Well 0820
WB 0820
FluidsReport 0820

The client is referring to this as an inconsistency between Fluids Reports and Log/Traj since Log/Traj handles a full transaction.  High Priority.

validated that a delete entire object (no sub elements) and delete sub element are both working now).  Note that client should not provide elements with data in a deleted query.


enum: Underlying type

Enumerations in PowerShell appear to be limited to Int32. Ideally enums should allow inheritance-style setting of the underlying type.
Steps to reproduce
PowerShell does not appear to allow an underlying type to be defined for an enum. All values are coerced to Int32.
The statement below will raise an exception:
enum Fruit { Apple = 0x0FFFFFFFFFFFFFFF }

Expected behavior
An enumeration wiith underlying type set to [Int64] to be created.
enum Fruit { Apple = 0x0FFFFFFFFFFFFFFF }
[Fruit].GetEnumUnderlyingType() -eq [Int64]

Actual behavior
An exception is throw when the attempt to cast fails.
Enumerator value is too large for a System.Int.

Environment data
> $PSVersionTable
Name                           Value
----                           -----
PSVersion                      6.1.0
PSEdition                      Core
GitCommitId                    6.1.0
OS                             Microsoft Windows 10.0.17134
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0

I'd recommend reusing the class inheritance syntax, which would be similar to C# syntax for specifying an enum type:
enum MyEnum : Int64 {
    Val1
    Val2
    Val3
}

@rjmholt For your classes collection. :-)

I suspect this was a deliberate design decision when enums were introduced. How clean was the error message @indented-automation?

Looks like this @rjmholt
At line:2 char:8
+ Val1 = 0x7FFFFFFFFFFFFFFF
+        ~~~~~~~~~~~~~~~~~~
Enumerator value is too large for a System.Int.
+ CategoryInfo          : ParserError: (:) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : EnumeratorValueTooLarge

And it may have been a deliberate decision, but given that class inheritance is a thing anyway, I don't see any particular reason to arbitrarily restrict enums in this fashion?

Yes I just tested it myself. Agreed that there's no particular reason to not support it other than because it makes the language more complicated. I suspect the original reasoning was that smaller enum values work already (and can be cast without effort) and larger enum values are a pretty obscure use case, especially when flag usage is not a primary use case.

That's fair, I suppose, but [Flags()] is a supported enum attribute, so it makes the most sense to provide the frameworks that might be necessary to use it, even if the overall need isn't immensely huge. 😄
(also, the inconsistency just bothers me somehow that class inheritance syntax exists but can't be used for enum specifically, idk)

Yeah there might be the odd .NET API out there that requires it maybe. The inheritance thing is more an expectation set by C#'s syntactic sugar -- enums can't really "inherit" (playing with the metadata, it's diabolical trying to find out what the underlying type of an enum is -- you need to look at the value__ field).


check_strandedness error with how_are_we_stranded_here

Hi,
I tried using the tool like below but it is giving an ERROR:
Initially, I loaded the modules.
module load RSeQC/3.0.0-foss-2018b-Python-3.6.6
module load how_are_we_stranded_here/626c1ab-foss-2018b-Python-3.6.6

Then it started and gave the following Error:
check_strandedness --gtf /annotation_data/GRCh38/gencode.v27.annotation.gtf --transcripts gencode.v27.transcripts.fa --reads_1 sample10_R1.fastq.gz --reads_2 sample10_R2.fastq.gz

Results stored in: stranded_test_sample10_R1_6
converting gtf to bed
using kallisto_index as kallisto index
creating fastq files with first 200000 reads
quantifying with kallisto

[quant] fragment length distribution will be estimated from the data
[index] k-mer length: 31
[index] number of targets: 200,401
[index] number of k-mers: 126,160,363
[index] number of equivalence classes: 825,248
Warning: 200401 transcripts were defined in GTF file, but not in the index
[quant] running in paired-end mode
[quant] will process pair 1: stranded_test_sample10_R1_6/sample10_R1_sample.fq
                             stranded_test_sample10_R1_6/sample10_R2_sample.fq
[quant] finding pseudoalignments for the reads ... done
[quant] processed 2,182 reads, 1,064 reads pseudoaligned
[quant] estimated average fragment length: 180.624
[   em] quantifying the abundances ... done
[   em] the Expectation-Maximization algorithm ran for 574 rounds
[  bam] writing pseudoalignments to BAM format .. done
[  bam] sorting BAM files .. done
[  bam] indexing BAM file .. done

checking strandedness
Reading reference gene model stranded_test_sample10_R1_6/gencode.v27.annotation.bed ... Done
Loading SAM/BAM file ...  Finished
Total 0 usable reads were sampled
Traceback (most recent call last):
  File "/apps/RSeQC/3.0.0-foss-2018b-Python-3.6.6/bin/infer_experiment.py", line 117, in <module>
    main()
  File "/apps/RSeQC/3.0.0-foss-2018b-Python-3.6.6/bin/infer_experiment.py", line 99, in main
    if other <0: other=0.0
TypeError: '<' not supported between instances of 'str' and 'int'
Traceback (most recent call last):
  File "/apps/how_are_we_stranded_here/626c1ab-foss-2018b-Python-3.6.6/bin/check_strandedness", line 11, in <module>
    load_entry_point('how-are-we-stranded-here==0+unknown', 'console_scripts', 'check_strandedness')()
  File "/apps/how_are_we_stranded_here/626c1ab-foss-2018b-Python-3.6.6/lib/python3.6/site-packages/how_are_we_stranded_here/check_strandedness.py", line 151, in main
    result = pd.read_csv(test_folder + '/' + 'strandedness_check.txt', sep="\n", header=None)
  File "/apps/Python/3.6.6-foss-2018b/lib/python3.6/site-packages/pandas-0.23.4-py3.6-linux-x86_64.egg/pandas/io/parsers.py", line 678, in parser_f
    return _read(filepath_or_buffer, kwds)
  File "/apps/Python/3.6.6-foss-2018b/lib/python3.6/site-packages/pandas-0.23.4-py3.6-linux-x86_64.egg/pandas/io/parsers.py", line 440, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/apps/Python/3.6.6-foss-2018b/lib/python3.6/site-packages/pandas-0.23.4-py3.6-linux-x86_64.egg/pandas/io/parsers.py", line 787, in __init__
    self._make_engine(self.engine)
  File "/apps/Python/3.6.6-foss-2018b/lib/python3.6/site-packages/pandas-0.23.4-py3.6-linux-x86_64.egg/pandas/io/parsers.py", line 1014, in _make_engine
    self._engine = CParserWrapper(self.f, **self.options)
  File "/apps/Python/3.6.6-foss-2018b/lib/python3.6/site-packages/pandas-0.23.4-py3.6-linux-x86_64.egg/pandas/io/parsers.py", line 1708, in __init__
    self._reader = parsers.TextReader(src, **kwds)
  File "pandas/_libs/parsers.pyx", line 542, in pandas._libs.parsers.TextReader.__cinit__
pandas.errors.EmptyDataError: No columns to parse from file

It looks like an issue with your kallisto index.
using kallisto_index as kallisto index
in the messages means you already have an index prebuilt and named kallisto_index in the folder where you are running this.
Warning: 200401 transcripts were defined in GTF file, but not in the index
This generally happens when you --transcripts fasta and --gtf GTF don't have the same transcript names. If you're using the Gencode annotations they should match...
I would try force rebuilding the index - either delete kallisto_index, or specify a different index name (-k) and it will build it for you.

It looks like an issue with your kallisto index.
using kallisto_index as kallisto index
in the messages means you already have an index prebuilt and named kallisto_index in the folder where you are running this.
Warning: 200401 transcripts were defined in GTF file, but not in the index
This generally happens when you --transcripts fasta and --gtf GTF don't have the same transcript names. If you're using the Gencode annotations they should match...
I would try force rebuilding the index - either delete kallisto_index, or specify a different index name (-k) and it will build it for you.

Now I don't see any problem with Kallisto index, but I still see the warning and other errors. Yes, I'm using the same version gencode annotation gtf and fasta files. Don't know why it is showing that warning.
check_strandedness --gtf gencode.v27.annotation.gtf --transcripts gencode.v27.transcripts.fa --reads_1 sample12_R1.fastq.gz --reads_2 sample12_R2.fastq.gz

Results stored in: stranded_test_sample12_R1
converting gtf to bed
generating kallisto index

[build] loading fasta file gencode.v27.transcripts.fa
[build] k-mer length: 31
[build] warning: clipped off poly-A tail (longer than 10)
        from 1550 target sequences
[build] warning: replaced 4 non-ACGUT characters in the input sequence
        with pseudorandom nucleotides
[build] counting k-mers ... done.
[build] building target de Bruijn graph ...  done
[build] creating equivalence classes ...  done
[build] target de Bruijn graph has 1284693 contigs and contains 126160363 k-mers

creating fastq files with first 200000 reads
quantifying with kallisto

[quant] fragment length distribution will be estimated from the data
[index] k-mer length: 31
[index] number of targets: 200,401
[index] number of k-mers: 126,160,363
[index] number of equivalence classes: 825,248
Warning: 200401 transcripts were defined in GTF file, but not in the index
[quant] running in paired-end mode
[quant] will process pair 1: stranded_test_sample12_R1/sample12_R1_sample.fq
                                          stranded_test_sample12_R1/sample12_R2_sample.fq
[quant] finding pseudoalignments for the reads ... done
[quant] processed 1,783 reads, 812 reads pseudoaligned
[quant] estimated average fragment length: 171.096
[   em] quantifying the abundances ... done
[   em] the Expectation-Maximization algorithm ran for 448 rounds
[  bam] writing pseudoalignments to BAM format .. done
[  bam] sorting BAM files .. done
[  bam] indexing BAM file .. done

checking strandedness
Reading reference gene model stranded_test_sample12_R1/gencode.v27.annotation.bed ... Done
Loading SAM/BAM file ...  Finished
Total 0 usable reads were sampled
Traceback (most recent call last):
  File "apps/RSeQC/3.0.0-foss-2018b-Python-3.6.6/bin/infer_experiment.py", line 117, in <module>
    main()
  File "apps/RSeQC/3.0.0-foss-2018b-Python-3.6.6/bin/infer_experiment.py", line 99, in main
    if other <0: other=0.0
TypeError: '<' not supported between instances of 'str' and 'int'
Traceback (most recent call last):
  File "apps/how_are_we_stranded_here/626c1ab-foss-2018b-Python-3.6.6/bin/check_strandedness", line 11, in <module>
    load_entry_point('how-are-we-stranded-here==0+unknown', 'console_scripts', 'check_strandedness')()
  File "apps/how_are_we_stranded_here/626c1ab-foss-2018b-Python-3.6.6/lib/python3.6/site-packages/how_are_we_stranded_here/check_strandedness.py", line 151, in main
    result = pd.read_csv(test_folder + '/' + 'strandedness_check.txt', sep="\n", header=None)
  File "apps/Python/3.6.6-foss-2018b/lib/python3.6/site-packages/pandas-0.23.4-py3.6-linux-x86_64.egg/pandas/io/parsers.py", line 678, in parser_f
    return _read(filepath_or_buffer, kwds)
  File "apps/Python/3.6.6-foss-2018b/lib/python3.6/site-packages/pandas-0.23.4-py3.6-linux-x86_64.egg/pandas/io/parsers.py", line 440, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "apps/Python/3.6.6-foss-2018b/lib/python3.6/site-packages/pandas-0.23.4-py3.6-linux-x86_64.egg/pandas/io/parsers.py", line 787, in __init__
    self._make_engine(self.engine)
  File "apps/Python/3.6.6-foss-2018b/lib/python3.6/site-packages/pandas-0.23.4-py3.6-linux-x86_64.egg/pandas/io/parsers.py", line 1014, in _make_engine
    self._engine = CParserWrapper(self.f, **self.options)
  File "apps/Python/3.6.6-foss-2018b/lib/python3.6/site-packages/pandas-0.23.4-py3.6-linux-x86_64.egg/pandas/io/parsers.py", line 1708, in __init__
    self._reader = parsers.TextReader(src, **kwds)
  File "pandas/_libs/parsers.pyx", line 542, in pandas._libs.parsers.TextReader.__cinit__
pandas.errors.EmptyDataError: No columns to parse from file

Okay... maybe not then
The other thing that raises flags to me is

[quant] processed 1,783 reads, 812 reads pseudoaligned

which eventually causes

Total 0 usable reads were sampled

You're using 200,000 reads from your fastqs and only managing to process ~1% of them. I've seen this happen in some lower quality sequencing. Have you checked that the quality is okay (fastqc) and that you don't have any major contamination (genomic or small RNA from any species - including human, or transcriptomic from any species)

Okay... maybe not then
The other thing that raises flags to me is

[quant] processed 1,783 reads, 812 reads pseudoaligned

which eventually causes

Total 0 usable reads were sampled

You're using 200,000 reads from your fastqs and only managing to process ~1% of them. I've seen this happen in some lower quality sequencing. Have you checked that the quality is okay (fastqc) and that you don't have any major contamination (genomic or small RNA from any species - including human, or transcriptomic from any species)

In fact, I tried this on multiple datasets...and I still get the same Error.

Can you send me a subset of the data you were getting the error with?

Can you send me a subset of the data you were getting the error with?

Yes, Please check the attachments.
Archive.zip

I believe I am experiencing this issue as well using the M26 (GRCm39) release from Gencode. I confirmed that I installed v1.0.1 from Pip this afternoon and based on the output, it looks like my run triggered the new code that was added in the release (link directly to line of code that prints).
$ pip freeze | grep stranded

how-are-we-stranded-here==1.0.1

$ check_strandedness --gtf ../gencode_m26_GRCm39/gencode.vM26.annotation.gtf --transcripts ../gencode_m26_GRCm39/gencode.vM26.transcripts.fa --reads_1 1_KPC46-PT_Org-1_S63_L002_R1_001.fastq.gz --reads_2 1_KPC46-PT_Org-1_S63_L002_R2_001.fastq.gz

Results stored in: stranded_test_1_KPC46-PT_Org-1_S63_L002_R1_001
converting gtf to bed
Checking if fasta headers and bed file transcript_ids match...
Can't find transcript ids from ../gencode_m26_GRCm39/gencode.vM26.transcripts.fa in stranded_test_1_KPC46-PT_Org-1_S63_L002_R1_001/gencode.vM26.annotation.bed
Trying to converting fasta header format to match transcript ids to the BED file...
generating kallisto index

[build] loading fasta file stranded_test_1_KPC46-PT_Org-1_S63_L002_R1_001/transcripts.fa
[build] k-mer length: 31
[build] warning: clipped off poly-A tail (longer than 10)
        from 757 target sequences
[build] counting k-mers ... done.
[build] building target de Bruijn graph ...  done
[build] creating equivalence classes ...  done
[build] target de Bruijn graph has 865656 contigs and contains 120547711 k-mers

creating fastq files with first 200000 reads
quantifying with kallisto

[quant] fragment length distribution will be estimated from the data
[index] k-mer length: 31
[index] number of targets: 140,670
[index] number of k-mers: 120,547,711
[index] number of equivalence classes: 508,846
[quant] running in paired-end mode
[quant] will process pair 1: stranded_test_1_KPC46-PT_Org-1_S63_L002_R1_001/1_KPC46-PT_Org-1_S63_L002_R1_001_sample.fq
                             stranded_test_1_KPC46-PT_Org-1_S63_L002_R1_001/1_KPC46-PT_Org-1_S63_L002_R2_001_sample.fq
[quant] finding pseudoalignments for the reads ... done
[quant] processed 200,000 reads, 104,629 reads pseudoaligned
[quant] estimated average fragment length: 188.124
[   em] quantifying the abundances ... done
[   em] the Expectation-Maximization algorithm ran for 785 rounds
[  bam] writing pseudoalignments to BAM format .. checking strandedness

stranded_test_1_KPC46-PT_Org-1_S63_L002_R1_001/kallisto_strand_test/pseudoalignments.bam does NOT exists.

Traceback (most recent call last):
  File "/Users/alexwenzel/miniconda3/bin/check_strandedness", line 11, in <module>
    sys.exit(main())
  File "/Users/alexwenzel/miniconda3/lib/python3.7/site-packages/how_are_we_stranded_here/check_strandedness.py", line 185, in main
    result = pd.read_csv(test_folder + '/' + 'strandedness_check.txt', sep="\n", header=None)
  File "/Users/alexwenzel/miniconda3/lib/python3.7/site-packages/pandas/io/parsers.py", line 686, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "/Users/alexwenzel/miniconda3/lib/python3.7/site-packages/pandas/io/parsers.py", line 452, in _read
    parser = TextFileReader(fp_or_buf, **kwds)
  File "/Users/alexwenzel/miniconda3/lib/python3.7/site-packages/pandas/io/parsers.py", line 936, in __init__
    self._make_engine(self.engine)
  File "/Users/alexwenzel/miniconda3/lib/python3.7/site-packages/pandas/io/parsers.py", line 1168, in _make_engine
    self._engine = CParserWrapper(self.f, **self.options)
  File "/Users/alexwenzel/miniconda3/lib/python3.7/site-packages/pandas/io/parsers.py", line 1998, in __init__
    self._reader = parsers.TextReader(src, **kwds)
  File "pandas/_libs/parsers.pyx", line 540, in pandas._libs.parsers.TextReader.__cinit__
pandas.errors.EmptyDataError: No columns to parse from file

If I look at the line [  bam] writing pseudoalignments to BAM format .. checking strandedness and compare it to venkan's output above, it almost looks as if Kallisto got cut off silently while writing a bam file. I can look for similar issues in the Kallisto repo in the meantime, but since this results in the same original error and appears to be triggering the new code, I thought it warranted a post in this issue as well.


Allow data-href to be used with data-turbo-method

data-turbo-method is a great way to create POST/PUT/DELETE actions but by directly reading href, users can still open them in new tab. And that tab would result in in 405 (method now allowed) which is not very user-friendly.
Feature request:
allow something like this:
Start processing

The order of finding action for hidden form:

data-turbo-action
href
"undefined"

From my basic understanding of JS, feature should be easy to add:
form.action = link.getAttribute("data-turbo-action") || link.getAttribute("href") || "undefined"

An  without an [href] attribute has no activation behavior.
Have you considered using a <form> and <button> pairing?
<form action="/my-url" method="post">
  <button>Start processing</button>
</form>

The example w/o href was for readability, I would have probably put href="#". Or maybe an URL to page that specifically says You can't do it, or even better, put a link to general 404 page.

Have you considered

I did, but I find single  element much easier to use when I have a table with multiple actions per row.

In this case, it is just a lots of buttons in a table, within protected /admin pages where google will never be:

Every link inside extractors and actions columns are these buttons; very easy to set them in HTML and accessibility is not an issue (just one admin).   elements are easier to customize than buttons and javascript is not an issue due to  and the fact admin will never turn it off .

But this idea is breaking too many HTML rules, feel free to close the issue. I am happy with Turbo even w/o this.


New package: MYOB.AccountRight version 2022.2

Checklist for Pull Requests

[x] Have you signed the Contributor License Agreement?
Resolve #163567

Manifests

[x] Have you checked that there aren't other open pull requests for the same manifest update/change?
[x] This PR only modifies one (1) manifest
[x] Have you validated your manifest locally with winget validate --manifest <path>?
[ ] Have you tested your manifest locally with winget install --manifest <path>?
[x] Does your manifest conform to the 1.6 schema?

Note: <path> is the directory's name containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

https://github.com/microsoft/winget-pkgs/pull/163651#issuecomment-2237563656

Depends on: https://github.com/microsoft/winget-pkgs/pull/163798

It's currently in progress, see https://github.com/microsoft/winget-pkgs/pull/163640#issuecomment-2239531624


True black theme

Please add a true black/OLED black theme option.

I'd rather not. But there's #4318.


Run the CPU in a web worker

The jor1k guys do this too: Run the CPU and devices code in a web worker, saving the time for rendering. For graphical modes, this requires a significant amount of data to be transferred between the worker and the window, so it might not actually be faster (needs to be researched).
This requires some significant changes in the code, including some sort of abstraction between the code that will run in the worker and the code that will run in the window.

Hi, I'm using v86 for JSIX.  The CPU component does run well asynchronously, i.e. under a web worker.  As for screen rendering under JSIX a message passing interface acts as the intermediary.  As for you project, I only use the CPU component, it's wineapi and my code that does the render.
If I have time in the future I'll help you out, lemme get a working release first.
Now, I don't recommend "transferable objects" as they are not widely supported [IE].  As for your framebuffer, a different approach might be better, such as diffing the data and sending what has changed.

@warpcoil JSIX sounds extremely cool, let me know if you need anything. I guess you'll at least want to hook system calls.
I will probably use transferable objects and fall back to something slower if they're not supported (that happens automatically anyway?). The data doesn't need to be diffed, it's known which pixels are drawn. But before that, some more changes to the architecture are necessary.

warpcoil, I would like to know how you solved the problem to run a web worker using 100% of the CPU but staying responsive.
I don't like my solution, which I linked above.

@s-macke setTimeout(tick, 0) should do the job. I'm aware of two limitations of setTimeout: A 1000ms limit in inactive tabs and a 4ms general limit. The 1000ms limit doesn't apply to web workers. The 4ms limit only applies if the executing function takes less than 4ms. You can see that by benchmarking function a() { var s = performance.now(); while(performance.now() - s < 4) {}; setTimeout(a, 0); }

Hello, ok this is a little complex.  In jsix I have 3 threads, the browser render thread (the main thread), the processing thread, contains my version of memory, your CPU [slightly modified], PE parser, dll Handlers and the kernel32.dll.  The third worker/thread you can ignore for now [what it basically does is download a file chunk of 8Kb and renders it to memory JIT for processing].
The browser thread spawns a new thread, the processing thread.
So the app starts be it a very small app, HxD [hex editor] and notepad++ is what I am working towards, couple of weeks to a month.  You can find these files at https://github.com/warpcoil/jsixappsv1, they are just the normal files chunked up and then compressed.  The third thread [will eventually] decompresses and renders to memory in this way we are not downloading a whole filesystem.  Dlls and Executables can become very large, and still be fast because when a call occurs only an 8Kb chunk for that function is downloaded, not the entire Dll.
So you have parsed the PE header in the processing thread and a v86 object is created.  The processor starts executing until a call is made.  When a call or jump occurs, a flag stored in the memory segment is checked to see if that call should be intercepted.  In this way we can resolve function calls outside of the program to external dlls and JavaScript functions.
When a paint occurs a postMessage is posted to the primary thread for the render.
I am using a form of iterative development, and this is the third iteration.  The first version was crap, based on MIPS, linux not good.
The second version took a different approach, it was based on your v86 processor, it downloaded entire files and dlls and what not.  It works! but only to a point that processing occurs, dlls are called and so forth, before changing tact.
This is the third and final iteration, targeting a release, most of the code is ported from iter 2 [not uploaded].  The memory architecture changed even more and I am at the point of parsing the PE header getting ready to execute.
Now, I have privatised my main repo, I am fearful that when everything works [which isn't far away now] some big business is going to come along and take, take, take and no give.
You have both been given access [and eventually used as you please].  It works [will work] at the application layer as opposed to the OS Layer.
Much of the original code hasn't been uploaded to this version but should be there before too long.
Best Regards
RL

oh darn, repo is here:
https://github.com/warpcoil/jsix, jsix, v86, kernel32.dll [to port from iter 2]
Wine on Windows dlls here: https://github.com/warpcoil/jsixw2012 all chunked up and encoded [public]
Test Apps here: https://github.com/warpcoil/jsixappsv1 all chunked up and encoded [public]

More than a year ago, I tried the setTimeout(tick, 0) solution. Unfortunately, it didn't check for any other messages in the message queue. So, I get a 100% usage of the worker thread, but it doesn't respond to external events. It works with a setTimeout(tick, 4); line But I would lose too much time performance.

Progress on JSIX.
We have a load screen:

And a window, but no menus, icons, they're all in place in the execution thread, just not in the UI thread yet.
  That said there is room for about atleast 30% optimisation and the new threading model in the next version, which downloads chunks in one web worker whilst processing them in the current one.  That separation was all planned just Chrome/Chromium needs to fix bugs in their nested web workers.  Strangle IE11 doesn't suffer from the issues Google has, shocking.
The window below is the frame of AkelPad (sans the background where you type stuff in).  Notepad++ is a little way off, I have a "SimpleWin.exe" program which works. 6 other apps are on the way too.

@warpcoil Very cool!

@warpcoil https://github.com/warpcoil/jsix is returning a 404 for me.

Now, I have privatised my main repo, I am fearful that when everything works [which isn't far away now] some big business is going to come along and take, take, take and no give.

If you license it under GPL, then you'll be OK. If they use it, they'll have to give you credit. If they modify a single line, they'll have to give people the source code.

@humphd If I remember correctly, more than just the CPU runs in the web worker, which may be the cause of that slowdown issue. Perhaps it's still open because of that?
Pinging @copy because he will know

@humphd If I remember correctly, more than just the CPU runs in the web worker, which may be the cause of that slowdown issue. Perhaps it's still open because of that?

Seems like #477 tries to address the slowdown

The slowdown should be fixed in master. We've had (experimental) worker support for a while now, so let's close this issue.


Contact Mechanics: Out of range float values are not JSON compliant

This recently happened during display of contact mechanics data:
Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/opt/conda/lib/python3.7/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/conda/lib/python3.7/contextlib.py", line 74, in inner
    return func(*args, **kwds)
  File "/opt/conda/lib/python3.7/site-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/app/topobank/analysis/views.py", line 1473, in contact_mechanics_data
    plots_json = {pn: json.dumps(json_item(plots[pn])) for pn in plots}
  File "/app/topobank/analysis/views.py", line 1473, in <dictcomp>
    plots_json = {pn: json.dumps(json_item(plots[pn])) for pn in plots}
  File "/opt/conda/lib/python3.7/site-packages/bokeh/embed/standalone.py", line 370, in json_item
    docs_json = standalone_docs_json([model])
  File "/opt/conda/lib/python3.7/site-packages/bokeh/embed/util.py", line 253, in standalone_docs_json
    docs_json, render_items = standalone_docs_json_and_render_items(models)
  File "/opt/conda/lib/python3.7/site-packages/bokeh/embed/util.py", line 294, in standalone_docs_json_and_render_items
    docs_json[docid] = doc.to_json()
  File "/opt/conda/lib/python3.7/site-packages/bokeh/document/document.py", line 901, in to_json
    doc_json = self.to_json_string()
  File "/opt/conda/lib/python3.7/site-packages/bokeh/document/document.py", line 930, in to_json_string
    return serialize_json(json, indent=indent)
  File "/opt/conda/lib/python3.7/site-packages/bokeh/core/json_encoder.py", line 166, in serialize_json
    return json.dumps(obj, cls=BokehJSONEncoder, allow_nan=False, indent=indent, separators=separators, sort_keys=True, **kwargs)
  File "/opt/conda/lib/python3.7/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/opt/conda/lib/python3.7/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/opt/conda/lib/python3.7/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)

Exception Type: ValueError at /analysis/card/contact-mechanics/data
Exception Value: Out of range float values are not JSON compliant
Request information:
USER: *********************

GET: No GET data

POST:
analysis_id = '330554'

We now have our own JSON encoder that handles some edge cases. I have not seen this occur recently.


Deploy to AWS Lambda

Interested in deploying to AWS-Lambda. Is there any official Frontity - AWS integration Template coming on the roadmap?

https://community.frontity.org/t/deploy-to-aws-lambda/814/4

The community users Sean has been able to use Cloudfront to cache everything:
https://community.frontity.org/t/deploy-to-aws-lambda/814/30

Related to #201
Will evaluate the priority of this issue when working in #201


Configuration of the new Data Sources - pie chart - Testing

since cQube 3.7 and cQube_Edu 1.5 are in merging process, hence this is not valid here. So closing this issue.


[Google Translate] ...

Extension
https://www.raycast.com/gebeto/translate
Description
Can you automatically recognize the language you entered?
I use English, Korean, and Chinese alternately. Is it possible to implement it so that you can automatically recognize the input language and translate it in a set language?
Who will benefit from this feature?
No response
Anything else?
No response

Yesss, it is a good idea, I'll do it soon

Yesss, it is a good idea, thanks! I'll try to implement it soon

Thank you!


Make sure LayoutBuilder rebuild in an inactive route

Prompted by https://github.com/flutter/flutter/issues/154060: widgets should always rebuild even when off-screen. The ancestor widget could be trying to pass down information that is not related to the UI state, or trying to pause video playback. Widgets with global keys should also always rebuild to make sure the widget tree is consistent in terms of global keys.
~Also prevents unnecessary repaints: https://github.com/flutter/flutter/issues/106306#issuecomment-1266432242~
This works by adding _RenderLayoutBuilder as a relayout boundary in the dirtly layout list so the layout callback always gets a chance to run if marked dirty.
Pre-launch Checklist

[ ] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[ ] I read the Tree Hygiene wiki page, which explains my responsibilities.
[ ] I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
[ ] I signed the CLA.
[ ] I listed at least one issue that this PR fixes in the description above.
[ ] I updated/added relevant documentation (doc comments with ///).
[ ] I added new tests to check the change I am making, or this PR is test-exempt.
[ ] I followed the breaking change policy and added Data Driven Fixes where supported.
[ ] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

Reverted the "prevent markNeedsPaint" part, I'm going to keep the tests here for PR number 3:
import 'package:flutter/material.dart';
import 'package:go_router/go_router.dart';

void main() async {
  runApp(MaterialApp.router(routerConfig: router));
}

final router = GoRouter(
  initialLocation: '/',
  errorBuilder: (_, state) => Container(color: Colors.red),
  redirect: (_, state) {
    return (state.matchedLocation != '/sign-in') ? '/sign-in' : null;
  },
  routes: [
    GoRoute(
      name: 'home',
      path: '/',
      builder: (_, state) => LayoutBuilder(builder: (context, constraints) {
        print('!!!!!');
        return const SizedBox();
      }),
      routes: [
        GoRoute(
          name: 'signIn',
          path: 'sign-in',
          builder: (_, state) => const SizedBox(),
        ),
      ],
    ),
  ],
);

(Flyby comment from triage) This has some merge conflicts now.

@LongCatIsLooong said they will come back to this pr once they finished Full keyboard access pr


Dataset for T2 measurement

Could we have a dataset (many 1D NMR spectra) that would allow to make the T2 measurement ?

13C - mostly for one signal (@70 ppm), and in file 13C-spec also for entire spectrum
list_13C_t2.txt
jcamp_t2_13C_1D.zip
jcamp_t2_13C_2D.zip
jcamp_t2_13C-spec_2D.zip

For 1H (single signal)
jcamp_t2_1H_1D.zip
jcamp_t2_1H_2D.zip
list_t2.txt

@lpatiny
it's done


Fixed client-side IP validator

I found a problem with client-side IP validation.
When there validated value does not have a CIDR, the matches[4] that represents CIDR part is undefined instead of expected empty sting.
It leads to a wrong comparison in the if statement below, causing invalid behavior.
My patch fixes the default value for the described case and restores expected behavior.

Merged, thank you!


block device cgroup options don't behave as expected

Output of docker version:
# docker version
Client:
 Version:      1.10.2
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   c3959b1
 Built:        Mon Feb 22 21:37:01 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.10.2
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   c3959b1
 Built:        Mon Feb 22 21:37:01 2016
 OS/Arch:      linux/amd64

Output of docker info:
# docker info
Containers: 74
 Running: 3
 Paused: 0
 Stopped: 71
Images: 19
Server Version: 1.10.2
Storage Driver: overlay
 Backing Filesystem: extfs
Execution Driver: native-0.2
Logging Driver: json-file
Plugins:
 Volume: local
 Network: bridge null host
Kernel Version: 3.19.8-031908-generic
Operating System: Ubuntu 14.04.4 LTS
OSType: linux
Architecture: x86_64
CPUs: 32
Total Memory: 58.97 GiB
Name: ip-10-65-89-167
ID: MLIQ:P4LO:CPCI:EMNV:DQ47:TRZL:QR3C:QOOE:4GSM:J5QY:K734:EH3S
Debug mode (server): true
 File Descriptors: 38
 Goroutines: 95
 System Time: 2016-03-22T20:20:10.305853814Z
 EventsListeners: 0
 Init SHA1: 3e247d0d32543488f6e70fbb7c806203f3841d1b
 Init Path: /usr/lib/docker/dockerinit
 Docker Root Dir: /var/lib/docker
WARNING: No swap limit support

Additional environment details (AWS, VirtualBox, physical, etc.):
AWS EC2, docker on root volume which is EBS SSD.
Steps to reproduce the issue:
# df -h | grep /mnt
/dev/xvdb       315G  8.1G  291G   3% /mnt

This uses cgroups to limit the bps for a given device, however it isn't the device we are writing to yet we notice extremely long execution times as if it is being constrained.
# docker run -it --device-write-bps /dev/xvda:4kb -v /mnt:/opt ubuntu:14.04 /bin/bash -c 'time $(dd if=/dev/zero of=/opt/testfile bs=1k count=1 && sync)'
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 6.88e-05 s, 14.9 MB/s

real	1m11.310s
user	0m0.000s
sys	0m0.016s

And here it is without the cgroup settings and we can see it is much faster, even though we weren't constraining this device to begin with.
# docker run -it -v /mnt:/opt ubuntu:14.04 /bin/bash -c 'time $(dd if=/dev/zero of=/opt/testfile bs=1k count=1 && sync)'
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.000100221 s, 10.2 MB/s

real	0m0.028s
user	0m0.000s
sys	0m0.016s

It may be because /dev/xvda is where /var/lib/docker is but that seems like a bug still if we aren't limiting the device of the location we are writing to and it's still limited.
UPDATE:
More unexpected behavior. This shows 1) limiting the throughput for a device we aren't using in the container limits the throughput of those operations in the container 2) If we limit the throughput of a device via docker, then the throughput is limited for the entire host for that device...
# df -h
Filesystem      Size  Used Avail Use% Mounted on
udev             30G   12K   30G   1% /dev
tmpfs           5.9G   17M  5.9G   1% /run
/dev/xvda1      126G   15G  107G  12% /
none            4.0K     0  4.0K   0% /sys/fs/cgroup
none            5.0M     0  5.0M   0% /run/lock
none             30G     0   30G   0% /run/shm
none            100M     0  100M   0% /run/user
/dev/xvdb       315G  8.1G  291G   3% /mnt

# time dd if=/dev/zero of=/opt/testfile bs=1k count=10 && sync
10+0 records in
10+0 records out
10240 bytes (10 kB) copied, 0.000200592 s, 51.0 MB/s

real	0m0.001s
user	0m0.000s
sys	0m0.000s

# docker run -d --device-write-bps /dev/xvda:4kb -v /mnt:/opt ubuntu:14.04 /bin/bash -c 'time $(dd if=/dev/zero of=/opt/testfile bs=1k count=10 && sync)'
af6bea445adb705dda8339036849f67d26d160d25177476b5a82077ee3033841

# time dd if=/dev/zero of=/opt/testfile bs=1k count=10 && sync
10+0 records in
10+0 records out
10240 bytes (10 kB) copied, 13.0005 s, 0.8 kB/s

real	0m13.483s
user	0m0.000s
sys	0m0.000s

I can't reproduce this on my system, it might be an issue with the Xen virtual disk.
Could run this command in the container: for f in /sys/fs/cgroup/blkio/blkio.throttle.* ; do echo $f ; cat $f ; done

Hi @mlaventure, Thanks for trying to reproduce.
running that command, It looks like as I'd expect it:
# docker run -it --device-write-bps /dev/xvda:4kb -v /mnt:/opt ubuntu:14.04 /bin/bash
root@9f9b5457e21a:/# for f in /sys/fs/cgroup/blkio/blkio.throttle.* ; do echo $f ; cat $f ; done
/sys/fs/cgroup/blkio/blkio.throttle.io_service_bytes
202:0 Read 4096
202:0 Write 0
202:0 Sync 0
202:0 Async 4096
202:0 Total 4096
Total 4096
/sys/fs/cgroup/blkio/blkio.throttle.io_serviced
202:0 Read 1
202:0 Write 0
202:0 Sync 0
202:0 Async 1
202:0 Total 1
Total 1
/sys/fs/cgroup/blkio/blkio.throttle.read_bps_device
/sys/fs/cgroup/blkio/blkio.throttle.read_iops_device
/sys/fs/cgroup/blkio/blkio.throttle.write_bps_device
202:0 4096
/sys/fs/cgroup/blkio/blkio.throttle.write_iops_device

on the host (not in a container):
root@ip-10-168-94-223:~# ls -l /dev | grep 202
brw-rw---- 1 root disk    202,   0 Mar 30 18:11 xvda
brw-rw---- 1 root disk    202,   1 Mar 30 18:11 xvda1
brw-rw---- 1 root disk    202,  16 Mar 30 18:11 xvdb
brw-rw---- 1 root disk    202,  32 Mar 30 18:11 xvdc
brw-rw---- 1 root disk    202, 112 Mar 30 18:11 xvdh

I haven't found any reports of Xen causing issues, but this may be an uncommon use case?

The output does indeed seem normal.
Writing to the cgroup file is as far as docker goes for this feature. It would point more likely to an issue with the underlying setup, maybe the kernel?
If you spin another identical AMI does it have the same behavior?
The only other difference I can see is that I did my test on 1.10.3 and a 3.13.0-79-generic kernel

Closing because of lack of activity, but let us know if this is still an issue


[bazel] Refactor bitstream splicing

Create a splicing python script that performs all aspects of splicing in a single operation and caches splice results.  We're keeping a cache of bitstreams so that tests can more easily express their ROM/OTP requirements in the test rather than needing additional splice targets.  This is especially relevant for SiVal and addition of new FPGA exec_envs.

SiVal can more easily audit the set of OTP configurations needed by the tests.
New FPGA exec_envs can specify a base bitstream and rely on the test setup to create the needed splices (rather than needing to manually create splices for each supported environment).

Spliced bitstreams are identified by the hash-of-hashes that are used in their creation: the original input bitstream and any ROM/OTP images and their corresponding MMI files.

Rewrite the bazel splicing rules to use the splicing script.

Generate bitstreams during test setup.

Superseded by #20332.


catch up with CI: rename controller's credentialsrequest to minify version

To catchup with this CI change.

/test ci/prow/e2e-aws-operator

/test e2e-aws-rosa-operator

/label docs-approved
/label qe-approved
/label px-approved
Proactive PR to prevent CI failures in the future.

/remove-label docs-approved
/remove-label qe-approved
/remove-label px-approved

/test e2e-aws-rosa-operator

/test e2e-aws-rosa-operator

/test e2e-aws-rosa-operator

/test e2e-aws-rosa-operator

Simple fix for maintaining CI on release-1.0
/lgtm
/approve

/label jira/valid-bug
Ref: https://github.com/openshift-eng/jira-lifecycle-plugin/issues/71

/label docs-approved
/label qe-apporved
/label px-approved
Minor change to sync with the CI step change, no need to ping all the parties.

/remove-label docs-approved
/remove-label px-approved

/label docs-approved
/label qe-approved
/label px-approved


[Sozo] Events cmd to retrieve events

I'm using starknet-probe (awesome tool btw), to retrieve world events. I think would be useful to have in sozo as well
Maybe something like:
sozo events --world [address] [event_names] --from-block xx --to-block xx

I can work on this. @broody can you explain how do you use starknet-probe to retrieve world events?

#LFG!
You could use the example project try this:
katana --seed 0
sozo migrate // executed in ecs examples folder
probe events --chunk-size 1 // dumps all events
Current world events are defined here: https://github.com/dojoengine/dojo/blob/main/crates/dojo-core/src/world.cairo#L48
Would be cool to have something like this dump out data from specific events
sozo events ComponentRegistered,PlayerJoined

@broody review please :)

Requested @kariy as this is his domain :)


Decrease observer calls

This commit avoids to use three times of the same observer on three
properties and sets one observer for all of those three properties. This
reduces the number of calls. For example, on the demo page, the observer
was triggered 25 times instead of 9 times with the new version. As you
can see, the default value of the date is the current one.
Closes #15

I'll use debounce to prevent multiple executions.

Using debounce has some side effects. It seems that setting a default value for date and use single observer is the best approach now.
Could you update the PR and add a unit test for setting default date value?


Feature/sensitive options

Update to latest sdk version and set the influxdb password as a secret option, this will prevent it from leaking the secret value set via an envvar into the usage message string.

Rolled this same change into #42 , closing this one.


couldn't unmarshal the config file: yaml: line 1: did not find expected hexdecimal number

OS: windows 10
command executed:
nehm get https://soundcloud.com/ashatack/28th-july-the-just-on-track-showcase-with-ashatack-2016
error message:
couldn't unmarshal the config file: yaml: line 1: did not find expected hexdecimal number
contents of .nehmconfig file:
permalink: "krakenskulls"
dl_folder: "c:\users\ryan\music"

It seems that you have extra spaces in your nehm config file: https://github.com/tenderlove/psych/issues/190#issuecomment-162658663
Try to delete all possible extra spaces in your nehm config
Or try to just copy past these lines:
permalink: krakenskulls
dl_folder: c:\users\ryan\music

Interesting. You are right, there were some trailing spaces after the values and before the newlines. Now when I execute the same command, I get this:

Downloading 28th July -The Just On Track Showcase With Ashatack - 2016
download failed: exec: "curl": executable file not found in %!P(MISSING)ATH%!(NOVERB)

But that is a separate issue I suppose.

@redwards510 yes, it is exactly separate issue. On windows there are no pre-installed curl command. That's why this bug occurs.
Try to install curl like here

@bogem I do have curl installed in my path though... I'm not sure what %!P(MISSING)ATH%!(NOVERB) is though.

I'm also don't know what is. I think, in future releases I will remove dependency from curl and build in my own tiny downloader.


test pr for ci changes in 6745

At the time of this PR, aaron-prindle/skaffold:main branch has the changes in https://github.com/GoogleContainerTools/skaffold/pull/6755.  This PR is to verify that CI passes and works as expected with the concurrency changes

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@08852b0). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main      #87   +/-   ##
=======================================
  Coverage        ?   69.53%
=======================================
  Files           ?      536
  Lines           ?    24432
  Branches        ?        0
=======================================
  Hits            ?    16990
  Misses          ?     6307
  Partials        ?     1135

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 08852b0...9cdb6c2. Read the comment docs.


New version: Google.AndroidStudio.Canary version 2023..3.1.6

[ ] Have you signed the Contributor License Agreement?
[ ] Have you checked that there aren't other open pull requests for the same manifest update/change?
[ ] This PR only modifies one (1) manifest
[ ] Have you validated your manifest locally with winget validate --manifest <path>?
[ ] Have you tested your manifest locally with winget install --manifest <path>?
[ ] Does your manifest conform to the 1.5 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

Automatic Validation ended with:

2024-01-25T03:13:29.0893102Z ##[error] Manifest Error: DisplayVersion declared in the manifest has overlap with existing DisplayVersion range in the index. Existing DisplayVersion range in index:  [ [2023.3, 2023.3]]

(Automated response - build 655.)


WIP: Refactor ChunkReader.Chunk() to ChunkOrIterable()

This change is related to OOO native histogram support: https://github.com/prometheus/prometheus/issues/11220.
Problem
oooMergedChunk currently implements the Chunk interface (i.e. it's considered a chunk). A chunk cannot have a histogram counter reset in the middle of it. With OOO native histograms, oooMergedChunk can break this expectation. This is because  oooMergedChunk can contain several OOO native histograms chunks. These chunks can start with counter resets. If you merge the chunks together, then there could be counter reset in the middle of the oooMergedChunk.
The reason that oooMergedChunk implements the Chunk interface is that the querier, specifically the populateWithDelChunkSeriesIterator, uses a ChunkReader with the Chunk(chunks.Meta) method. This method expects only a single Chunk to be returned. A single chunks.Meta can actually refer to multiple chunks for OOO (memSeries.oooMergedChunk finds all of the chunks within a given mint and maxt for a given chunks.Meta).
This is problematic as during compaction (which uses populateWithDelChunkSeriesIterator), an oooMergedChunk could be written as a single chunk in a block, even if it has a counter reset in the middle. Also if any tombstones are applied, the iterator will iterate through all of the samples in the oooMergedChunk and attempt to append it to a single chunk - this will error if there's a counter reset in the middle of oooMergedChunk.
Proposal
My proposal is to change the ChunkReader.Chunk() to ChunkOrIterator() - so it'll return either a single chunk or an iterator. The caller will have to convert the samples in the iterator into chunks. This could result in multiple chunks.
The other changes required for OOO native histograms are currently being done in https://github.com/grafana/mimir-prometheus/tree/cedwards/ooo-native-histograms. This change has been separated into its own PR so the proper OOO PR will be smaller.
Further advantages of the refactoring

This is also useful for the case when there are multiple different encodings in a single oooMergedChunk - we will create a new chunk if the encoding for the next sample is different from the previous one.
Improves performance of compacting OOO chunks in the head. Currently Chunk.Bytes() is called multiple times during compaction. This is an expensive option for oooMergedChunks as that iterates through all of the samples in its underlying OOO chunks every time bytes is called. Now the populateWithDelChunkSeriesIterator will use the iterator returned from ChunkOrIterator() to create proper chunks which have more efficient Bytes() functions.  See benchmark comment below for stats.
I was able to remove some methods from oooMergedChunks and boundedChunk that were only implemented to satisfy the Chunk interface. Now these struct only need to satisfy the chunkenc.Iterable interface that has a single Iterator() method.

Behaviour change

If there are multiple encodings inside a chunk, populateWithDelChunkSeriesIterator will create multiple chunks rather than erroring, even for in-order chunks. I think this is an acceptable change. I don't think it's currently possible to hit this case with in-order chunks, or with float-only OOO support. TODO: double check

Additional refactoring left for later
There is additional refactoring I'd like to consider (e.g. moving Chunks from chunkenc.Meta since with OOO, it's not guaranteed a single chunkenc.Meta will map to a single Chunk) but this PR is big enough that I'll leave that for another PR once this one is done.
Discarded solutions

Returning a chunk slice from Chunk(chunks.Meta). The samples would be read once within Chunk(chunks.Meta), but we might need to read and copy the samples again if any tombstones are applied, or if the per-sample iterator was used (populateWithDelSeriesIterator).
Keep returning a "Chunk" for oooMergedChunks, but in populateWithDelChunkSeriesIterator actually split that into multiple chunks. It looks less messy and less complicated for callers to handle if there's only a single thing returned from the Chunk() function but we could easily introduce errors later on if the Chunk() method was used elsewhere and we didn't do the oooMergedChunks check.
Always return an iterable. This was discarded because if a single chunk is already available and doesn't need modifications, it's not worth reading the samples and copying to a new chunk that will be exactly the same.

already loving the fact that Appender.AppendHistogram doesn't need the "appendOnly" bool flag. I've added it out of necessity, but glad to see it go.

@krajorama I decided to remove the appendOnly change from this PR so that there are fewer changes to review. I'll open another PR for that after this one is closed.

Benchmarking
I ran the compaction benchmarks on my computer (go test -run=XXX -bench=BenchmarkCompaction -count=5 -timeout=50m) since this change affects that. I added some simple OOO compaction tests in https://github.com/prometheus/prometheus/pull/13123/commits/bbc0d09a2452b9513907ae95dfc22d01fe3cadcc.

There is a slight memory increase for some of the non-ooo compaction cases (up to +1.42%).
An increase in latency for three of the non-ooo compaction cases (up to +9.00%) but the latency seems to vary a lot between each run so not sure how significant that is.
The ooo compaction speed and latency has improved as expected (since we're not iterating through the OOO samples multiple times anymore).

I think the reason for the increase in non-ooo compaction cases is because populateWithDelChunkSeriesIterator is appending chunks created from currDelIter to a slice, instead of just setting the chunk as a field in the iterator struct. The slice is necessary because in the case of ooo native histograms, it's possible to end up with several chunks when reading currDelIter. But the other cases (in order chunks, ooo floats) are more likely to happen.

I'll experiment more with the slice used in populateWithDelChunkSeriesIterator (see last section of https://github.com/prometheus/prometheus/pull/13123#issuecomment-1814346958) to see if that helps. Also one alternative would be to have separate chunk iterators for ooo and in order chunks, but that would take a lot more refactoring.

/prombench main

Made some optimisation attempts in https://github.com/prometheus/prometheus/pull/13123/commits/f4a4f9fb720e7a7a11b54582f59b2a6ffd12ae50 and https://github.com/prometheus/prometheus/pull/13123/commits/61e2b8695a14215d39ce09ebf65191e0d1a77c37. From local benchmarks, not sure if that helped with anything. However, we noticed that this PR hadn't been rebased recently, and there were changes like https://github.com/prometheus/prometheus/commit/f13bc1a5c964504dc2953c6dc8420a70874e0d2e that might have affected the comparison. I've reverted my optimisation attempt and rebased, and prombench is looking a lot better.
From the increased memory usage in the local compaction benchmarks (https://github.com/prometheus/prometheus/pull/13123#issuecomment-1814346958), those are because of the new fields that were added to the populateWithDelChunkSeriesIterator struct (chunksFromDelIter, chunksFromDelIterIdx). Even if the fields are not touched, the memory is still increased. This doesn't seem to affect memory usage when running the prombench benchmarks.
I've just added a couple of new commits.
https://github.com/prometheus/prometheus/pull/13123/commits/488b88f32042357567504d20cdbdd0760a1bda14 skips resetting the chunks slice in populateWithDelChunkSeriesIterator.Next() if we've not been reading from the deletion iterator as it's unnecessary. That might be a slight improvement for the in-order compaction path - for in-order chunk metas, populateWithDelGenericSeriesIterator.next() will return a single chunk rather than a deletion iterator unless there are tombstones for deleted data.
https://github.com/prometheus/prometheus/pull/13123/commits/e7c2de2276eed2f583c4bb284c42719a8661efd1 makes sure the chunks slice is reset when reset() is called.

/prombench cancel

/prombench main

In https://github.com/prometheus/prometheus/pull/13123/commits/9bc41a17447a2e9eb9e81f159e12c4981a111dbc, I split the reading of currDelIter into two functions - one to be called when a single chunk is returned from the chunk reader (i.e. for in-order cases), and the other to be called when an iterable is returned from the chunk reader (i.e. for ooo cases). This was done to optimise the in-order case since you know there's only going to be at most one chunk created (e.g. if there's only one chunk you don't need to add chunks created from currDelter to a slice, you can just set the chunk in currMetaWithChunk directly). This seems to have helped with compaction latency in prombench which now looks similar for this PR vs main.
Due to this change, we're not going to be able to get rid of the appendOnly parameter inAppendHistogram, since it's still being used for the single chunk case (cc @krajorama).

There is a bug I noticed with the chunks iterator, where if currDelIter doesn't return any samples for a meta, then we don't read the samples from the remaining metas. See this test which currently fails: https://github.com/prometheus/prometheus/commit/e2156bc8e12589aeee9f607d2c1dcd57a012487f. This bug existed before my PR and my PR doesn't fix that - once this is merged, I'll make another PR for that.

/prombench cancel


ARROW-236 Added CHANGELOG.md

We need to add the file to MANIFEST.in:
missing from sdist:
  CHANGELOG.md


Callback url is accessible after logout.

Hello there,
Thank you for using our module. bKash sometimes makes it difficult for people. But this is approved by bKash and is widely used in Bangladeshi hosting providers.
WHMCS doesn't have any way to prevent the file to access but you can write rules for .htaccess file or something else. But the file will not work or execute if the user has been logged out.
You can more see here: https://github.com/NawabHost/bkash-checkout-for-whmcs/blob/main/modules/gateways/callback/bkashcheckout.php#L442-L444

After many discussions, they understand this issue. Now they want bKash query API implementation data. This module can implement query API?


Adding SSMLTextBuilder helper type to help build SSML strings.

Convenience types and some methods to make it easier to build an SSML output string.

Here is a sample of what it would look like using this:
result := builder.
	AppendEmphasis("Some emphasis", "strong").
	AppendAmazonEffect("whisper it", "whispered").
	Build()

response := skillserver.NewEchoResponse()
response.OutputSpeechSSML(result)

This is a cool idea. Can you break the SSMLTextBuilder code out in to it's own file just to keep things clean?

Ya i will go ahead and make that change. do you want all of the SSML stuff in that file? Like OutputSSML for example? or just the new SSMLTextBuilder stuff?

I guess it probably makes sense to put all the SSML in there for consistency sake.

Split the SSML stuff into a new file. Added an append method for plain speech that could be used in between special effects.

Don't kill me, but after reviewing this change now and I think we should leave your SSML builder code in one file and move the original SSML function back to echo.go. Happy to get other input, but I think that makes the most sense. Sorry for the change.
If you can put those two functions back, I'll merge this in.

Fixed. Does GitHub provide an option to squash commits when merging them in? I guess I should have done that on my side maybe.

Any other changes you would like to be made?

Looks good. Merging in.


upgrade(flow-bin): ^0.21.0 -> ^0.22.0

Current coverage is 47.72%

Merging #591 into master will not affect coverage as of 764cb1d

@@            master    #591   diff @@
======================================
  Files            6       6
  Stmts           44      44
  Branches         0       0
  Methods          0       0
======================================
  Hit             21      21
  Partial          0       0
  Missed          23      23

Review entire Coverage Diff as of 764cb1d

Powered by Codecov. Updated on successful CI builds.


php api 500 error

in my local server i did all the instructions that appear in github and it's worked great.
but now when i moved it to my cloud server it's give me
500 - Internal server error.
you can see it here
http://control-center.perspective-media.me/modules/sms/examples/send.php
I copy all my php folder from local to cloud so it's the same setting and the same files
just for the testing i gave all permission to the sms folder.
i tried everything and i don't get nothing.
the only thing that i get in error log:
PHP Warning: PHP Startup: pthreads: Unable to initialize module
Module compiled with module API=20151012
PHP compiled with module API=20131226
These options need to match
in Unknown on line 0
please heip

This seems to be an issue with the pthreads extension; however, I'm not sure what dependency in the library requires that extension. Are you sure this error happens only when using the library, and not with any php file in the same environment?

Im sure 100%
To be 100% I did the following tests:

  Try on another vm and it worked

  Test on few php versions

  Test on local url and few other domain

  Test on few location of folders with all permissions

In my local server it's worked so I copy the same setting and the same php folder to my remote cloud and didn't work.
When I tried to do it locally url I get this error on the browser.
C:\php\v5.6\php-cgi.exe - The FastCGI process exited unexpectedly
Module
FastCgiModule
Notification
ExecuteRequestHandler
Handler
php-5.6.24
Error Code
0x000000ff
I spent on it for hours and didn't find solutions.
Please help me.
I can send you php info if you need but to private mail.
Thank you for your replay
Avi
From: Tim Lytle<EMAIL_ADDRESS>Sent: Sunday, January 8, 2017 2:38 AM
To: Nexmo/nexmo-php<EMAIL_ADDRESS>Cc: topart1<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [Nexmo/nexmo-php] php api 500 error (#18)
This seems to be an issue with the pthreads extension; however, I'm not sure what dependency in the library requires that extension. Are you sure this error happens only when using the library, and not with any php file in the same environment?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/Nexmo/nexmo-php/issues/18#issuecomment-271120661 , or mute the thread https://github.com/notifications/unsubscribe-auth/AWVm-A4ne-ttFOeVUliyQmGd8LGVBxFkks5rQC_ogaJpZM4LdhcD .  https://github.com/notifications/beacon/AWVm-Mxkgy3KKuoG8YDNNLCNrfd2Ljkjks5rQC_ogaJpZM4LdhcD.gif

Pretty sure that error has something to do with your configuration. However, it's just a warning, so your script should still run. Are you getting any other errors in your error log (not the browser)? Perhaps put some debug logging around the use of the library?