MM-46408: adds priority in posts

Summary
Enables users to pick a priority for their message.
Priority takes a value between 'standard', 'important', and 'urgent'.
Posting a message with a priority different than 'standard' will add a
label with that priority in the messages's header.
PS: pending the Give feedback function, and the tour.
Ticket Link
https://mattermost.atlassian.net/browse/MM-46408
https://mattermost.atlassian.net/browse/MM-46414
Related Pull Requests

Has server changes (https://github.com/mattermost/mattermost-server/pull/20875)

Screenshots

normal
compact

picker

Release Note
Added an option to select a priority label for new messages. Labels are 'standard', 'important', and 'urgent'.

@rmonga71 can you help take a first pass PM review this week?
@rmonga71 do we have a ticket for what the "Give Feedback" button does?

@rmonga71 do we have a ticket for what the "Give Feedback" button does?

It should send a feedbackbot survey, I'll add it to this PR once I figure out how to send one 😅

@koox00 it will likely require changes to the feedbackbot plugin that runs the nps surveys. https://github.com/mattermost/mattermost-plugin-nps
@hmhealey might be able to help with guidance if needed to accomplish the desired UX

@koox00 it will likely require changes to the feedbackbot plugin that runs the nps surveys. https://github.com/mattermost/mattermost-plugin-nps

@esethna is this something we'll be doing going forward with new features? Adding a link to generate the Feedbackbot in new features released in beta? I really like the idea I'm just worried about the level of effort required, especially considering we need to make sure we get the telemetry right. I was thinking an interim step could be to link to the MPA feature channel (or a Google form even).
@koox00 this looks really great, thank you again for such a quick turn-around (and for nailing the UX). Some quick comments:

The message priority labels should render in permalink previews.
In the picker, I think the relative sizing may need to be adjusted slightly. Specifically, the the "Cancel" and "Apply" button outlines a look a little bigger than designed
Similarly, the text in the the "Cancel" and "Apply" buttons should be smaller than the "Standard", "Important", and "Urgent" options (12px vs 14px according to the designs)
The "Beta" label in the picker needs a little more padding beside the Message Priority text
While adding a label is only available for Root messages, the label should still render on the root message when you open up a thread in the RHS:

The label should also be shown in the list of saved posts:

The separation between the message timestamp and the priority label needs a little more padding:

The text in the labels once a message should be evenly centered in the background box (looks like the text might be slightly closer to the top and to the right):

@rmonga71 thank you for the review!

The message priority labels should render in permalink previews.

✅

In the picker, I think the relative sizing may need to be adjusted slightly. Specifically, the the "Cancel" and "Apply" button outlines a look a little bigger than designed
Similarly, the text in the the "Cancel" and "Apply" buttons should be smaller than the "Standard", "Important", and "Urgent" options (12px vs 14px according to the designs)

removed the buttons based on @matthewbirtch feedback.
however, I had used the sass mixins we have for buttons, which should be defined according to the design spec.

The "Beta" label in the picker needs a little more padding beside the Message Priority text

✅

While adding a label is only available for Root messages, the label should still render on the root message when you open up a thread in the RHS:

✅

The label should also be shown in the list of saved posts:

✅

The separation between the message timestamp and the priority label needs a little more padding:

✅

The text in the labels once a message should be evenly centered in the background box (looks like the text might be slightly closer to the top and to the right):

✅

@matthewbirtch thank you for the review

If we plan on releasing this piece of the feature for a while without the other pieces, we should leave out the Cancel and Apply buttons and just make it a simple menu selection. The extra click feels unnecessary without the other settings. When we add in the acknowledgement and persistent notification pieces, then we could include the Apply button. cc @esethna

✅

I think we should leave out the Give feedback button until we add the rest of the features. This will also make this popover more of a simple menu and we can go much narrower on the width.
Here's what I'm thinking:

✅

Can we align the popover with the left edge of the icon button? Right now it's kind of aligning arbitrarily

✅
PS: at first I had aligned it similarly to the emoji picker, which has a right alignment but a bit offset.

The padding on the BETA tag is not equal on the top and bottom and is causing the label to look vertically misaligned

✅

The padding on the URGENT and IMPORTANT labels are also not equal as @rmonga71 mentioned

✅

Also agree with @rmonga71 that we need to add 4px more margin between the timestamp and the priority label in the message header

✅

Can we make the popover title sentence case: "Message priority"

✅

Can we ensure the message input field regains focus once I select a priority option? Right now if I choose a priority the cursor and focus does not return to the message input field to write my message.

✅

For the priority label that appears within the message input, can we reduce the space below it and the actual message text? I think we can tighten this up by 4px or so.

✅
PS: the bottom of the label is indeed 0, the space we see is the textbox's top padding, I'm adding a negative margin top (-3px) to the textarea just only when labels exist.

For the x that appears next to the label when hovering over it within the message input field, can we reduce its left margin to 6px (rather than 10px)

✅

Can we add a tooltip to x that appears next to the label within the message input field? It could say "Remove important label" / "Remove urgent label"

✅

If I edit a message, I am unable to edit its priority. Was that left out intentionally? We should allow this as outlined here: https://www.figma.com/file/cHuA3plNM4R4GuKZcEfuqG/MM-43840-Message-Priority---Webapp?node-id=1091%3A123028

the edit assumes we have the advanced text editor for editing posts as well, which we don't.
If we need to add that I'd prefer it would be a different issue, unrelated to the post priority one.

As @rmonga71 mentioned, the priority label does not show in the RHS thread view or in the saved messages, search results, recent mentions, etc

✅

Nice work @koox00. Looks like you've resolved all issues - with the exception of the below:

23. If I edit a message, I am unable to edit its priority. Was that left out intentionally? We should allow this as outlined here: https://www.figma.com/file/cHuA3plNM4R4GuKZcEfuqG/MM-43840-Message-Priority---Webapp?node-id=1091%3A123028

the edit assumes we have the advanced text editor for editing posts as well, which we don't. If we need to add that I'd prefer it would be a different issue, unrelated to the post priority one.

We can add this in another PR - could you create a ticket for this? (sounds like it must be dependent on adding the advanced text editor to editing posts)

Oh, one other thing I just noticed

Is it possible to hide the tooltip as soon as I click on the button to show the priority popover? Right now I get the tooltip showing beyond the popover while I'm hovering over the button, but it should be hidden in this state.

We can add this in another PR - could you create a ticket for this? (sounds like it must be dependent on adding the advanced text editor to editing posts)

Essentially we need two tickets, one for adding advanced text editor to editing posts, and the second for adding/confirming post priority works on edit.

hey @hmhealey, I just want to bring your attention to this PR since I'm using post.props
to add a priority to messages e.g: post.props.priority = 'important'.
Semantically, IMO, it suits perfectly to use post.props for this,
but after the dev meeting where you spoke about the "danger" of untyped props I wonder
if we should take another route.
What's your take on this?

/update-branch

post.props seems fine for this. I think it's similar to a lot of other features where it might be nice to add a separate column for it, but the Posts table is the absolute worst one to add a column too.
I didn't mean to make it sound like using post props was inherently dangerous, but I think it's something to be careful when using since it's poorly documented and not validated super well. In this case, we're presumably just checking something like post.props.priority === 'important' which will never cause an error even if priority is an invalid type or undefined

@koox00 looks good, you can dismiss Rohan's review (I can't seem to do figure out how)

post.props seems fine for this. I think it's similar to a lot of other features where it might be nice to add a separate column for it, but the Posts table is the absolute worst one to add a column too.
I didn't mean to make it sound like using post props was inherently dangerous, but I think it's something to be careful when using since it's poorly documented and not validated super well. In this case, we're presumably just checking something like post.props.priority === 'important' which will never cause an error even if priority is an invalid type or undefined

@hmhealey we are using any type for post.props is it because of some known issue, or simply due to the effort required to write down the keys?

hey @rmonga71, can you approve to unblock merging?

E2E tests not automatically triggered, because the PR is not in a mergeable state. Please update the branch with the base branch and resolve outstanding conflicts.

@jgilliam17 the server URL is on the related server PR ;)

post.props seems fine for this. I think it's similar to a lot of other features where it might be nice to add a separate column for it, but the Posts table is the absolute worst one to add a column too.
I didn't mean to make it sound like using post props was inherently dangerous, but I think it's something to be careful when using since it's poorly documented and not validated super well. In this case, we're presumably just checking something like post.props.priority === 'important' which will never cause an error even if priority is an invalid type or undefined

@hmhealey we are using any type for post.props is it because of some known issue, or simply due to the effort required to write down the keys?

I think it was just ignored during the original TS migration, and we never went back and updated the types. It's also not typed on the server either, so we don't really have a reference for those types. https://mattermost.atlassian.net/browse/MM-46598 and https://mattermost.atlassian.net/browse/MM-46693 are some tickets I filed in the last couple weeks for improving its type safety on the server and client respectively.

Thanks @koox00
few questions and issues

Should global threads display post priority? Can be a follow up ticket
Maybe top threads in Insights too if we add to global threads. 0/5 just exploring around and thinking of the places we coud show it

https://mattermost.com/pl/message-priority/ link in system console is broken. Maybe not yet implemented.

Can we make the label and menu accessible? Users should be able to tab through the post and to have the priority label read. This can be done in a follow up ticket

Hovering over RHS post with priority label causes user name to truncate, top of the post to shift

https://user-images.githubusercontent.com/52937121/188032294-7dea160d-c1e0-428b-852e-2cc6f6d8043d.mov

Dark theme standard message priority icon is not visible

Also x in dark themes is not visible, hard to discover

This error on Safari (latest), macOS Monterey
I posted as a different user, and this guest user was just observing the posts with priority label. I am not sure which part of the post causes it.

I missed this yesterday
8. alignment of username and time stamp on permalink posts, with/without priority label

/e2e-test MM_ENV="MM_FEATUREFLAGS_POSTPRIORITY=true","MM_SERVICESETTINGS_POSTPRIORITY=true"

Triggering e2e testing with options:
&map[MM_ENV:MM_FEATUREFLAGS_POSTPRIORITY=true","MM_SERVICESETTINGS_POSTPRIORITY=true]

Successfully triggered e2e testing!
https://git.internal.mattermost.com/qa/cypress-ui-automation/-/pipelines/237759

Thank you @jgilliam17!

Maybe top threads in Insights too if we add to global threads. 0/5 just exploring around and thinking of the places we coud show it

Yes, it makes sense to me for insights to know about priority messages.
cc @esethna

Dark theme standard message priority icon is not visible

cc @matthewbirtch

Noticed that Important icon is using sidebar text active border? Just wanted to confirm if this is as designed.

I'm changing the colors of the post priority picker menu items to match the priority labels, cc @matthewbirtch

I'm changing the colors of the post priority picker menu items to match the priority labels, cc @matthewbirtch

@koox00 based on the chat we had about the semantic colors right?

Dark theme standard message priority icon is not visible

@koox00 this icon should be using 56% Center Channel Color

@koox00
Latest e2e report: https://mattermost-cypress-report.s3.amazonaws.com/238461-af68778-onprem-ent-server-pr-20875-mm-ee-test:af68778/mochawesome.html
One test failure is related to PR changes: MM-T1458

@koox00 can we also add Cypress tests for post priority?
I created 3 test cases that can be used as test IDs MM-T5139 MM-T5140 and MM-T5142

Thanks @koox00
The issues I have raised above have all been addressed.

Tested, looks good to merge - I'm leaving it up to you to merge server and webapp PRs in the order you see fit
Verified message priority for posts. See more info in playbook used QA Post Priority review
Created a follow up for e2es MM-46930, please fix the failure related to PR change there too: existing test case  MM-T1458
Confirmed that link to documents page is in development and will be available for v7.4 (link location: system console setting help text)


Allow origins to do TLS renegotiation

Ran into an issue where IIS will do a TLS renegotiation after the first request is made.  Since the handshake has already been completed it is failing this check.
CI has a test for TS_SSL_VERIFY_SERVER_HOOK and I will see if this change breaks that functionality.
Discussion about IIS: https://security.stackexchange.com/questions/24554/should-i-use-ssl-tls-renegotiation

This commit has the logic change: https://github.com/apache/trafficserver/pull/10385/commits/ede1f34e6260551726bc9e466c886278d305980c the rest of the PR is just cleanup.

Logic seems reasonable.  Do we want to make this configurable?  I dimly recall that renegotiation is frowned up these days.  If it is configurable, I assume that is something we can set during the TLS negotiation to indicate whether the client will handle renegotiations.
Oh I see the 10 year old article you reference does discuss some of the renegotiation concerns.


could I load the GPTQ-for-SantaCoder/starcoder-GPTQ-8bit-128g model?

could I load the GPTQ-for-SantaCoder/starcoder-GPTQ-8bit-128g model offline?
when i use the command "python main.py --pretrained {the path to starcoder-GPTQ-8bit-128g}, there is a error:
File ... transformers/pipelines/init.py", line 779, in pipeline
framework, model = infer_framework_load_model(
File ...transformers/pipelines/base.py , line 271, in infer_framework_load_model
raise ValueError(f"Could not load model {model} with any of the following classes: {class tuple}.")
ValueError: Could not load model starcoder-GPTQ-8bit-128g with any of the following classes: (<class 'transformers.models.auto.modeling_auto.AutoModelForCausalLM'>, <class 'transformers.models.gpt2.modeling_gpt2.GPT2LMHeadModel'>).

I couldn't find any configuration file under mayank31398/starcoder-GPTQ-8bit-128g, I thought this model couldn't be loaded directly through transformers.


dependencyDependency version upgrade

Proposed changes

Keep hadoop-aliyun version consistent with hadoop main version (3.3.5)
upgrade jackson to 2.14.3
upgrade netty version to 4.1.94.final
binding check.freamework version to 3.32.0
upgrade snappy-java to <IP_ADDRESS>
upgrade hudi version to 0.13.1
upgrade spring version to 2.7.13
upgrade orc version to 1.8.4
revert nonsensical changes

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

run clickbench


⚠ Mitsubishi has degraded performance

In 38aa07e, Mitsubishi (https://mitsubishimotorsid-mobileapp.com/app/login) experienced degraded performance:

HTTP code: 200
Response time: 11693 ms

Resolved: Mitsubishi performance has improved in b657ab7 after 7 minutes.


Fixes #64 preload was being effected by the loader

@kevinlang I'm not certain, but the integration tests pass which leads me to believe it works. I find it odd though that the query isn't expanded with an inner join.

Added the test for it. Better safe than sorry.

Nice :)
I think this will unblock some other tests in repo.exs too.

Yea


Seems to be we have issue with memory model

This code has assumption about how data sitting on the stack.
int main(int argc, char *argv[])
{
    int x = 42;
    int a[10];
    return a[-1];
}

Cesium compiles, but returns parts of internal .NET structure instead of 42

Technically, this is UB(and I don't think we should even test this behavior), but we should investigate what exactly happens here, anyway.

That's what ILSpy shows
public unsafe static int main(int argc, byte** argv)
{
	int num = 42;
	int* ptr = stackalloc int[10];
	return ptr[-1];
}

For me, this C# program returned a "right" value of 42:
    int x = 42;
    int* p = stackalloc int[10];
    Console.WriteLine(x);
    Console.WriteLine(p[-8]);

A good idea is to check with C#; I didn't thought about that from the beginning for some reason :)

Or even better:
    int x = 42;
    int* p = stackalloc int[10];
    Console.WriteLine("x address: " + (long)&x);
    Console.WriteLine("p address: " + (long)p);
    Console.WriteLine("diff: " + ((long)p - (long)&x));

x address:<PHONE_NUMBER>76
p address:<PHONE_NUMBER>20
diff: -156

-156 is very far one from another.

So, I don't believe that relying on the stack layout is a good idea, and UB turns out to be really undefined.
You may proceed with the experiments, but I'm not sure this is a valuable direction of work.


logic seems flawed if the intent is to simulate server session

I thought ServerSession was to simulate a session object as in PHP, Pyramid or other web servers. This is not at all what it does...
An 'old fashioned' session object persists user specific session data between HTTP calls, ie each user session has its own data, persisted between calls.
Here the Collection holds values set on Server side, but these are shared across all users:
If i set a ServerSession value on the server for UserA, the collection published as such, is also visible to UserB.
Worse, if User A's action triggers change of the value in the ServerSession, then User B sees the changed value as it is published to the client.
How is this any different than using a collection in general?
A ServerSession object store should first

stay on serverside (i.e. not publish to the client)
persist user specific data
prevent sharing data between users

To do so with Mongo Collections one would need to index the object with the userId as key (for SET), and refer to it with this userId (for GET)
Or maybe just use the Meteor.users Collection for this purpose.
Did I just misinterpret the functionality of this package? If I did, I think the name is somewhat misleading.

Yes, I think it's important to update the README.md to show what this package was or is supposed to do. see #6

see #6


[Feature] Support Gitlab SAST report-type

We've recently started using audit-ci in our pipelines. For the tool semgrep that we use, they can output a Gitlab SAST compatible reporting format that GitLab can understand and integrate into its UI. Would be really cool to have similar reporting supporting with audit-ci.
I'm more than willing to write this reporter support into the repository if people feel like this could be valuable.
Gitlab SAST reporting schema: https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/master/dist/sast-report-format.json

I don't know if it can help but for that we are using a separate npm module @elpete/gitlab-npm-audit-parser after the audit.json report is generated.
Maybe it's possible to integrate the parser module directly to the audit-ci module?

Hi @doricci, I have two ideas for this:

I have a WIP PR #297 that attempts to improve the programmatic usage of audit-ci vastly. That way, you can use it as a library and pipe the result to any parser that you want.
With the improvement of its programmatic usage of audit-ci, we could add it as a first-party output format.


Notification manager not cleared after game end

We will probably change the way this stuff works anyway, but I will fix temporarily


Add Cevelop C++ IDE v1.4.0

Cevelop extends Eclipse CDT with many additional features: CUTE unit
testing with Test Driven Development support, new refactorings and
quick fixes, and much more. And it's free!
It is developed by the Institute for Software at the University of Applied
Sciences in Rapperswil, Switzerland (HSR) within the context of the
REPARA project, co-funded by the European Commission within the
Seventh Framework Programme

Thank you for the contribution. It needed some fixes, so they were made in 2a865a1c84b6ef7bf1b810b79170524fb8a8ae20. Your contribution is still included (and still credited to you), with the appropriate modifications. Please feel free to ask about any of the changes.

Thanks for your help! I was impressed with the workflow for adding new casks. The docs and tools were extremely helpful :)
I see that you added the archive-timestamp to the version number. For me this makes sense. Now, to add future releases we only have to adapt the version variable and the sha256 hash. The URL should automatically be correct, provided we keep our folder hierarchy.
Also, the relationship between the version number and corresponding archive is clearly visible.
I hope my assumptions are correct :)
Cheers

Thanks for your help! I was impressed with the workflow for adding new casks. The docs and tools were extremely helpful :)

Thank you, great to know.

I see that you added the archive-timestamp to the version number. For me this makes sense. Now, to add future releases we only have to adapt the version variable and the sha256 hash.

Precisely, that’s even why we visually separate that section from the rest of the cask, with a blank line.

The URL should automatically be correct, provided we keep our folder hierarchy.

Yep, and that’s an uncommon change.

Also, the relationship between the version number and corresponding archive is clearly visible.
I hope my assumptions are correct :)

They are. Everything is right on the money.


doggo: init at 0.5.4

Description of changes
Added a new package for doggo.

doggo is a modern command-line DNS client (like dig) written in Golang. It outputs information in a neat concise manner and supports protocols like DoH, DoT, DoQ, and DNSCrypt as well.
https://github.com/mr-karan/doggo

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[x] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 186894 run on x86_64-linux 1

1 package built:

doggo

Thanks, I implemented your suggestions. Is it possible to back-port this package to NixOS 22.05?

Is there something more needs to be done, or do we just have to wait?

@DeeUnderscore I added your suggestions.

@SuperSandro2000 I added your suggestions.

Is there  something else to do?

Result of nixpkgs-review pr 186894 run on x86_64-linux 1

1 package built:

doggo

@SuperSandro2000 , is this sufficient?


Functional tests are not working

Running ./ddl.sh exec ./docroot/vendor/bin/phpunit -c ./docroot/core ./docroot/core/tests/Drupal/KernelTests/Core/Form/FormCacheTest.php works fine.
But for example running ./ddl.sh exec ./docroot/vendor/bin/phpunit -c ./docroot/core ./docroot/core/modules/media/tests/src/Functional/MediaAccessTest.php
throws issues like:

Drupal\Tests\media\Functional\MediaAccessTest::testMediaAccess
Behat\Mink\Exception\ElementNotFoundException: Button with id|name|label|value "Log in" not found.

/var/www/html/docroot/core/tests/Drupal/Tests/WebAssert.php:76
/var/www/html/docroot/core/tests/Drupal/Tests/BrowserTestBase.php:816
/var/www/html/docroot/core/tests/Drupal/Tests/BrowserTestBase.php:753
/var/www/html/docroot/core/modules/media/tests/src/Functional/MediaFunctionalTestTrait.php:69
/var/www/html/docroot/core/modules/media/tests/src/Functional/MediaUiFunctionalTest.php:28

@chrfritsch This test passes for me with Drupal core 8.5.x, if you're running on an old/existing drupal-docker-lite checkout, could you run git pull && ./ddl.sh rebuild then try the test again?
If it's still failing, please let me know.

Damn it, i tried it on my working machine and there it works.
I think there must be something wrong on my private machine. But i have no freaking idea what....


add Nash averaging

Hello there, I have implemented a version of nash averaging using cvxopt. Please let me know if any issues. Thanks.

Cool!
To enable the python test, you need to add it to the list here: https://github.com/deepmind/open_spiel/blob/2acbd485af75fc92a4815fefd610c37292537127/open_spiel/python/CMakeLists.txt#L281
Can you make that change?

The egt subdirectory is mainly for evolutionary game-theoretic methods or algorithms .. is that where you intended to put this? (Wondering if it make more sense to out it under python/algorithms...)

@lanctot Thanks! Please take a look at the changes.
Yes I initially intended to put it in egt/ because I thought it is close to what alpharank tried to do. But on second thought nash averaging does not have evolutionary theoretical ground so yes maybe putting it in algorithm/ makes more sense.

@lukemarris Thanks for the comments! Please take a look at the revision.

Hi @lukemarris thanks for the comments! Please take a look at the current version.


rpc: Do not require a privkey corresponding to ownerAddress in protx register_* to be known by the wallet

This requirement was introduced in the initial implementation when we were signing protx payload with the owner key (#2246). This was changed later when we implemented external collateral references (#2366). The owner key is not used for anything in ProTxReg but to get CKeyID since then which we can do by simply decoding an address instead. This simplifies masternode registration process by letting an Operator to issue protx register_prepare on its own instead of asking an Owner. An Owner still have to sign the message provided by an Operator with his collateral address to prove collateral ownership (and authorize masternode registration).
Note: protx register_* rpc will no longer accept privkeys for ownerAddress param. Some 3-rd party software like DMT might need to be patched to work correctly with nodes running with this fix.

General concept ACK, but is it good to introduce that breaking behavior of not allowing privatekeys to be used?

I think it's fine for a major version to introduce breaking changes and discouraging the usage of plaintext privkeys when it's not really needed is the right thing to do too.

I think it's fine for a major version to introduce breaking changes and discouraging the usage of plaintext privkeys when it's not really needed is the right thing to do too.

@bertrand256 would this be breaking for you?
Generally, I lean towards breaking rpc changes should have 1 major version of gap / deprecation to allow people to migrate without problems

@PastaPastaPasta Yes, it will require adjustments in DMT code, but this is not a big problem.
Imho, the change is ok - I'm also inclined to eliminate private key transfers whenever possible.

it'd be nice if there was a test for this


🚚 Centralize progressbar and convert to percentage

Can't say i like this approach, we don't need the progress bar on every download or upload, but it is forced now.

Like we now have progress bar on every sqlite sync

Can't say i like this approach, we don't need the progress bar on every download or upload, but it is forced now.

Oh, apologies, I thought it's a minor refactor towards more code ergnomics, better user feedback and more aesthetics (percentage, Valentine was confused by the absence of a unit)

Like we now have progress bar on every sqlite sync
Yes, I'm aware. I tested and I felt it made the waiting time more pleasant because I saw what was going on

Also Jeremie used the download_to and upload_to and I'm sure he would have wanted clearer logging (I speced out some code for him and missed it myself)
If you'd like to hide it again, happy to hide in paces. But the SQLite synching I'd keep it - we can also ask the team for this

Ok, i am home and can expand a bit on the problems i see with this PR.
2 minor technical issues:

No way to disable the progress bar now.
The callback kwarg is completely blocked for upload_to and download_to.

But there is also a more important objection.
These functions were designed to be a thin utility layer on top of upath with pass-through arguments and no side effects. And i really expect them to be so when i work with storage. Now they have some very specific logic with fsspec callback inside. And it is not about purity, we will certainly just forget to change the things like this when we need to change something zarr related, for example (who would guess that you need to go the lowest level of storage code we have for this?). If you remeber we had a discussion of mixing levels, this is still valid here.

@falexwolf we can discuss keeping the progress bar on sqlite sync if needed. But we should check, i expect a lot of progress bar appearing out of nowhere now.

If you are fine with it, i will move the logic related to the progress bars back to lamindb, but keep the classes related to progress bars here. I can also add the progress bar explicitly to the sqlite sync.

Ok, i am home and can expand a bit on the problems i see with this PR. 2 minor technical issues, easily fixable (will fix):
[...]

All your points taken, and I don't have a strong opinion on them. It's perfectly fine with me to keep refactoring and fixing these minor issues.

If you remember we had a discussion about mixing levels, this is still valid here.

Given that we didn't add new methods to UPath, unlike in our discussion, I thought this would be OK.

we can discuss keeping the progress bar on sqlite sync if needed.

Yes, I think this is very helpful. My own feeling and the feedback from Valentine and others were that there is not enough responsiveness around waiting times.

i expect a lot of progress bars appearing out of nowhere now.

This is one of the three things I wanted to achieve by refactoring the code. Whenever an explicit download or upload occurs, I thought we'd like some responsiveness. I agree that for very small files this is nonsense, but the sqlite file already seems to take some time.

If you are fine with it, i will move the logic related to the progress bars back to lamindb, but keep the classes related to progress bars here. I can also add the progress bar explicitly to the sqlite sync.

I'm in principle fine with it, just regret the amount of time I invested and that you're going to invest. I felt the code is now better in the sense that I was able to remove several manual ad hoc definitions related to response, and fill in gaps that were there. So, simply reverting it isn't great either, I guess.
One could also add a flag print_progress to mute the progress bar in download_to and upload_to, and default it to False` and then take care to switch it on in sufficiently many places.

One could also add a flag print_progress to mute the progress bar in download_to and upload_to, and default it to False` and then take care to switch it on in sufficiently many places.
Ok, lets do like this.


Only show specific countries

I needed to filter the country list to show only those who are in a supplied list of ISO country codes

Wow. This looks amazing. I will check it out and merge this as soon as possible! Thank you, @kh0r.

Hi, I've added these changes myself to master. So I'm closing this pull request.


[RegExp] Exceptions for RegExp in Annex B of the ES2015 spec shouldn't be applied when the "unicode" flag is present

Annex B.1.4 contains exceptions that allow web browsers to diverge from the normal RegExp grammar. These exceptions aren't allowed when the "unicode" flag is present, but our current implementation doesn't take this into account.
There are many exceptions, but a few examples would be:

PatternCharacter: {
CharacterClassEscape: [\d-a]
DecimalEscape: \3 when the number of capturing parentheses is less than 3
IdentityEscape: \h, \u
QuantifiableAssertion Quantifier: (?!.)+

The above should throw a SyntaxError when used as, e.g.,/{/u

Section in ES 2017 B.1.4 #sec-regular-expressions-patterns:
https://tc39.github.io/ecma262/#sec-regular-expressions-patterns


Adds a PodDisruptionBudget and topologySpreadConstraints to web

What was changed

added configurable podDisruptionBudget to web component
added configurable topologySpreadConstraints to web component

Why?

To enable the web component to have a PodDisruptionBudget created and topologySpreadContraints configured due to the requirement of these configurations being enforced by policy.
Checklist

Closes

How was this tested:

helm template .

Any docs updates needed?

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


I built the environment in docker mode, and the configuration file is not read。。。help me

I access Oracle, but when the mirror operation, found the configuration file is not read, I changed the application.env configuration file WHZ_ETL_JOBS_DIR= "under the wherehows-backend package /var/tmp/jobs, but when I started the docker mirror found no read to my Oracle job configuration

Dear issue owner,
Thanks for your interest in WhereHows. We have recently announced DataHub which is the rebranding of WhereHows. LinkedIn improved the architecture of WhereHows and rebranded WhereHows into DataHub and replaced its metadata infrastructure in this direction. DataHub is a more advanced and improved metadata management product compared to WhereHows.
Unfortunately, we have to stop supporting WhereHows to better focus on DataHub and offer more help to DataHub users. Therefore, we will drop all issues related to WhereHows and will not accept any contribution for it. Active development for DataHub has already started on datahub branch and will continue to live in there until it's finally merged to master and project is renamed to DataHub.
Please check the datahub branch to get familar with DataHub.
Best,
DataHub team


Add resource references

Description of changes:
Adds every possible resource reference to each of the resources in the group
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

/approve

/lgtm


Clarity intended use of create_credentials_file: false

TL;DR
Explain what the use-case is for create_credentials_file: false and/or provide a way to disable the warning about not having checked out a project while still creating a credentials file.
Detailed design
I have a CI job which purely runs gcloud commands. As a result I'm not doing a checkout of my project before using this action. In turn I'm getting the warning which tells me that my workspace is empty.
However in my use-case the workspace is expected to be empty and the warning is a distraction. It would be great if this use-case could be supported.
Additional information
No response

Hi @PeterJCLaw - this is documented in the README:

create_credentials_file: (Optional) If true, the action will securely generate a credentials file which can be used for authentication via gcloud and Google Cloud SDKs in other steps in the workflow. The default is true.

The credentials file is exported into $GITHUB_WORKSPACE, which makes it available to all future steps and filesystems (including Docker-based GitHub Actions). The file is automatically removed at the end of the job via a post action. In order to use exported credentials, you must add the actions/checkout step before calling auth. This is due to how GitHub Actions creates $GITHUB_WORKSPACE:
There have been a number of issues where users have not used the "actions/checkout" step before our action. Our action relies on the creation of the $GITHUB_WORKSPACE directory, which is created by actions/checkout. Worse, if a user puts "actions/checkout" after our action, it will delete all contents including the exported credential.
To verify that a checkout has occurred, we do a small check to see if there are any files in the directory. If there are no files, we emit a warning if there are no files, since there may be legitimate use cases for authenticating without checking out the repository.
The warning is just a warning. If you know what you're doing and it's working as intended, you can ignore this particular warning. If it's really bothersome, you can create a file (touch $GITHUB_WORKSPACE/.file) before running the auth action which will trick it into thinking there's a checkout. Alternatively, you can use google-github-actions/setup-gcloud directly and manually run gcloud auth login with a credentials file or service account key.

Thanks, yeah, I've put in place a workaround very much like what you describe. I appreciate the warning and the goal it has -- I can well imagine users getting caught out by that behaviour.
I was kinda hoping you might be open to adding an option which squelched the warning without a workaround file?
Separately, the thing I couldn't work out from the docs was what the expected utility of this action was in the case where a credentials file isn't created. Since doing that doesn't affect the rest of the steps in any way it feels like an expensive no-op. Is the goal just to be able to validate some credentials that might later be re-used?

Separately, the thing I couldn't work out from the docs was what the expected utility of this action was in the case where a credentials file isn't created. Since doing that doesn't affect the rest of the steps in any way it feels like an expensive no-op. Is the goal just to be able to validate some credentials that might later be re-used?

If you're creating an access token or OIDC token that you use as an output in future steps. For example, if you were building a Docker container and pushing to Artifact Registry, you can skip "create_credentials_file" and pass the access token directly as the password to the docker-login action.


Distributed builds.

This is moot! And may ever be complete out of the scope of the project.
I'm sure I already asked about it before, probably on Synth, and would not expect something like this to be implemented anytime soon ever if "accepted".
The ( corner ) user cases would be:

people who want/need to build/test ports against different OSes, architectures, etc. and use dedicated builders ( but not only ); and

when there are more than one person building/testing ports against the same builder. And everything between those.

The idea is to have "ravenadm" working in a Client+Agent fashion, with the Agent Endpoint being a daemon which receive the build request, queue them, and then build the ports. Similar to the 'poudriere daemon' feature ( which actually was never implemented ) but on steroids.
In the Client Endpoint it would be interesting if possible to store/configure several Agent Endpoints, and also to be able to group them. Also, I suppose a small IPC would be necessary to keep the Client Endpoint fed with the building information.
It means, if when building ports the user always need to build for some/several different architectures or OS versions, he/she could just dispatch a single command and leave, ever if it is a single builder used for different objectives ( because they would all be queued ).
This would probably also lead to the need to put ravenadm and ravenports in separated packages/ports, because the actual "Ravenports" would just need to be in one of the endpoints ( probably at the Agent Endpoint ).
I suppose this kind of arrangement would also facilitate Ravenports to eventually be integrated in CI tools, or ever become one ( somehow ).
[EDITED]

Actually I think there is something planned for the future to allow easier testing against multiple platforms. Ideally this could look like this: Somebody uploads a buildsheet. This is sent to multiple servers running several / all supported operating systems, the port built and the build results (success or failure) are reported back. No idea however when something like this will be reality. The tooling does not exist, the necessary dedicated build servers don't exist. So far it's pie in the sky. But I'm really looking forward to the little steps towards this. :) And the more people, organizations and ultimately corporations adopt Ravenports the more likely it will become.

I guess technically somebody could build the server and just issue commands to ravenadm program (across a network even) and just check the results later.  It doesn't have to be incorporated in ravenadm.


Migrated to null safety

Separated generation logic to source package intl_utils so this package could be used only as a wrapper which provides arb files from your own server

why wasn't it merged still?

@Nikzed sorry, Arbify is no longer maintained


fix: palette breaking visual changes

Nothing too exciting here just that https://github.com/artsy/palette/pull/1130 will introduce a breaking visual change. Since dropdowns are now fixed they are outside of their anchor's stacking context, which is also fixed with a z-index of 100.
This fixes that and introduces a z-index map into the picture for all of our fixed elements. Modal isn't included here since Palette controls this and sets some weird super high value. (We may wish to bring that in here in the future though).

(Will rebase once Palette is released)


No Build Option, FOB shows up as Huron Cargo Container, Cannot Deploy FOB even at far distances

Bug report

I've tried searching and looking that is related to my problems but I cannot see them

Basic Information

Mission version:
// Version of the KP Liberation framework
KP_liberation_version = [0, 96, "7a"];
Map used:
Chernarus 2035
Mods used:
custom modlist  / custom modpack (SABRE)

CBA_A3	     Steam	https://steamcommunity.com/sharedfiles/filedetails/?id=450814997
RHSAFRF	     Steam	https://steamcommunity.com/sharedfiles/filedetails/?id=843425103
RHSUSAF	     Steam	https://steamcommunity.com/sharedfiles/filedetails/?id=843577117
ace	             Steam	https://steamcommunity.com/sharedfiles/filedetails/?id=463939057
RHSGREF	     Steam	https://steamcommunity.com/sharedfiles/filedetails/?id=843593391
USAF Mod - Main	                  Steam	https://steamcommunity.com/sharedfiles/filedetails/?id=2397360831
FIR AWS(AirWeaponSystem)	 Steam	https://steamcommunity.com/sharedfiles/filedetails/?id=366425329
Hatchet Framework - Stable Version	Steam	https://steamcommunity.com/sharedfiles/filedetails/?id=2941986336
Task Force Arrowhead Radio (BETA!!!)	Steam	https://steamcommunity.com/sharedfiles/filedetails/?id=894678801
LAMBS_Danger.fsm	                                Steam	https://steamcommunity.com/sharedfiles/filedetails/?id=1858075458
USAF Mod - Fighters	                                Steam	https://steamcommunity.com/sharedfiles/filedetails/?id=2397371875
USAF Mod - Utility	                                Steam	https://steamcommunity.com/sharedfiles/filedetails/?id=2397376046
Hatchet H-60 pack - Stable Version	        Steam	https://steamcommunity.com/sharedfiles/filedetails/?id=1745501605
USS Nimitz	                                         Steam	https://steamcommunity.com/sharedfiles/filedetails/?id=643530417
CUP Terrains - Core	                                 Steam	https://steamcommunity.com/sharedfiles/filedetails/?id=583496184
3den Enhanced	                                         Steam	https://steamcommunity.com/sharedfiles/filedetails/?id=623475643
Project OPFOR           	Steam	https://steamcommunity.com/sharedfiles/filedetails/?id=735566597
RHSSAF	                        Steam	https://steamcommunity.com/sharedfiles/filedetails/?id=843632231
FA-18 Super Hornet	        Steam	https://steamcommunity.com/sharedfiles/filedetails/?id=743099837
Sabre Group Essential	Steam	https://steamcommunity.com/sharedfiles/filedetails/?id=2931162694
Sabre Group Tactical  	Steam	https://steamcommunity.com/sharedfiles/filedetails/?id=2931166760
Jbad	                                Steam	https://steamcommunity.com/sharedfiles/filedetails/?id=520618345
Chernarus 2035             	Steam	https://steamcommunity.com/sharedfiles/filedetails/?id=1113631358

https://drive.google.com/drive/folders/1cEeyJnqUYaqPdLVY6RgRui1oRIJe3d33?usp=share_link
(the html file)
Dedicated or local game:
Dedicated Server
tried at local host still the same issue.
Source of the mission file (URL):

https://drive.google.com/drive/folders/1cEeyJnqUYaqPdLVY6RgRui1oRIJe3d33?usp=share_link
Individual Things
Did you edit anything within the mission files?
If yes, what did you edit exactly?
Cannot Remember but something to do so with our modlist that will be compatible in the game.
Short Issue Description
Note.
Since our dedicated server got wiped, this mission file is from that wiped server
and I have downloaded the mission file before the server got wiped, once we got the new dedicated server
I just pasted the mission file in the server
(ingame problems)
Permission to build have been set
Even at far distances there is option to build/ scroll wheel
The FOB container shows up as an  Huron Cargo container, cannot deploy it
No  build in scroll option even as the admin or commander.
The AL marker is nowhere to be found
The problem has been going on for weeks.
Expected Behaviour

Can Deploy Fob
Can Build Storage Area
Steps to Reproduce the Issue

none since I think it's something with the files.
Screenshots, Serverlog or any other helpful Information

I have the same issue with a dedicated server and locally hosted


npm run deploy:prod fails

Seeing a not helpful bunch of errors when running npm run deploy:prod, see screenshot http://cl.ly/1u1J103o0Y1x
However, if I set env : process.env.NODE_ENV || 'production', in config/index and run npm run deploy then it builds fine :/
I have all the latest code from the starter kit 3.0.0-alpha.0
npm run lint shows no errors/warnings
npm-debug.txt

ok, my colleague said that when he ran 'npm run deploy:prod manually in SSH' he got:
ERROR in ./src/routes/Restaurant/components/ModifierModal/ModifierModal.js
Module build failed: TypeError: /home/forge/collectfood.in/react-redux/src/routes/Restaurant/components/ModifierModal/ModifierModal.js: Property object of JSXMemberExpression expected node to be of a type ["JSXMemberExpression","JSXIdentifier"] but instead got "MemberExpression"

[3:02]
ERROR in ./src/components/LoginModal/LoginModal.js
Module build failed: TypeError: /home/forge/collectfood.in/react-redux/src/components/LoginModal/LoginModal.js: Property object of JSXMemberExpression expected node to be of a type ["JSXMemberExpression","JSXIdentifier"] but instead got "MemberExpression"

Why can we not see these errors in development?

It's because of https://github.com/thejameskyle/babel-react-optimize, which is only run in production. At this point I'm leaning toward removing it since it can cause hard-to-debug issues if you components are written in an unsupported way. If you remove it from here: https://github.com/davezuko/react-redux-starter-kit/blob/master/build/webpack.config.js#L134 your app should compile just fine.

Ah I see. The errors were being caused by me using react-bootstrap like this:
import { Modal } from 'react-bootstrap'
...
<Modal>
<Modal.Header>

when it should be:
import { Modal, ModalHeader } from 'react-bootstrap'
...
<Modal>
<ModalHeader>

Yup, that'd be it. If you can do the latter and keep the optimization plugin I recommend doing that, but if you want to save yourself any future headaches it's not the end of the world to drop it. It's a bit strange that you didn't see the babel errors during your initial report, though.

I was having the same problem (latest starterkit).. the deploy:prod was failing.. every other compile command worked..
Removing react-optimize is a solution; though the optimization should not be ignored. So why is it failing...
I worked out that, in my case, it was due to; Hopefully this helps someone; Older versions of the sk (starter-kit) did not use react-optimize so my code was opinionated on previous sk: A React.Component exported as a redux connected component. This seems to be a breaking anti-pattern when enforcing react-optimize.
(what whaaat?) ;
Fail
import React from 'react';
import { connect } from 'react-redux';

// longhand
export class View extends React.Component {
  render () {
    return
FAIL on deploy:prod with react-optimize
;
  }
}

// export default redux connected component
export default connect()(View);

Works
is a separation of concerns (sic); redux-container/react-view
View.js;
import React from 'react';

// longhand
export default class View extends React.Component {
  render () {
    return
aight on deploy:prod
;
  }
}

Container.js
import View from './View';
import { connect } from 'react-redux';

export default connect()(View);

This works as the React.Component is the exported class, not a connected/wrapped class.. It seems to be down to react-optimize recognizing a React.Component. Though I could be wrong; atm I have no time to look into react-optimize any further ;-)
Possibly the sk needs to mention that this separation is now explicitly required? (or did  I miss a beat?)

This is not an issue with the starter kit, but I wanted to add how I was able to fix another issue related to 'react-optimize'.
Some react-bootstrap components (ie: Navbar.Form) are not exported how @FutureKode was able to address this issue, since there is no NavbarForm export equivalent.  Not sure this is the best way, but I was able to keep production 'react-optimize' by directly importing (alias is optional):
import {Form as NavbarForm} from 'react-bootstrap/lib/Navbar'
render() { <NavbarForm pullRight>{}</NavbarForm> }
https://github.com/react-bootstrap/react-bootstrap/blob/master/src/Navbar.js
https://github.com/thejameskyle/babel-react-optimize/issues/19


[bug] Can't open web3modal with an active WalletConnect session

Link to minimal reproducible example
https://github.com/csmartinsfct/wallet-connect-no-modal
Summary
Steps to reproduce:

launch app
connect to Uniswap by inputing the wallet connect URI, thus creating a session.
click the "open modal" button at the top

Modal does not open.

Clear app's storage.
Refresh app.
click the "open modal" button at the top

Modal opens.
Here is a GIF showcasing the issue:
I open the app with a clean local storage, connect to Uniswap, try to open the modal and it doesn't open. Clear the storage, refresh, try to open the modal and it opens.

List of related npm package versions
"@walletconnect/utils": "2.9.1",
"@walletconnect/web3wallet": "1.8.7",
"@web3modal/ethereum": "^2.7.1",
"@web3modal/react": "^2.7.1",
"ethers": "5.6.6",
"wagmi": "^1.3.9"
In another app where I have this integrated I see this error on startup if I have any active sessions:
TypeError: Cannot convert undefined or null to object
    at Function.getPrototypeOf (<anonymous>)
    at H (https://local-app.meld.com:3000/static/js/bundle.js:180382:17)
    at Sa (https://local-app.meld.com:3000/static/js/vendors-node_modules_walletconnect_ethereum-provider_dist_index_es_js.chunk.js:8687:75)
    at Yg (https://local-app.meld.com:3000/static/js/vendors-node_modules_walletconnect_ethereum-provider_dist_index_es_js.chunk.js:8680:13)
    at https://local-app.meld.com:3000/static/js/vendors-node_modules_walletconnect_ethereum-provider_dist_index_es_js.chunk.js:9287:13
    at Array.forEach (<anonymous>)
    at hr.createProviders (https://local-app.meld.com:3000/static/js/vendors-node_modules_walletconnect_ethereum-provider_dist_index_es_js.chunk.js:9283:114)
    at hr.checkStorage (https://local-app.meld.com:3000/static/js/vendors-node_modules_walletconnect_ethereum-provider_dist_index_es_js.chunk.js:9263:81)
    at async hr.initialize (https://local-app.meld.com:3000/static/js/vendors-node_modules_walletconnect_ethereum-provider_dist_index_es_js.chunk.js:9267:66)
    at async hr.init (https://local-app.meld.com:3000/static/js/vendors-node_modules_walletconnect_ethereum-provider_dist_index_es_js.chunk.js:9167:12)
    at async v.initialize (https://local-app.meld.com:3000/static/js/vendors-node_modules_walletconnect_ethereum-provider_dist_index_es_js.chunk.js:264:145)
    at async v.init (https://local-app.meld.com:3000/static/js/vendors-node_modules_walletconnect_ethereum-provider_dist_index_es_js.chunk.js:100:12)
    at async WalletConnectConnector.initProvider_fn (https://local-app.meld.com:3000/static/js/bundle.js:510353:87)```

What are you trying to achieve? Atm it looks like you are adding modal (dapp sdk) into a wallet, which will likely not work / have storage conflicts. Also, wagmi > 1 no longer compatible with ethers, they use their own library now https://viem.sh/

Just a small update on this, we investigated and this will need a bit of core work done on our end (clients in walletconnect-monorepo) to decouple storages for different sdks i.e. ones you use as a wallet and as a dapp with web3modal, will let you know when this is available.

Hey there, seems like it's been months after this issue and we have v3 of web3modal, which is our main focus and actively iterating. Soon, we're planning to gradually drop the support for the v1 and v2.
@csmartinsfct, if you still building your app, we'd encourage you to upgrade to v3 by following the steps in our documentation, and let us know if you encounter any issues. We're ready to help anytime.

https://docs.walletconnect.com/web3modal/about


1422 pass sage uri in site info message

Pull Request Overview

Sage URI now in site info message

REST API Updates
[GET] /api/v1/site-settings/site-info
{
        'houston': {
            'version': version number,
            'git_version': git version number,
        },
        'sage': {
            'version': '3.5.1.dev23',
            'uri': 'http://sage:5000/',
        },
    }

I think that the permissions on this are too open. Sage URIs are only expected to be visible to staff level users.

This is not secure information so there is no security implication in it being visible in the message.
As the FE needs to display different information for staff users, it makes sense in terms of simplification if the FE is the place that does this staff user checking so that this information is only displayed if the user is a staff user.


[Events] IE targeting problem

Problem
setSelectValueClick() does not work correctly in IE.
Proposed solution
Use e.currentTarget instead of e.target

Can no longer reproduce, closing with res-invalid.


Speedup initial hostscan

Have you run this on the live network, and if so were you able to find what the longest successful Dial was? I'm curious how many hosts are close to the total timeout. Maybe we should do a test run where we log.Debugln  the timeouts for all the hosts.
I'm tempted to say that we should extend this PR to support only caring about the worst 95%, instead of caring about the worst 100%. That makes the math more complicated though.


[µTVM] Add serial transport, parameterize µTVM Zephyr test, run on physical HW

This PR adds a pyserial transport to µTVM, allowing it to be used with physical hardware attached to the machine. This splits #6703 in half and adds just the zephyr-specific runtime components. To test this change, you can either run it locally if you have the necessary dependencies or use the VM in #6703

cc @tmoreau89 @tqchen @u99127 @manupa-arm @leandron @tom-gall @liangfu @mshawcroft

Thanks @areusch @tom-gall


Seeing argument types for Deno APIs requires to go through 3 separate pages.

I want to see parameter types for Deno.chmod API.
To do that I go as follows:
https://doc-land.deno.dev/ -> click "Deno CLI APIs (stable) ->
https://doc-land.deno.dev/deno/stable -> select "Deno" namespace ->
https://doc-land.deno.dev/deno/stable/~/Deno -> select "chmod" function
This requires to go through 3 separate pages to see it.
I think I'd be nice if there was any indication about argument types on the page for "Deno" namespace.

I have been thinking about this.
The "problem" that I want to avoid is huge un-readable blocks of text.  For example, expanding 10 classes in a mod.ts with the JSDoc blocks for each method and property is not workable.  They really need a dedicated page to view.
That being said, I can agree the for functions and variables, having to navigate to too deep is just annoying.
So this is what I would suggest as a compromise:

we flatten namespaces, but still preserve the namespace pages.  So with builtins, you would have an entry for things like Deno.connect on the root page, but you could also navigate to https://doc.deno.land/deno/stable/~/Deno and just see the children of the Deno namespace.
we expand functions and variables of the doc "listing" page, so instead of just Deno.connect on the index page, it would be Deno.connect(options: ConnectOptions): Promise<Conn> on https://doc.deno.land/deno/stable/ without any other navigation, but the JSDoc @param docs wouldn't be expanded, you would need to navigate to the page to get the "full" function documentation, just like today.

So, effectively in the use case above, it would just be just navigating to /deno/stable, but for something like Deno.File, you would see the entry on the index page as a class and the class level JSDoc, but you would need 1 click to navigate to /deno/stable/~/Deno.File to see what properties it had.


Three Issues - Extra Scrollbar, Overlapping Content and Visible Content in Chrome/Opera

I have a sidebar on a site that I am developing. Typically this sidebar contains content related to the site. Occasionally, I need to add additional content to the sidebar and this is where flip comes in.
In stead of having one very long sidebar, I have a back and front sidebar. It was working okay (using v1 of Flip) but then I upgraded my site from the Foundation Framework v5.5.3 to v6.1.1 and switched to a Flexbox layout. This is when I started having issues. I also made other changes so the issues I ran into may not be related to Foundation.
There are three issues I see.

The "back" sidebar is long. Both sides of the sidebar show an extra scrollbar.

When I flip it to the back, which is the long sidebar, it doesn't expand the container to fill the sidebar. Instead it just overlaps the content in the footer.

The third issue I believe is related to Issue #53. When viewing the front, portions of the .back are showing through. I'm seeing icons (I am using Fort Awesome (previously Font Icons) which is by the same guy being Font Awesome)  but also seeing text. In addition to that, I have progress bars that are showing through (progress bars are part of Foundation Framework v6.1.1 and if you do not see progress bars, cast a vote in Question of the Week to get the progress bars to show).

I've test these three issues in the following browsers:

Firefox v44
IE v11
Edge v25 (aka v13)
Chrome v47
Opera v34
Safari v7 (my Safari isn't up-to-date on my mac)
Safari v9 (on iPad)

Number 1 happens on all browsers excluding browsers on a mac or touch devices where scrollbars are not typically seen.
Number 2 happens in all browsers.
Number 3 happens in Chrome and Opera but the other browsers are okay.
Here is a link to see a test page that shows the problem. To flip the sidebar click the "More" button at the top of it.
Note: I'm using v1.0.19 of the JS. I did try v1.0.15 per #53 but that didn't make a difference.

Anyone have any thoughts on this issue?

For your 3rd issue, can you try putting a white background on your aside element ?

No news for bug #3 ?

For issue #3, try using:
$(".back").css('z-index', '0');


_get_files doesn't return files in a deterministic order across OSes

_get_files in local.data.transforms.py  doesn't return files in a deterministic order across OSes.
This is an issue when getting files, then splitting using a fixed seed. For example, in 08_pets_tutorial.ipynb (I added the seed parameter):
items = get_image_files(source)
split_idx = RandomSplitter(seed=42)(items)

In this case, 2 users on different OSes would have the same split_idx, but different train/validation sets.
It would be straightforward for a user to correct this by sorting items before passing this list into the splitter, but I wouldn't expect that many people would know to do this.

I had an issue with matching a sorted CSV file with labels, to files from folder, only to find that get_image_files was returning in arbitrary non sorted order.
Also think that it's good practice to sort by default so it's deterministic (and then shuffle when needed).

It may be possible to introduce some sorting criteria, but I'm wondering whether sorting everything alphabetically could be a problem. Maybe they can be sorted based on a hash function on the filename?

I originally thought this is only an issue when using different OSes, but I now noticed that it returns a different order on 2 different Ubuntu systems.


Rumble motors

I apologize for creating another issue for this since just saw you closed one a few days ago about this.
What I want to ask if it will be posible to get an example of how to process the data for the rumble motors.
In a comment you wrote this

When the receiveXInput function is called it checks whether it is an led command or a rumble command. If it's a rumble byte 0 will be 0x00 and byte 1 will be 0x08. The values are separated into variables automatically.
Hopefully these explanations are helpful.
//If the data is a rumble command parse it
//Rumble Command
//8bytes for this command. In hex looks like this
//000800bbll000000
//the bb is speed of the motor with the big weight
//the ll is the speed of the motor with the small weight

In the documentations you mention that what receiveXinput() return is a 8 bit integer. Dont know if I am missing something, but dont understand where is the rumble command data stored.
Hope you can help me a little with this.

You overlooked the next paragraph.

if ((RXData[0] == 0x00) & (RXData[1] == 0x08))
{
//Process Rumble
rumbleValues[0] = RXData[3];	//Big weight
rumbleValues[1] = RXData[4]; //Small weight
return 1;
}

So rumble commands are received and placed in rumbleValues array with the big weighted motor in index 0 and the small weighted motor in index 1.


Sidebar not compatible

After setting up with the provided documents, and using GENERATE_TREEVIEW      = YES, the resulting page unexpectedly looks like the following.

With GENERATE_TREEVIEW      = NO, it appears as I would expect.

The generation did not use any additional external styles/formatting. It would be difficult to navigate this particular documentation without some sort of side bar.

bump

Currently the sidebar is not supported, I think I made of note of it in the readme. With the current fixed width layout it's not clear to me how a sidebar would work/layout with respect to the rest of the page. I'm open to suggestions, maybe supporting fluid and fixed width with a compile time flag?

I updated a patch that fixes most of the styles with
GENERATE_TREEVIEW = TRUE

But the tree itself it's still broken. Shouldn't be very hard to fix it from this point.


Refactoring in 2 method of AlbTextEditorLineSegmentPieceMap [WIP]

This optimizes a bit dragging the scroll bar.
See: #34
In my informal profile measurements, I saw a 2x speedup in selectionRectangles , still slow.
I enclosed this calculation in a [ ] timeToRun and log shows between 5ms and 12ms on each execution. Probably we can achieve 0ms by caching. Still understanding what's the motivation of this selectionRectangles. Solutions may be caching the result, or count with the inverted Dictionary that it now problematic (one where we search over the keys of it, not the values).
Maybe @plantec has an idea

After mocking selectionRectangles ^ #() I see more reasonable profile results...
10% layout
13% measuring text extent via Cairo
28% drawing with Cairo
Bah... at least FFI starts to be more relevant in the profiler result, I think that makes sense as for the moment we are redrawing each time. Anyway if we do a step back, we could avoid measuring text and redrawing if it really didn't change. On this could help https://github.com/pharo-graphics/Bloc/pull/268

@plantec I marked it "WIP" (work in progress) because I wanted to push more commits before merging... but finally didn't do more. Do you agree to merge it, isn't it? it 's a refactoring that speedsup.


Failed to distributedLoad data on dataset with many small files

Alluxio Version:
What version of Alluxio are you using?
alluxio version -r
2.3.1-SNAPSHOT-2c412267e0e749ced63262d53043cf753cc11927
https://github.com/Alluxio/alluxio/commit/2c412267e0e749ced63262d53043cf753cc11927
Describe the bug
A clear and concise description of what the bug is.
Alluxio cluster consists of 1 master node and 1 worker nodes. The under storage is local, and data size is 84.29GB.
alluxio fsadmin report capacity
Capacity information for all workers:
    Total Capacity: 200.00GB
        Tier: SSD  Size: 200.00GB
    Used Capacity: 36.50GB
        Tier: SSD  Size: 36.50GB
    Used Percentage: 0%
    Free Percentage: 100%

Worker Name      Last Heartbeat   Storage       SSD
<IP_ADDRESS>    0                capacity      200.00GB
                                  used          0GB (0%)

When I distributedLoad the data, it succeed and say:
time  alluxio fs distributedLoad  --replication 1 /
Successfully loaded path /insightface/images_2/329390.png after 1 attempts
Successfully loaded path /insightface/images_2/3199053.png after 1 attempts
Successfully loaded path /insightface/images_2/3579696.png after 1 attempts
Successfully loaded path /insightface/images_2/1401421.png after 1 attempts
Successfully loaded path /insightface/images_2/1245001.png after 1 attempts
Successfully loaded path /insightface/images_2/1625644.png after 1 attempts
Successfully loaded path /insightface/images_2/3788067.png after 1 attempts
Successfully loaded path /insightface/images_2/452706.png after 1 attempts

real    83m36.100s
user    21m31.902s
sys     13m36.277s

But When I checked the result:
bash-4.4# alluxio fsadmin report capacity
Capacity information for all workers:
    Total Capacity: 200.00GB
        Tier: SSD  Size: 200.00GB
    Used Capacity: 36.50GB
        Tier: SSD  Size: 36.50GB
    Used Percentage: 18%
    Free Percentage: 82%

Worker Name      Last Heartbeat   Storage       SSD
<IP_ADDRESS>    0                capacity      200.00GB
                                  used          36.50GB (18%)
bash-4.4# alluxio fs du -sh /
File Size     In Alluxio       Path
84.29GB       36.50GB (43%)    /

To Reproduce
Steps to reproduce the behavior (as minimally and precisely as possible)
Expected behavior
A clear and concise description of what you expected to happen.

The distributed time took too long
only no more than half data is loaded

Urgency
Describe the impact and urgency of the bug.
Additional context
Add any other context about the problem here.

diagnose_fluid_1605581604.tar.gz

@bradyoo  can you take a look ?

More context for @bradyoo : the previous related issue  (#12116) was fixed by increasing RPC timeout as explained inside the issue. After applying the fix, @cheyang was able to complete distributedLoad for bigger files (e.g., 1k files of 100+MB each). With the same fix, however, distributedLoad still cannot complete for his datasets with millions of smaller files. The original issue is closed and now let's iterate on this issue

Are you sure this is the correct set of logs?
For instance, all the logs in JobMaster logs are:
2020-11-16 16:49:30,098 INFO  TieredIdentityFactory - Initialized tiered identity TieredIdentity(node=<IP_ADDRESS>, rack=null)
2020-11-16 16:49:30,123 INFO  ExtensionFactoryRegistry - Loading core jars from /opt/alluxio-2.3.0-SNAPSHOT/lib
2020-11-16 16:49:30,161 INFO  ExtensionFactoryRegistry - Loading extension jars from /opt/alluxio-2.3.0-SNAPSHOT/extensions
2020-11-16 16:49:30,187 INFO  ProcessUtils - Starting Alluxio job master @ <IP_ADDRESS>:20004.
2020-11-16 16:49:30,189 INFO  UfsJournalCheckpointThread - JobMaster: Journal checkpoint thread started.
2020-11-16 16:49:30,191 INFO  UfsJournalCheckpointThread - JobMaster: Journal checkpointer shutdown has been initiated.
2020-11-16 16:49:35,201 INFO  UfsJournalCheckpointThread - JobMaster: Journal checkpoint thread has been shutdown. No new logs have been found during the quiet period.
2020-11-16 16:49:35,201 INFO  UfsJournalCheckpointThread - JobMaster: Journal shutdown complete
2020-11-16 16:49:35,214 INFO  UfsJournal - JobMaster: journal switched to primary mode. location: /journal/JobJournal/JobMaster/v1
2020-11-16 16:49:35,248 INFO  AbstractMaster - JobMaster: Starting primary master.
2020-11-16 16:49:35,250 INFO  AlluxioJobMasterProcess - Alluxio job master web server version 2.3.1-SNAPSHOT starting. webAddress=/<IP_ADDRESS>:20005
2020-11-16 16:49:35,257 INFO  log - Logging initialized @5818ms
2020-11-16 16:49:35,342 INFO  WebServer - Alluxio Job Manager Master Web service starting @ /<IP_ADDRESS>:20005
2020-11-16 16:49:35,344 INFO  Server - jetty-9.2.z-SNAPSHOT
2020-11-16 16:49:35,363 INFO  ContextHandler - Started o.e.j.s.ServletContextHandler@3943a2be{/metrics/json,null,AVAILABLE}
2020-11-16 16:49:35,364 WARN  SecurityHandler - ServletContext@o.e.j.s.ServletContextHandler@3cc1435c{/,null,STARTING} has uncovered http methods for path: /
2020-11-16 16:49:41,000 INFO  ContextHandler - Started o.e.j.s.ServletContextHandler@3cc1435c{/,null,AVAILABLE}
2020-11-16 16:49:41,006 INFO  ServerConnector - Started ServerConnector@64beebb7{HTTP/1.1}{<IP_ADDRESS>:20005}
2020-11-16 16:49:41,006 INFO  Server - Started @11566ms
2020-11-16 16:49:41,006 INFO  WebServer - Alluxio Job Manager Master Web service started @ /<IP_ADDRESS>:20005
2020-11-16 16:49:41,006 INFO  AlluxioJobMasterProcess - Alluxio job master version 2.3.1-SNAPSHOT started. bindAddress=/<IP_ADDRESS>:20004, connectAddress=<IP_ADDRESS>:20004, webAddress=/<IP_ADDRESS>:20005
2020-11-16 16:49:41,006 INFO  AlluxioJobMasterProcess - Starting Alluxio job master gRPC server on address /<IP_ADDRESS>:20004
2020-11-16 16:49:41,151 INFO  MasterProcess - registered service JOB_MASTER_WORKER_SERVICE
2020-11-16 16:49:41,151 INFO  MasterProcess - registered service JOB_MASTER_CLIENT_SERVICE
2020-11-16 16:49:41,254 INFO  AlluxioJobMasterProcess - Started Alluxio job master gRPC server on address <IP_ADDRESS>:20004
2020-11-16 16:49:56,928 INFO  JobMaster - registerWorker(): WorkerNetAddress: WorkerNetAddress{host=<IP_ADDRESS>, containerHost=, rpcPort=20006, dataPort=20008, webPort=20007, domainSocketPath=, tieredIdentity=TieredIdentity(node=<IP_ADDRESS>)} id:<PHONE_NUMBER>184

Perhaps there was a previous instance that got killed in some way?

@cheyang  could you help Bradley to double check ? thanks

Sorry for that.
Here is the new logs, please check:
diagnose_fluid_1605755296.tar.gz

Ah yes. This issue was discovered and fixed quite recently here: https://github.com/Alluxio/alluxio/pull/12468 which modifies a default of a setting of a configuration that causes this issue.
But for now, I recommend changing the configuration
alluxio.job.master.finished.job.retention.time
(See https://docs.alluxio.io/os/user/2.3/en/reference/Properties-List.html)
to a much lower value like 60 seconds or perhaps even less (like 30 seconds).
Alternatively (or with it), you can also increase "alluxio.job.master.job.capacity" to a higher value.

@cheyang  I updated branch-2.3-fuse by cherry-pick #12468 . Can you make another try?

Is there any scientific way to set this value? Is there a way to calculate the  alluxio.job.master.finished.job.retention.time based on the file number or the size of the file?  Or is it fine to set alluxio.job.master.finished.job.retention.time=1 or 0?

And can we test with the following way:
time  alluxio fs distributedLoad -Dalluxio.job.master.finished.job.retention.time=5  --replication 1 /

No. You can not set it in that way. The setting is a global setting for the job service.
This is the behavior of the JobMaster when the JobMaster is at capacity. Lowering this value so much has the potential behavior where the client that needs to check that the job is complete misses it because it was removed. But even then, 30 seconds is plenty high enough (compared to 5 minutes) to not run into this problem.

May I ask if there is any impact when I decrease the default value from 60 seconds to 30 seconds?

Thank you, I've verified. It works.


Restore Transactions event tracking for Google Analytics

What it solves
Resolves #2546
How this PR fixes it
Adds again tracking event when opening transactions view

So I see the event for Tx is showing now in the "Realtime" tab. Still I'll check tomorrow if I can filter it in the Behaviour tab

I can see Tx actions in rinkeby now in the behaviour > events tab
https://analytics.google.com/analytics/web/#/report/content-event-events/a180094170w216025621p206479581/_u.date00=20210712&_u.date01=20210726&_r.drilldown=analytics.eventCategory:Safe Navigation


1.0.1rc1 Python documentation

Because GitHub's review system still doesn't allow comments on non-diff-parts-of-files...
We should probably mention Beta deprecation here.


DC-640 Add instructions for updating App Store listing

Added a short section ("Updating an App Store listing") at the bottom of "Listing in the App Store".

Merging without review. Just a minor change anyway.


cmake -D build options?

This is really just a question.
Intel's OpenVINO installed a customized OpenCV 4.1.0 that supports their MYRIAD tensor co-processors.  But it seems they have introduced a issue where rtsp stream decoding can only achieve about half the frame rate as your Pypi 4.1.0 builds.  OpenVINO has recently been open sourced so its possible to recompile it.
My question is,  do you publish the cmake -D options you use to build?  I'd like to try to rebuild OpenVINO and make its options match yours in hopes of getting the good rtsp decode performance of your builds.

The build options are scattered around ```setup.py``: https://github.com/skvark/opencv-python/blob/master/setup.py
However, I think that OpenCV doesn't handle decoding. It's handled by FFmpeg. FFmpeg requires some assembler libraries (nasm and yasm) to work efficiently. I have used this guide to build FFmpeg: https://trac.ffmpeg.org/wiki/CompilationGuide/Centos
See the Dockerfiles for more info: https://github.com/skvark/opencv-python/blob/master/docker/Dockerfile_x86_64

Thanks, so if I understand correctly you are shipping a statically linked ffmpeg, same as with QT?
That should explain it.

FFmpeg libraries which are shipped with opencv-python are shared libraries (the only way to ship them to comply with LGPL) and they have been dynamically linked with the OpenCV binary.

I guess I mis-read something in the faq about QT.  Back to square one if the same ffmpeg libraries are being used -- I only have one version of ffmpeg installed as far as I can tell.
I've both contrib-opencv-python and the Intel OpenVINO openCV installed on the same system.  Default is opencv-python, but OpenVINO sets a PYTHONPATH to make its version be found first.
I'm seeing this decrement on both ARM and Intel cpus, although rtsp stream decoding is poor on ARM, its still showing about the same percentage performance decrements with Intel's built for ARM vs. the  pyWheels openCV version.
The Intel NCS v1 SDK appears to install the pypi verison of OpenCV and didn't have this issue,  I've discovered it moving my code to OpenVINO to allow using the faster NCS2.

These packages do not use system FFmpeg so it does not matter which FFmpeg you have installed.
auditwheel (https://github.com/pypa/auditwheel) fixes the wheel archives during build so that they become self-contained which means that they include all dependencies (except some system libraries such as glibc). Quotint the README:

auditwheel repair: copies these external shared libraries into the wheel itself, and automatically modifies the appropriate RPATH entries such that these libraries will be picked up at runtime. This accomplishes a similar result as if the libraries had been statically linked without requiring changes to the build system. Packagers are advised that bundling, like static linking, may implicate copyright concerns.

OpenCV binary itself is a very big static library (cv2.cpython-36m-x86_64-linux-gnu.so below) which contains many dependencies such as libjpeg etc.. Additionally, the OpenCV binary is linked with some external dependencies which must be redistributed as shared libraries due to LGPL license.
If you have a look at the installation folder you should see something like this inside the .libs folder under cv2:
~/.local/lib/python3.6/site-packages/cv2:

drwxrwxr-x 5 skvark skvark     4096 Jun 17 19:32 .
drwx------ 7 skvark skvark     4096 Jul  4 09:50 ..
-rwxrwxr-x 1 skvark skvark 28481592 Jul  4 09:49 cv2.cpython-36m-x86_64-linux-gnu.so
drwxrwxr-x 3 skvark skvark     4096 Jun 17 19:32 data
-rw-rw-r-- 1 skvark skvark      233 Jul  4 09:49 __init__.py
drwxrwxr-x 2 skvark skvark     4096 Jul  4 09:50 .libs
-rw-rw-r-- 1 skvark skvark    30910 Jul  4 09:49 LICENSE-3RD-PARTY.txt
-rw-rw-r-- 1 skvark skvark     1097 Jul  4 09:49 LICENSE.txt
drwxrwxr-x 2 skvark skvark     4096 Jun 17 19:32 __pycache__

~/.local/lib/python3.6/site-packages/cv2/.libs:

libavcodec-874f3d51.so.58.47.106   libQtCore-3dbacd8a.so.4.8.7  libswresample-a49c020a.so.3.4.100  libz-a147dcb0.so.1.2.3
libavformat-fb41c63f.so.58.26.101  libQtGui-6d0f14dd.so.4.8.7   libswscale-4e6f4703.so.5.4.100
libavutil-473e9eb1.so.56.26.100    libQtTest-1183da5d.so.4.8.7  libvpx-8459aeef.so.6.0.0

In short: OpenCV wheels installed from PyPI use always a custom FFmpeg version which is built specifically on the Cent OS 5 Docker container to achieve maximum portability across different distros. I don't know how the other OpenCV versions you have installed are packaged, but most likely they use the system FFmpeg unlike opencv-python wheels.

Thanks for this clarification,  I was stumbling around and just noticed the .libs directory and saw the various shared libraries that were being used with opencv-python. I often forget about looking for "hidden" files and directories :(
Achieves the same net effect as my incorrect assumption about static linking, albeit with a lot more complexity to keep everybody happy.
Thanks for the ldd tip. I'll poke around some more and see if I can figure something out.  I have found a work-around.   I can run my rtsp decode code as a separate Python3 program using the opencv-python version to get the good decode performance. and run my AI code in a different Python3 process using the OpenVINO PYTHONPATH for its openCV and the NCS "myriad" support.   This is actually more inline with my ultimate plans were the rtsp decoding is done on one machine and the AI runs on another.
Trying to run the OpenVINO code without the "myriad" functions is giving me errors like this:
`QObject::moveToThread: Current thread (0x1883c70) is not the object's thread (0x19a3f00).
Cannot move to target thread (0x1883c70)
Segmentation fault
`
Presumably because my python path is not finding consistent sets of libraries.  I believe virtual environments are my only solution here.  I'm not too familiar with them but I have used them.
Thanks for your help.


command 'cobolplugin.processAllCopybookFiles' not found

I'm getting this error notification when attempting to run this command via the Command Palette. Although my CPU spikes as if it is processing.

Its taking a long time to process (this is fairly large codebase), this has been running for 30 minutes now.

You will need to turn coboleditor.Cache_metadata to "off" or "partial", as unfortunately parsing lots of sources takes time.
If you turn it off, then you use other forms of source-to-source navigation.

Changed the default to only process files in the workspace when caching is "on"


Multiline column descriptions result in SQL errors

Describe the bug

When columns have a multiline description, the base_node_columns and base_source_columns models fail to build due to a SQL error.
Steps to reproduce

We have a model config that looks like so:
version: 2
models:
  - name: int_ga360_hits_custom_fields_stage
    columns:
      - name: article_completions
        description: >
          Indicates whether the user completed an article. If so, then this
          column will contain 1 (otherwise null).

When running dbt run --select base_node_columns, the command fails with the following error:
18:18:14    Database Error in model base_node_columns (models/staging/graph/base/base_node_columns.sql)
  Syntax error: Unclosed string literal at [1025:1]
  compiled Code at target/run/dbt_project_evaluator/models/staging/graph/base/base_node_columns.sql

The compiled BigQuery statement illustrates that a line break in the .yml file is being propagated as a newline in the corresponding SQL INSERT INTO statement, leading to a SQL error.

Expected results

dbt run --select base_node_columns succeeds when a column has a multiline description
Actual results

dbt run --select base_node_columns errors when a column has a multiline description
Screenshots and log output

System information
The contents of your packages.yml file:
packages:
  - package: dbt-labs/dbt_utils
    version: [">=1.1.0", "<1.2.0"]
  - package: calogica/dbt_expectations
    version: [">=0.10.0", "<0.11.0"]
  - package: dbt-labs/codegen
    version: [">=0.12.0", "<0.13.0"]
  - package: data-mie/dbt_profiler
    version: [">=0.8.0", "<0.9.0"]
  - package: dbt-labs/dbt_project_evaluator
    version: [">=0.10.0", "<0.11.0"]
  - package: brooklyn-data/dbt_artifacts
    version: [">=2.6.0", "<2.7.0"]
  - package: elementary-data/elementary
    version: [">=0.14.0", "<0.15.0"]
  - package: dbt-labs/audit_helper
    version: [">=0.9.0", "<1.0.0"]
  - package: grafana/leaner_query
    version: [">=0.1.12", "<0.2.0"]

Which database are you using dbt with?

[ ] postgres
[ ] redshift
[x] bigquery
[ ] snowflake
[ ] trino/starburst
[ ] other (specify: ____________)

The output of dbt --version:
18:26:25  Running with dbt=1.7.10
18:26:26  Registered adapter: bigquery=1.7.7

Additional context

base_source_columns also fails under the same conditions
Are you interested in contributing the fix?

Sure! I would need a hand

Feel free to close this as a duplicate of https://github.com/dbt-labs/dbt-project-evaluator/issues/421

Managing it in #421


Issue with auto playing Swipers with iOS 17.4.1 and menus

Check that this is really a bug

[x] I confirm

Reproduction link
https://38b9mtkc5cuq9t0y-55090610358.shopifypreview.com
Bug description
I have checked this error with multiple devices.
Everything was working fine, until I did the iOS update to 17.4.1. People that not have updated to 17.4.1 dont have any issues.
The Problem: On iOS devices there appears to be some sort of conflict, when you have a autoplaying swiper and close a menu. See in the video. The menu just keeps appearing. However when I disable the autoplaying everything works fine.
https://github.com/nolimits4web/swiper/assets/115485629/73ea803c-0a45-4dbe-9d24-a386cca1f25b
I am using version 8 in the example, but this also happens with v11
Expected Behavior
Expected behavior would be that the menu just closes normally and that also works without the autoplaying or lower iOS versions.
Actual Behavior
The menu keeps opening. When autoplaying is disabled everything works. Autoplaying can be enabled when you are on a version lower than iOS 17.4.1
Swiper version
8.0.0 and 11.0.0
Platform/Target and Browser Versions
iOS 17.4.1
Validations

[X] Follow our Code of Conduct
[X] Read the docs.
[X] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.
[X] Make sure this is a Swiper issue and not a framework-specific issue

Would you like to open a PR for this bug?

[ ] I'm willing to open a PR

Autoplaying Swiper buggy since new iOS release

It seems like this only affects 100vh menus and autoplaying swipers

The issue can be reproduced on for example: www.webflow.com

t0ggles-create swiper

It seems that Apple has changed something in the accessibility settings :/


Where paper or technical report ?

Excellent work !!!!

Thank you! we're working on tidying this up and will put on here/ArXiv shortly.

We've updated the repo and our paper is here. Apologies for the delay!


SQL Server restore - where to find progress page in portal

Cant find where to look for progress for ongoing SQL restore ?

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 7d57fe19-3671-884c-f6fd-4095b68ec039
Version Independent ID: 9a758d0b-ecdf-0cd3-2dfa-d2e07b2d7ac7
Content: Restore a database from a backup - Azure SQL Managed Instance
Content Source: azure-sql/managed-instance/recovery-using-backups.md
Service: sql-managed-instance
Sub-service: backup-restore
GitHub Login: @MilanMSFT
Microsoft Alias: mlazic

Hi @diepes ,
Thanks for sharing the feedback. We will look into this and update you.
Hi @MilanMSFT,
Could you please look into this document request. Thanks

Hi, @diepes , currently you can track the progress of the restore process by using the sys.dm_exec_requests  and sys.dm_operation_status dynamic management views.


igate and digi + wx ?

any way to make it a igate and digi + wx ?

any way to make it a igate and digi + wx ?

hi, igate and digi and wx  (with temperature, pressure and humidity) is already working
do you need more Wx data like wind direction , speed and more?

did you read the wiki on how to add BME280 as this is used and needed to send WX data?

Yes and bme280 works fine .. im asking for a mode where lora tx/rx (like beacon and digi) works and on the igate on the same time

soo i updated to newest version and now it works .. thanks 73


DuckDB extension

One potential integration that would be great — a DuckDB extension which can allow DuckDB to accept PRQL as well as SQL.
At initial version could be very thin — accept PRQL, compile to SQL, pass to DuckDB. It would have similar functionality to a simple wrapper.
I haven't yet read up on how to build it. There's a Rust framework: https://github.com/Mause/duckdb-extension-framework, and here's an example for Deltatable: https://github.com/Mause/duckdb-deltatable-extension

The rust framework is fairly basic at the moment; here's an example of its use: https://github.com/Mause/duckdb-deltatable-extension/blob/main/src/table_function.rs — every function is unsafe...
So this would probably be a fairly technical piece of work for someone who is familiar with C++ andor DuckDB...

Check out @ywelsch's https://github.com/ywelsch/duckdb-prql!!

For anyone wanting to use this with Docker, try out ducker!
https://github.com/duckerlabs/ducker

Closing as completed


GPU only using half of available memory

Hello,
Whether I use 1 or 2 GPUs (RTX 2080 Ti), only half of the capacity is used:
1 GPU

2 GPUs

I tried to increase the batch size (FRCNN.BATCH_PER_IM in train_stg1.sh) and also the number of workers (_C.DATA.NUM_WORKERS in third_party/FasterRCNN/FasterRCNN/config.py) but it doesn't seem to make a difference.
Since TensorFlow is supposed to use all available memory, is this being done on purpose?
Thank you for your time.

Hi
Are you using the correct image size? Do you change some other settings? The GPU usage should around 16GB and 12GB GPU shouldn't fit in. I think your setting has some issues. Pls closely follow the guidance without modification at first trial.

@zizhaozhang
I am not changing any settings except the number of GPUs.
About the image size, how can I check I am using the right one?

@zizhaozhang
There are actually a few differences in the setting because otherwise I couldn't get it to work.

Environment
When I used your method to install the environment I get the following error

ERROR: Could not find a version that satisfies the requirement tensorflow-gpu==1.14.0 (from -r requirements.txt (line 5)) (from versions: 2.2.0rc1, 2.2.0rc2, 2.2.0rc3, 2.2.0rc4, 2.2.0, 2.3.0rc0, 2.3.0rc1, 2.3.0rc2, 2.3.0)
ERROR: No matching distribution found for tensorflow-gpu==1.14.0 (from -r requirements.txt (line 5))

So I used conda instead.

Data processing

Apparently some files can't be processed (I tried on 2 different computer to make sure).
[26]   Done                    python3 prepare_coco_data.py --percent $percent --seed $seed
[27]   Done                    python3 prepare_coco_data.py --percent $percent --seed $seed
[28]   Done                    python3 prepare_coco_data.py --percent $percent --seed $seed
[29]   Done                    python3 prepare_coco_data.py --percent $percent --seed $seed
[30]   Done                    python3 prepare_coco_data.py --percent $percent --seed $seed
[31]   Done                    python3 prepare_coco_data.py --percent $percent --seed $seed
[32]   Done                    python3 prepare_coco_data.py --percent $percent --seed $seed
[33]   Done                    python3 prepare_coco_data.py --percent $percent --seed $seed
[34]   Done                    python3 prepare_coco_data.py --percent $percent --seed $seed
[35]   Done                    python3 prepare_coco_data.py --percent $percent --seed $seed
[36]   Done                    python3 prepare_coco_data.py --percent $percent --seed $seed
[37]   Done                    python3 prepare_coco_data.py --percent $percent --seed $seed
[38]   Done                    python3 prepare_coco_data.py --percent $percent --seed $seed
[39]   Killed                  python3 prepare_coco_data.py --percent $percent --seed $seed
[40]   Done                    python3 prepare_coco_data.py --percent $percent --seed $seed
[41]   Done                    python3 prepare_coco_data.py --percent $percent --seed $seed
[42]   Killed                  python3 prepare_coco_data.py --percent $percent --seed $seed
[43]   Done                    python3 prepare_coco_data.py --percent $percent --seed $seed
[44]   Done                    python3 prepare_coco_data.py --percent $percent --seed $seed
[45]   Done                    python3 prepare_coco_data.py --percent $percent --seed $seed
[46]   Done                    python3 prepare_coco_data.py --percent $percent --seed $seed
[47]   Killed                  python3 prepare_coco_data.py --percent $percent --seed $seed
[48]   Done                    python3 prepare_coco_data.py --percent $percent --seed $seed
[49]-  Killed                  python3 prepare_coco_data.py --percent $percent --seed $seed
[50]+  Done                    python3 prepare_coco_data.py --percent $percent --seed $seed

Download link is corrupted

I don't think it really matters for the first part of training but you can't use wget to download JSON files for unlabeled images of COCO data and PASCAL VOC data. You need to log in with a Google account. Also noticed that the file name is different in the download link and the processing link
wget https://storage.cloud.google.com/gresearch/ssl_detection/STAC_JSON.tar
tar -xf STAC_JSON.tar.**gz**

Hi
Thanks for explaining them and give many trails

I think It is your pip version issue. Pls search Google to find solution of this. We need to use tensorflow 1.14 becuase of the requirement of tensorpack.

This data processing pipeline has been validated by several people and did not find any errors. So I suggest you double check your actual executed commands.

Thanks for pointing this. We will double check what happened here. It seems like the issue of Cloud Storage. I found manual download does change the file name to ssl_detection_STAC_JSON.tar. Pls modify accordingly.

@zizhaozhang
To install tensorflow 1.14 with pip you need Python 3.6, it does not seem to work with Python 3.7=<
The command virtualenv -p python3 --system-site-packages env3 seems to assume that your system is using Python 3.6, which is not my case, so I needed to replace it with virtualenv --python=/usr/bin/python3.6 --system-site-packages env3.
Still no difference, GPUs are only use at half capacity.
If any one else is having the same problem please comment.

Oh I see. you are training stage 1. They it is correct to use this much GPU memory. Tensorflow can be configred to use only necessary GPU memory by setting https://www.tensorflow.org/guide/gpu#limiting_gpu_memory_growth

@zizhaozhang
Thank you for the link. There seem to be 3 methods. The first one is just limit the number of GPUs, which I don't think is what you are doing. Then, there is
tf.config.experimental.set_memory_growth(gpu, True)
but I couldn't find this line in your code.
The last one is
tf.config.experimental.VirtualDeviceConfiguration(memory_limit=1024)
which seems like what you are doing but I couldn't find this in your code either.
If you have the time, I'd really appreciate you indicating where I can make the relevant changes.
Thank you.

It is set here and configured at here
In tf1, these are usually comment
config = tf.ConfigProto()
config.gpu_options.allow_growth=True
sess = tf.Session(config=config)


info: table fields

Hi guys
I was wondering if fields represented in table (like the line items fields) are supported.
If Yes, how to set them up ?
If Not, that would really be a nice to have.

I'm not completely sure which fields you're referring to. Do you have a sample image you can display in this ticket?
In general however, it should be possible to train a model for any field in a document as long as it's not a field which can have multiple occurrences. You can train a model even for fields that have multiple occurrences, but you will only be able to use one of the occurrences as the true label and the final extraction of such a model would also only be able to extract a single occurrence for this field.

Thank you.
You answered me already. I meant multiple occurrences such as the purchased articles described in the invoice, or "line items". You normally find more than one therefore you have Item1, Item2, Item3, and so on..
They typically are represented with a similar vertical and horizontal alignment.
Any chance that this is going to be included in the future or any idea how to start to develop in this direction ?
Thanks

Hi @IzzyHibbert , you can try this API dedicated to invoices https://scandocflow.com

It seems InvoiceNet cant handle the tables for example.

How can we extract the items from the table as the criteria of using the custom field take only a single key-value pair?

It seems InvoiceNet cant handle the tables for example.

How can we extract the items from the table as the criteria of using the custom field take only a single key-value pair?

@mirfan899, @IzzyHibbert have you found a solution!?

@mirfan899 That's great. Can you provide a link to the repository?

https://github.com/ultralytics/yolov3

@mirfan899
Thank you.
I'm not sure how yolo will extract invoice data though.
Did you write your custom network?

I labeled the dateset. Here are the results.

using yolo and then train a yolo v3 model.

cant you at least use the OPTIONAL data type for small lists?

@mirfan899
Thanks for sharing. Could you use Yolo to extract the line item details from the table? For example, if you want to extract payment history lines from your last photo i.e. something like:-
{"Month": "Dec 2021", "HM3": 0.622, "Current Bill": 433.36, ...},
{"Month": "Nov 2021", "HM3": 0.387, ...}]

, is it possible? As far as I understood, neither InvoiceNet nor Yolo can do that.

Why not. Yolo can solve the table issue. Just label the table and after detection use ocr to extract text.

I guess he's aiming for extracting formatted line items with labels not just text.
Extracting text using ocr from the table will just give you some text.

Thank you both for your quick replies
Yes I wanted them to be formatted so that I know which text corresponds to which column. I will need to store these extracted data and process them depending on their columns.

I have done similar to this. You need to label columns with yolo. Detect and OCR.  You need more data to get better accuracy.
Around 50 samples of a single template.

Can you show a sample of how you labeled the columns with yolo to detect single line items? I'm also interested in this.

Can you provide a repository with sample code?