Why does a Cypher query involving relations require relation tables to exist?

When running a query such as this, that asserts that there are no relations of a certain kind connected to a given node, kuzu throws an exception if there are no relational tables at all.
Why should that be the case? We are working around that now by creating an empty, unrelated dummy relationship table, but this looks like a bug - if there are no relation tables, statements that involve relationships simply shouldn't match, rather than throwing. @whimo

This is even more general. I run this for an initially empty database:
CREATE NODE TABLE label1 (id SERIAL, PRIMARY KEY(id));
CREATE NODE TABLE label2 (id SERIAL, PRIMARY KEY(id));
CREATE (n:label1) RETURN n;
CREATE (n:label2) RETURN n;

MATCH (n:label1)-[r]->() WHERE n.id = 0 RETURN r;
// Binder exception: No rel table exists in database.

CREATE REL TABLE rel1 (FROM label1 TO label2);
MATCH (n:label1)-[r]->() WHERE n.id = 0 RETURN r;
// OK

MATCH (n:label1)-[r]->(k:label1) WHERE n.id = 0 RETURN r;
// Binder exception: Query node k violates schema. Expected labels are label2.

MATCH (n:label2)-[r]->() WHERE n.id = 0 RETURN r;
// Binder exception: Query node n violates schema. Expected labels are label1.

So, to query relationships from/to specific labels, there has to exist a relation table from/to exactly the same labels.
What's your view of this? It seems to me that much more logical would be to return empty results in such cases, because in many applications the node/relation tables are created on the go, and it should not affect the MATCH queries IMO.

This is a fair point though I can see the benefit of both erroring and not erroring. On the one side, if you have a database that does not have any relationship records and you're searching for relationship patterns, you're writing something against the schema you defined. So you might want the system to error. I know of cases when people write wrong queries during development that are guaranteed to return empty results, but are never notified by the system about this mistake.
On the other side, I see that the singleton pattern NOT (a)-[]->(b) that you're using is a very natural pattern, so I see why you just want to get an empty result.
I don't think there is an obvious position to take here but I'm open to changing the behavior for now and return empty results on these. And if some other users demand that they want the system to error, we can debating having a configurable "strict mode" where people can choose their behavior.
We can provide a fix to this by the next release but probably sooner on dev releases. We need to see internally which intern can take this on and when.

On the one side, if you have a database that does not have any relationship records and you're searching for relationship patterns, you're writing something against the schema you defined. So you might want the system to error.

I would totally agree with this if we were discussing relational DBs. With graph databases, though, the concept of schema is not as relevant. Moreover, in openCypher, which Kuzu claims to implement, such queries would always be valid and just return empty results.

Well, we have a clear position about how we want people to look at graph databases and particularly Kuzu, which is not as database that supports schema-less records. In fact, we are very explicit that graphs in Kuzu have schema. Therefore our data model is quite close to the relational model. One way to look at Kuzu's model is that it is the relational model with explicit identification of some records as nodes and some records as edges. We are particularly optimizing the system for cases where people have clear schema on their records but still want graph-based modeling so they can use a graph query language, specifically openCypher (with some variations). This gives users access to many good features of graph querying, though maybe not all the features offered by systems like Neo4j that adopt a more flexible model. But a priori declaration of schema is critical for our querying performance and scalability, even if it limits some other features we can offer.

Thanks for clarifying. Looking forward to the change of behavior regarding relationship querying!
P.S. It would be very nice to have a page in docs which describes in what aspects Kuzu's implementation of Cypher is different from openCypher, as well as your view on data models.

This should be fixed in #3506 and should be available in the nightly build version.


HL-162: handle breaking changes in application

Description :sparkles:

Handle breaking changes in the application
date range validation of previous benefits
Handle validation of pay_subsidy_granted

Issues :bug:
HL-162
Testing :alembic:
Screenshots :camera_flash:
Additional notes :spiral_notepad:

Codecov Report

Merging #252 (62f4935) into develop (1c5b5ef) will decrease coverage by 1.22%.
The diff coverage is 96.85%.

@@             Coverage Diff             @@
##           develop     #252      +/-   ##
===========================================
- Coverage    95.57%   94.34%   -1.23%
===========================================
  Files          135      101      -34
  Lines         4610     3662     -948
===========================================
- Hits          4406     3455     -951
- Misses         204      207       +3

Impacted Files
Coverage Δ

backend/benefit/common/utils.py
82.45% <81.81%> (-0.88%)
:arrow_down:

backend/benefit/applications/models.py
94.21% <90.00%> (+0.22%)
:arrow_up:

backend/benefit/applications/api/v1/serializers.py
95.17% <98.38%> (+0.81%)
:arrow_up:

...017_searchable_encrypted_social_security_number.py
100.00% <100.00%> (ø)

backend/benefit/applications/tests/factories.py
100.00% <100.00%> (ø)

...enefit/applications/tests/test_applications_api.py
99.79% <100.00%> (+0.02%)
:arrow_up:

...ns/tests/test_applications_api_breaking_changes.py
100.00% <100.00%> (ø)

backend/benefit/common/tests/test_utils.py
100.00% <100.00%> (ø)

backend/benefit/helsinkibenefit/settings.py
94.11% <100.00%> (+0.05%)
:arrow_up:

backend/shared/shared/audit_log/viewsets.py
46.96% <0.00%> (-42.43%)
:arrow_down:

... and 43 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1c5b5ef...62f4935. Read the comment docs.


Fix issue 45

Coverage remained the same at 97.336% when pulling 3ec4955987eb80017c027d37019a2ace295ba217 on grumpy-curmudgeon:master into bb368201761b2381bb982c875f47c55d441e9578 on sidecars:master.


JavaTemplate matching on parameterized types

This is a partial implementation, adding support for parameterized types in the grammar of the template parser.
For next step, we need to likely change marker.typeName from a string to a JavaType so that it can be a JavaType.Parameterized. The JavaType argument form of TypeUtils.isAssignableFrom I think already supports JavaType.Parameterized.

I think now it is basically working. I had to change the internal representation we use for template parameters to use a JavaType rather than a plain String for the parameter type, otherwise we can't match parameterized types.
In this area there is still some performance improvements that can be made: We should change the JavaTemplate#matcher() instance method so that it caches the matcher in a precompiled form (like Pattern#compile() basically). That would then end up caching the template parameters we create in JavaTemplateSemanticallyEqual#createTemplateParameters(): https://github.com/openrewrite/rewrite/blob/6acddbf161bd51e92d034dec12943a095fe2c483/rewrite-java/src/main/java/org/openrewrite/java/JavaTemplateSemanticallyEqual.java#L65-L124
But I will leave this optimization for another PR.
Further, I am quite sure there are still bugs in the area of TypeUtils#isAssignableTo() (which has grown very organically), but also that I will leave for another day.


gio: require Send also for the source object

Dunno... since you have to explicitly do task: gio::LocalTask<i64> etc in callbacks etc, having more generics that are often not used would be annoying imho

True. I guess we can worry about that when someone needs it

Needs a cargo fmt run


Reverse sort the list of tasks in status

Reverse the list of the status to be chronological instead of reverse
chrono.
Signed-off-by: Chmouel Boudjnah<EMAIL_ADDRESS>

Nevermind the current status since it's not updated.


2fa and Save user bugs

Due to the link at the bottom of the plugin admin going to the general admin buglist there are 2 bugs on there relating to this module.
#1988
ADMIn 2FA saving a user with 2fa enabled regenerates the 2fa code and locks out the user
#1992
attempting to save user calls with null

Both issues you mentioned are unrelated to this plugin. They are already fixed in the Core and Login plugin.

Both issues you mentioned are unrelated to this plugin. They are already fixed in the Core and Login plugin.


If 64 bit Windows but 32 bit Office (because that was the longstanding advice), then 32 bit or 64 bit Teams

If I have 64 bit Windows, then I suppose I should install 64 bit Teams.
But I have 32 bit Office 365 ProPlus, because up until last year this was the recommended option and the default.  Let's assume I didn't install Teams with Office.
Should I use 32 bit Teams or 64 bit Teams?  Teams adds extensions to Office, so I don't know.  This page doesn't offer any guidance either.
What should I do?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b2f71d9c-6b35-72c9-dd94-7f62319c1656
Version Independent ID: 86172986-6cc8-eee5-1309-704d9de0dcd0
Content: Install Teams using Microsoft Endpoint Configuration Manager - Microsoft Teams
Content Source: Teams/msi-deployment.md
Service: msteams
GitHub Login: @LanaChin
Microsoft Alias: v-lanac

It seems that 64 bit Teams integrates fine with 32 bit Office (maybe 64 bit Teams contains both 32 bit and 64 bit addins for Teams?)
Anyway, would be helpful to clarify here.

@anwarmahmood1 thank you for submitting feedback.
I think the best way forward is if you open a service ticket in your tenant so this can get resolved ASAP. Based on the outcome let me know if it is something that can be called out in the docs.
Please follow this link to contact support for business products:https://docs.microsoft.com/office365/admin/contact-support-for-business-products
Please keep us posted here on the resolution so we can feed whatever information you discover into the content.

I just GAVE you the resolution! You can add "Use Teams 64-bit with Windows 64-bit. If you have Microsoft Office 365 ProPlus installed as well, Teams 64-bit will integrate with both 32-bit or 64-bit editions of Microsoft Office 365 ProPlus."

From: Manuela Scanu<EMAIL_ADDRESS>Sent: Friday, April 24, 2020 5:01:49 AM
To: MicrosoftDocs/OfficeDocs-SkypeForBusiness<EMAIL_ADDRESS>Cc: Anwar Mahmood<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [MicrosoftDocs/OfficeDocs-SkypeForBusiness] If 64 bit Windows but 32 bit Office (because that was the longstanding advice), then 32 bit or 64 bit Teams (#4665)
@anwarmahmood1https://github.com/anwarmahmood1 thank you for submitting feedback.
I think the best way forward is if you open a service ticket in your tenant so this can get resolved ASAP. Based on the outcome let me know if it is something that can be called out in the docs.
Please follow this link to contact support for business products:https://docs.microsoft.com/office365/admin/contact-support-for-business-products
Please keep us posted here on the resolution so we can feed whatever information you discover into the content.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/MicrosoftDocs/OfficeDocs-SkypeForBusiness/issues/4665#issuecomment-618788835, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AKMOECG2IHR4FUPIIAR4EFTROEFK3ANCNFSM4MOX22EQ.
[University of Central Lancashire]
Please consider the environment before printing


feat: header operator link

Add a way to "split" the enlarge link when an operator logo is given with additional link props

Looks good to me. Thanks for taking the time to contribute upstream @lsagetlethias


the fixed dataloader bug (PCN)

You have fixed the bug in datasets/PCNDataset.py(#27), and updated the performance of PoinTr on PCN benchmark (CD from 8.38 to 7.26). But I can't find the fixed code in your code "PCNDataset.py", could you show me? Thanks in advance.

see here and here

When I test the pretrained model on KITTI using "python KITTI_metric.py". It runs very  slowly (the result "36%, 846/2401" takes 12 hours ). Is it right?

Yes, i notice that, too.  Evaluating a sample using MMD metric requires comparisons with every sample in the PCNCars, taking a lot of time.

Thanks a lot. @yuxumin


Updated way to add Affiliate Tracking Links to Checkout URL

Hi -
Is there an updated doc on how to add the affiliate tracking link to a checkout URL? I'm creating the checkout url in the backend (next.js serverless function) so assume I have to add this myself.
Since the SDK has changed now I could not find an updated method like GetId, Build or Affiliate that I could use to retrieve this. I suppose I can parse it out in the url using searchParams but I just want to know the best practice for doing this.
I'm using this doc as reference: https://docs.lemonsqueezy.com/help/affiliates-for-merchants/getting-referrals
Thanks,

Hey @btahir,
The SDK is designed for backend use and does not offer methods for frontend interaction. For working with the frontend, consider using Lemon.js: https://docs.lemonsqueezy.com/guides/developer-guide/lemonjs.
If you have access to the affiliate ID on the backend, you can create a checkout using the SDK and then append the affiliate ID in the following format: ?aff=affiliateId.
Here's an example:
import { createCheckout, lemonSqueezySetup } from "../src/index.js";

const apiKey = import.meta.env.LEMON_SQUEEZY_API_KEY;

// Setup
lemonSqueezySetup({ apiKey });

// dummy data, replace with your own
const storeId = 50039;
const variantId = 144465;
const affiliateId = "123";

// Get the checkout URL
const checkoutResponse = await createCheckout(storeId, variantId);
const checkoutUrl = checkoutResponse?.data?.data.attributes.url;

if (checkoutUrl) {
  // Create a URL object from the checkout URL
  const url = new URL(checkoutUrl);

  // Append the affiliate parameter
  url.searchParams.append("aff", affiliateId);

  const affiliateUrl = url.toString();

  console.log(affiliateUrl);
} else {
  console.error("Checkout URL not found in the response.");
}

@brankoconjic Thanks for the reply.
This makes sense but I'm still trying to think of the best way to be able to do this without adding unnecessary code.
Just to clarify - say I use my backend to generate this url e.g. https://my-store.lemonsqueezy.com/checkout/buy/0a841cf5-4cc2-4bbb-ae5d-9529d97deec6
If I have he affiliate lemonjs scripts in my html

And I navigate on the client to this url e.g. router.push(checkout)url)
Would the script automatically add the affiliate tracking cookie to the checkout session?

Yes, the script will automatically create a cookie and include it with the request.

Great!


External IP not exposed.

Hi,
I had this working at one time, now it doesn't work anymore.
What I'm trying to do is create a dask cluster on microk8s kubernetes.
According to the Helm website: https://hub.helm.sh/charts/dask/dask, to deploy the cluster I must type in the following:
helm repo add dask https://helm.dask.org/
helm repo update
helm install --name my-release dask/dask

However, I performed a microk8s kubectl get svc I don't see an external IP signed:
NAME                TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)           AGE
kubernetes          ClusterIP   <IP_ADDRESS>     <none>        443/TCP           40h
my-dask-jupyter     ClusterIP   <IP_ADDRESS>   <none>        80/TCP            12m
my-dask-scheduler   ClusterIP   <IP_ADDRESS>    <none>        8786/TCP,80/TCP   12m

When I performed a the following I get a null value:
  echo http://$DASK_SCHEDULER_UI_IP:$DASK_SCHEDULER_UI_PORT     -- Dask dashboard
  echo http://$JUPYTER_NOTEBOOK_IP:$JUPYTER_NOTEBOOK_PORT       -- Jupyter notebook

Please help, I think there may be some setup I need to perform with microk8s?
Thanks,

@vinhdiesal ExternalIP is only applicable if you have a load balancer in place.  Most cloud providers have their cloud controller create an external ip for you.
In microk8s, we have Metallb addon, which can provide an external IP.
microk8s enable metallb:<specify ip range>

Thanks @balchua I re-installed the helm with the load balancer override and it now works.


Standard/default foundries

Standard or default foundries section should be configurable for each instances. For the main KorAP instance, it should also include the structure (base), dependency (malt), constituent (corenlp), and morphology (marmot) foundries.

Can you explain what this is refering to? Are you refering to the documentation?


Have tired following information on webpage to activate magic wand but no luck getting it to appear

I'm trying to use the magic wand feature but there is no option for it, looking in the closed logs I found a similar issue and a link to the label-studio web page. I've followed the information on the webpage to the best of my ability but no luck.
I'm running on a windows machine and anaconda.
I tried setting the environment variable and couldn't fathom out how to launch the sever as the page suggested. Furthermore I couldn't find the files to set it manually.
Might just be me not being super au fai with this stuff but I don't really know what else to do!
Thanks in advance for any help you can offer. Although I have to admit I'm a little confused as to why the headline feature release isn't available by default.

Do you see Magic Wand in toolbar? If yes, but it doesn't work, that's most probably because of conflict in feature flags, you have to disable one to use this tool. They can be disabled by env vars like this in unix systems: fflag_feat_front_lsdv_4583_6_images_preloading_short=0 label-studio. Not sure how to set it in your setup, maybe via SET in bash or via anaconda toolset.
@makseq can you help with this?

And if you don't see it, please, check config from Magic Wand doc and set fflag_feat_front_dev_4081_magic_wand_tool=1 in addition to disabling aforementioned var.

Thank you so much. I have successfully activated Magic Wand according to your advice.

@YugangDuan did you install LS from pip? pip install label-studio?

Hi, I've tried following the above and still can't get it to appear. I've created a new env to make sure I wasn't being dumb and still nothing. I have to also say that I'm not a developer or anything fancy. Just your fairly newish python user who spends a lot of time in ML space. So please feel free to reply to this as if I know literally nothing. Thanks again!

What labeling config are you using?

I've tried semantic segmentation with polygons and masks

Further information: when I run "fflag_feat_front_dev_4081_magic_wand_tool=1 label-studio start" I get the error 'fflag_feat_front_dev_4081_magic_wand_tool' is not recognized as an internal or external command,
operable program or batch file." I also can't seem to find anything about feature_flags where you'd set something as true.

You need add “fflag_feat_front_dev_4081_magic_wand_tool=1” in your environment. Not run it on the terminal.

Update: So I worked my way around environment vars. Using
conda env config vars set fflag_feat_front_dev_4081_magic_wand_tool=1 from within my env. Before reactivating it. However, sadly still no sight of the wand tool.

Do you set the labeling config?

Check labeling config examples here:
https://labelstud.io/tags/magicwand.html#Example
You must use <Magicwand> tag.


[question] How to pass data from meddleware to route withou using decorator

since there is no context object and request object can not be modified, so how can i pass data from middleware to route function? assign data in request.args is not secure.
thanks for any suggestion

This is a great question.
There is no proper way to do it at the moment.
I have a plugin where I need to pass a message from a Request middleware to a corresponding Response middleware. That is very difficult and similar to your question.
They way I do it, is to add an entry in the request headers dict:
# set a const for your variable
MY_MIDDLEWARE_VAR = "x-my-middlware-var"
#in middleware:
async def my_middlware(request):
    result = do_processiong()
    request.headers[MY_MIDDLEWARE_VAR] = result

#in route handler
async def index(request):
    middlware_result = request.headers.pop(MY_MIDDLEWARE_VAR)
    ...

@ashleysommer thanks, i actually did the same, but instead, the key name i use is hashed  in order to improve security level

As this was a question, not a bug report or feature request, if you believe your question has been adequately answered, can you please close this issue.
Go ahead and open an issue for a feature request if you'd like to see a request context added, however that might be better implemented using a plugin.


Capture logs, print them all if it failed, otherwise, just a summary

Capture logs while running a @test_rel generated test set. If the test passes, just print a summary info. If it does not pass, bundle and log all of the captured logs along with metadata.
Example output: https://github.com/RelationalAI/raicode/actions/runs/4915893587/jobs/8779111254
Questions:

What is QuietTestSet? I've hijacked it a bit and am not sure what the implications are here.

Would we get any of this "for free" if we integrated with ReTestItems?

Would we get any of this "for free" if we integrated with ReTestItems? I think I remember hearing that it does this

It's  bit unclear as the concurrency model of ReTestItems vs RAITest is pretty different due to cpu bound vs network bound

removed the tests - logs are smaller by more than half now
https://github.com/RelationalAI/raicode/actions/runs/4932132166/jobs/8814945176


Authentication Layer Implementation

Following the discussion on #10 this PR will address the implementation for the Authentication Layer of the framework.
This is still in an early stage, so I'll be updating the description or adding more comments as I go.
I created it just to get initial feedback if I'm going in the right direction.
Can you take a look and see if I should change anything on this initial approach @brendt?

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

2 of 22 (9.09%)  changed or added relevant lines in 6 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-1.1%) to 84.964%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/Auth/Authenticators/GenericAuthenticator.php
0
1
0.0%

src/Auth/Exceptions/InvalidLoginException.php
0
1
0.0%

src/Auth/AuthenticatorInitializer.php
0
2
0.0%

src/Auth/Authenticators/DatabaseAuthenticator.php
0
16
0.0%

Totals

Change from base Build<PHONE_NUMBER>:
-1.1%

Covered Lines:
1294

Relevant Lines:
1523

💛  - Coveralls

@brendt I worked on the changes from the previous comments and created a really simple and minimalist implementation for the DatabaseAuthenticator, can you take a look and let me know other things that you want the auth layer to provide? Also I created a HasIdentity trait that can be used in a model, for example in a User class:
final class User implements Identifiable, Model
{
    use HasIdentity;
}

That would already set this as the model used for authentication and by default it would use the email and password fields for the authentication.
To customize this, it would need to simply provide new values for the IDENTIFIER and SECRET constants:
final class User implements Identifiable, Model
{
    use HasIdentity;

    protected const string IDENTIFIER = 'username';

    protected const string SECRET = 'secret';
}

I'm not encrypting anything here, do you think we should offer encrypting the password to be transparent for the devs?

I'm not encrypting anything here, do you think we should offer encrypting the password to be transparent for the devs?

Definitely, passwords should always be encrypted and devs shouldn't worry about it

I'm not encrypting anything here, do you think we should offer encrypting the password to be transparent for the devs?

Definitely, passwords should always be encrypted and devs shouldn't worry about it

To clarify, passwords should be hashed + salted, not just encrypted. 🙂
This is probably worth a separate PR for that functionality.

@brendt I worked on the things that you mentioned and also added an example of how the usage of the Auth layer would look with the current implementation in the app folder.
Please let me know if things are looking good and what should be the next steps!

To clarify, passwords should be hashed + salted, not just encrypted. 🙂

The password_hash function already uses a salt.

To clarify, passwords should be hashed + salted, not just encrypted. 🙂
This is probably worth a separate PR for that functionality.

True, sorry for the confusion!
I think we can use PHP's hashing library? I don't see a need to add a layer on top of it for now tbh

Oh damn...
I knew I had typed the wrong alias for some git commands
I screwed the git history for this branch 😢

@brendt can you take a look and check if the implementation is going in the right direction now and what should we improve? I can fix the git history before the merge if needed.

Hey @brendt I worked on some changes, please let me know which approach you want to follow for the AuthenticationCall interface. The other observations that you made I think I addressed all of them.
Regarding the Git History, when this looks good and no other changes are needed I'll create a new PR with it fixed.

Gonna merge this in a feature branch, and finetune it on there 👍 Thanks for the help, and sorry it took so long to get merged


No errors are returned, but the devices do not receive notifications.

The logs from Apns_PHP echo "STATUS: Sending message" once per notification, they never fail, and then it echoes "Disconnected". No errors are reported, but the devices never receive the notifications (trying with 3 different devices, all have their notifications enabled for the app)
Attempting to open a socket and sending a binary notification with a simple example using the same certificate file and tokens works.

Same Issue for me!

@xabaras check out #86 for a temporary fix, and the ultimate fix/reason

I have the same issue.


Toggle Comment does not work

Neither line nor block commenting seems to work.

Fixed in 53a876da0759183411fdecbc0427acc69b4adfda and released in 1.1.0


OpenXR: Fix enabling passthrough via environment blend mode in project settings

PR https://github.com/godotengine/godot/pull/87630 made it possible to enable passthrough by setting the environment blend mode to "alpha blend", even on headsets that need to use a special passthrough extension - in that case it works by emulating alpha blend mode.
Currently, this is working great if you set the environment blend mode via code!
However, it you set it in project settings, you'll get this error:
OpenXR: XR_ENVIRONMENT_BLEND_MODE_ALPHA_BLEND isn't supported, defaulting to XR_ENVIRONMENT_BLEND_MODE_OPAQUE

This is because we're checking the blend mode before OpenXR extension wrappers have had the opportunity to enable emulation of alpha blend mode.
This PR basically moves these checks to later, after on_session_created() is run on all the extension wrappers.

Thanks!


Add ICEES COVID dataset to ICEES KG

This issue is to suggest that we add the ICEES COVID dataset to ICEES KG. The suggestion is driving by the fact that Ashok and I are renewing our efforts with ICEES COVID, but focusing on LONG COVID rather than acute coronavirus infection. I updated the covid all_features YAML file and have an outstanding PR, so I'm hoping that this is a relatively simple request.

Closing as this is complete ...


Support for nested config files

Maybe this I've just been setting it up wrong, but I don't think this currently works (or maybe it's a CLI/VS Code issue). Similar to ESLint, sometimes subfolders need a separate set of rules from the root configuration.

.markdownlint.json
docs

new docs
old docs

.markdownlint.json (to turn off some rules)

The VS Code extension chooses the “nearest” version of .markdownlint.json. The CLI uses a single version, either what you specify as an argument or from the current directory. The Node package uses whatever you pass in the config object.
Because it’s configured once and scripted, my suggestion is to make multiple passes with the CLI, ignoring directories and passing different configurations as needed to treat each set of files like you want.
Or open an issue on the CLI to match the VS Code behavior. This isn’t the first time the question has come up.

To be clear, I do not think this is the responsibility of the Node package – it just does what you tell it and is there to be used by other tools for richer scenarios like this one.

This isn’t the first time the question has come up.

I took a quick search first, but didn't find anything 😊
I'm finding this come up on the MicrosoftDocs repos, because they sometimes have "legacy" docs they don't really want touched. I'm trying to figure out how a plain markdownlint "**/*.md" and VS Code editing experience should work in your view.
EX: Should the nested configs override a root config, or should they merge/extend? I can see both ways can be preferable to different users.

Sorry, I didn’t mean to suggest you didn’t look, just that you’re not the only one thinking about this. :) For example: https://github.com/igorshubovych/markdownlint-cli/issues/53
For the scenario you describe above, I might explore using extends first because that lets you exclude unwanted rules in just one place. DRY, but for configuration.

Closing this issue as it does not seem to be applicable to the library and is tracked in the CLI with the issue above.


Auto Persist Model Changes to Folder (Git Repo)

Hi Daniel,
At the moment the only way to persist to folder is to save to folder which sometimes dumps a load of stuff your not interested in.
When I open a new branch in a git repo to do some work it would be nice for TE to write the changes directly to the folder automatically. That way only the stuff I'm adding or changing will show up as items to stage and commit.
This could be enabled for both offline and connected models and would speed up the development process (GitHub Flow).
What do you think?
Regards
Toby

Sorry for the delay in communication.

sometimes dumps a load of stuff your not interested in

Could you be a little more specific about this? Tabular Editor should only save changes back to the file system. So I'm wondering what the stuff you're not interested in is. For example, if you rename a measure, it should be expected that DAX expressions in other measures, calc tables, etc. might also change due to formula fix-up.


Transformations

Finally added the document transformation code!!
Updated SkoogleDocs::Document class to use the new SkoogleDocs::Transformer
Disabled RSpec monkey patching
Included tests and documentation

Coverage increased (+0.65%) to 95.85% when pulling 2ded5168748338b9d1dc88b384d62428919a9222 on transformations into 7979687c98c476cfa0dc83ad0c822103c6fc38ec on master.

Looks really good. I especially like the use of FactoryGirl for the HTML docs. Just had one very minor comment, otherwise :shipit:

very nice, @Enriikke

Coverage increased (+3.2%) to 98.4% when pulling f6e0d25ba42c67412932ef7bbbd5d21db5ea5135 on transformations into 7979687c98c476cfa0dc83ad0c822103c6fc38ec on master.

Coverage increased (+3.2%) to 98.4% when pulling f6e0d25ba42c67412932ef7bbbd5d21db5ea5135 on transformations into 7979687c98c476cfa0dc83ad0c822103c6fc38ec on master.


technology CSV should be updated to deal with new attributes

See https://github.com/quintel/etmoses/issues/595

Quick question though; should the user be able to specifiy these attrbiutes by hand in their csv or does this need to be filled automatically from EtMoses or both (depending on the fact if the columns in question are empty or not?)

Quick question though; should the user be able to specifiy these attrbiutes by hand in their csv or does this need to be filled automatically from EtMoses or both (depending on the fact if the columns in question are empty or not?)

Both. ETMoses should fill in the values that are in the interface, whether the user created those or not (when they are the default values). The user should be able to edit the CSV though (otherwise there would be little use for it).

This is already fixed


Signature extraction failed

$ youtube-dl --verbose https://www.youtube.com/watch?v=6MyjFHd04R8
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['--verbose', 'https://www.youtube.com/watch?v=6MyjFHd04R8']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2014.08.05
[debug] Python version 2.7.8 - Linux-3.16.0-44-generic-x86_64-with-Ubuntu-14.10-utopic
[debug] Proxy map: {}
[youtube] Setting language
[youtube] 6MyjFHd04R8: Downloading webpage
[youtube] 6MyjFHd04R8: Downloading video info webpage
[youtube] 6MyjFHd04R8: Extracting video information
[youtube] 6MyjFHd04R8: Encrypted signatures detected.
[youtube] {22} signature length 46.43, html5 player new
[youtube] 6MyjFHd04R8: Downloading js player new
ERROR: Signature extraction failed: Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/youtube_dl/extractor/youtube.py", line 479, in _decrypt_signature
video_id, player_url, s
File "/usr/lib/python2.7/dist-packages/youtube_dl/extractor/youtube.py", line 383, in _extract_signature_function
res = self._parse_sig_js(code)
File "/usr/lib/python2.7/dist-packages/youtube_dl/extractor/youtube.py", line 454, in _parse_sig_js
u'Initial JS player signature function name')
File "/usr/lib/python2.7/dist-packages/youtube_dl/extractor/common.py", line 391, in _search_regex
raise RegexNotFoundError(u'Unable to extract %s' % _name)
RegexNotFoundError: Unable to extract Initial JS player signature function name; please report this issue on https://yt-dl.org/bug . Be sure to call youtube-dl with the --verbose flag and include its complete output. Make sure you are using the latest version; type  youtube-dl -U  to update.
; please report this issue on https://yt-dl.org/bug . Be sure to call youtube-dl with the --verbose flag and include its complete output. Make sure you are using the latest version; type  youtube-dl -U  to update.
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/youtube_dl/extractor/youtube.py", line 479, in _decrypt_signature
video_id, player_url, s
File "/usr/lib/python2.7/dist-packages/youtube_dl/extractor/youtube.py", line 383, in _extract_signature_function
res = self._parse_sig_js(code)
File "/usr/lib/python2.7/dist-packages/youtube_dl/extractor/youtube.py", line 454, in _parse_sig_js
u'Initial JS player signature function name')
File "/usr/lib/python2.7/dist-packages/youtube_dl/extractor/common.py", line 391, in _search_regex
raise RegexNotFoundError(u'Unable to extract %s' % _name)
RegexNotFoundError: Unable to extract Initial JS player signature function name; please report this issue on https://yt-dl.org/bug . Be sure to call youtube-dl with the --verbose flag and include its complete output. Make sure you are using the latest version; type  youtube-dl -U  to update.
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/youtube_dl/YoutubeDL.py", line 516, in extract_info
ie_result = ie.extract(url)
File "/usr/lib/python2.7/dist-packages/youtube_dl/extractor/common.py", line 170, in extract
return self._real_extract(url)
File "/usr/lib/python2.7/dist-packages/youtube_dl/extractor/youtube.py", line 860, in _real_extract
encrypted_sig, video_id, player_url, age_gate)
File "/usr/lib/python2.7/dist-packages/youtube_dl/extractor/youtube.py", line 489, in _decrypt_signature
u'Signature extraction failed: ' + tb, cause=e)
ExtractorError: Signature extraction failed: Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/youtube_dl/extractor/youtube.py", line 479, in _decrypt_signature
video_id, player_url, s
File "/usr/lib/python2.7/dist-packages/youtube_dl/extractor/youtube.py", line 383, in _extract_signature_function
res = self._parse_sig_js(code)
File "/usr/lib/python2.7/dist-packages/youtube_dl/extractor/youtube.py", line 454, in _parse_sig_js
u'Initial JS player signature function name')
File "/usr/lib/python2.7/dist-packages/youtube_dl/extractor/common.py", line 391, in _search_regex
raise RegexNotFoundError(u'Unable to extract %s' % _name)
RegexNotFoundError: Unable to extract Initial JS player signature function name; please report this issue on https://yt-dl.org/bug . Be sure to call youtube-dl with the --verbose flag and include its complete output. Make sure you are using the latest version; type  youtube-dl -U  to update.
; please report this issue on https://yt-dl.org/bug . Be sure to call youtube-dl with the --verbose flag and include its complete output. Make sure you are using the latest version; type  youtube-dl -U  to update.

You are using an outdated version of youtube-dl. Read our FAQ if you have troubles updating.

I'm using the latest version from the repository vailable Ubuntu using apt-get, who is responsible for that software in the Ubuntu repositories are not functional and updated?
Anyway I will use the version found in git, thank you very much.

who is responsible for that software in the Ubuntu repositories are not functional and updated?

Ubuntu maintainers obviously.


Faster wires

Context:
New, faster wires, and a few touches here and there.
Description of the Change:
Wires now works with sets and it's unordered in its internal operation. When it has to return modes in an order it will choose increasing order.
Benefits:
Much simpler implementation, super fast

I fixed the tests and addressed the last comments.
However, I wasn't able to address the most important one, namely how to change the ids (this comment). I am worried that they may not be changeable because of the caching. Let's make sure before we merge


Add guidelines for pulling improvements into netlify-deployed clones

Not sure if/how this is possible but as Netlify-deployed versions aren't formal forks I'm not sure how to pull improvements in.

Good shout. What I'm doing with the releases is listing out file changes, but any ideas to help make things easier for folks are more than welcome in this camp.

I think this route requires a fair bit of git knowledge. I think the trick is documenting various options, including this one, @orx57.

These two links
https://help.github.com/en/github/getting-started-with-github/fork-a-repo#keep-your-fork-synced
https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/syncing-a-fork
helped me to upgrade a Netlify-deployed version from v0.6.0 to v0.7.0 without any problems.
I think it's the same solution which @orx57 described.


How to close the app after intercepting with options.preventDefault?

Is there a method I can call below to close the app?
    tabris.app.on("backnavigation", function(app, options) {
        if (tabris.ui.get("activePage").get("topLevel")) {
            navigator.notification.confirm(
                'Do you wish to exit the app?',
                function(buttonIndex) {
                    if (buttonIndex == 1) {
                        // What should I do here?
                    }
                }, 'Exit', ['Yes','No']
            );
            options.preventDefault = true;
        }
    });

Small hint: It is considered back practice to blog the user from leaving the app. Such question dialog could make sense in the context of a question stating:

"Your "input" will be lost. Do you want to leave 'this screen'?"

That being said... You could close the current page with page.close() which would also close the app if it is the last page on the stack.

I tried that too. Unfortunately, closing the last page on the stack doesn't work. It throws the error "Cannot close the last page".
All too often, the user will accidentally exit the app by pressing the Back button in quick succession. Hence many commercial apps actually introduce this mechanism on Android to prevent accidental exits, since re-opening the app may take awhile.

Ah okay. I wasn't aware that you couldn't close the app by closing the last page. We will discuss how to better handle that case. Thanks for opening the issue.
Btw... those other apps "are doing it wrong". ;)

I thought of a half-baked workaround that shows my own custom dialog (created with a Composite), instead of the native one, with a message saying "Press back again to exit the app", and a single Cancel button to close the "dialog". Although this still doesn't stop the user from accidentally exiting the app by spamming the back button in quick succession, it does help users who forgot that they are already at a top level page, and the next press would exit the app.

That sounds like sort of valid workaround. It will not be as pretty but could do the job.

does "backnavigation" work on iOS?

@bFlood Natively there is no concept of "app-closing button" on iOS. So the answer is no. The only way is to go to task manager and close app from there.

thanks, I realize that but I was wondering if the event is fired when you go back a page in the stack (either by tapping the navbar button or swiping)
it doesn't appear to but it would be nice to make this similar to android

I still think that closing the last page on the stack should also close the app. Therefore we keep this issue around.

With the migration to the NavigationView we should consider adding a concrete method like tabris.app.close() to close the app since the the page back metapher will not work with multiple NavigationViews.

This is impossible on iOS.

With the addition of the App.close() on Android this issue has been resolved since such a feature is not available on iOS.
http://docs.tabris.com/latest/api/app.html#close


after restore hexa Notification does not work

After hexa restoration keeper can't send notification to hexa because keeper does not have new fcm token so can not complete approval process after restore.

Old KBHR  flow


docs: Add a sample script for VM automation of partitioning

This PR adds an example shell script that will generate partitions for a table and then run validations in parallel.
It is an example for a customer who wants to run parallel validations locally on a GCE VM.

/gcbrun

Neil,
This is a great idea. I don't know how you came up with some of the constants used to decide on the size and number of partitions.
I am concerned that MB_PER_MILLION_ROWS=1500 is too aggressive based on my experience. With Cloud Run, I noticed that for a 512MB Cloud Run instance, I could run validations on partitions of about 60K rows and end up with nearly 80% memory utilization. In my testing and review of memory usage, there were two components to the memory usage. The first was the rows we pulled from the database. The next was the combiner which combines both set of rows to come up with a new dataframe. If there are large number of Fail, the combiner will have more rows and may need more memory. I also don't know if the combiner's memory needs scale linearly with the number of input rows or they scale at ^2 or nlog(n). My measurements also showed that the combiner used more memory than fetching the rows.
How do we reconcile this?
Thanks.
Sundar Mudupalli

Neil,
I don't know how you came up with some of the constants used to decide on the size and number of partitions.
I am concerned that MB_PER_MILLION_ROWS=1500 is too aggressive based on my experience.

To arrive at this value I used the vmstat command on Linux while running validations. For example note the values in the free column below during a row hash validation of 10 million rows.
$ vmstat 5
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0      0 32374884  17164 283280    0    0   171     7   59   58  1  0 98  0  0
 1  0      0 32112184  17164 283408    0    0    26     0 3107 3135  7  2 90  0  0
 0  0      0 31658860  17164 283408    0    0     0     0 11767 10985  7  2 92  0  0
 1  0      0 31212372  17164 283408    0    0     0     0 11110 10430  8  2 91  0  0
 0  0      0 30749292  17164 283408    0    0     0     0 11580 10966  6  2 92  0  0
 0  0      0 30270072  17164 283408    0    0     0     0 12088 11535  7  2 91  0  0
 0  0      0 29882364  17164 283408    0    0     0     0 8664 8982  6  2 91  0  0
…
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 2  0      0 20931412  17172 283408    0    0   123     5  237  225  5  1 94  0  0
 1  0      0 22020376  17172 283408    0    0     0     0  281   36 19  6 75  0  0
 1  0      0 22158892  17180 283408    0    0     0     2  290   51 21  4 75  0  0
 1  0      0 22149188  17196 283408    0    0     0     3  284   42 25  0 75  0  0
 1  0      0 20654892  17196 283408    0    0     0    10  276   34 20  5 75  0  0
 1  0      0 19715592  17204 283408    0    0     0     1  281   38 22  3 75  0  0
 1  0      0 19781640  17204 283408    0    0     0     0  283   49 24  1 75  0  0
 0  0      0 32372096  17204 283408    0    0     0     0  239   40 17  3 79  0  0

Free memory dropped from 32.3GB to 19.7GB (used 12.6GB). I saw similar memory usage (occasionally a little higher) for my 1 million and 20 million row test tables, occasionally a little higher. I could round it up to 2GB to give some extra wiggle room?
I assumed the combiner would have to join all records in order to figure out which are fails and which are success. I'll repeat my tests without --filter-status=fail.

@sundar-mudupalli-work , you are correct, the memory consumption is greater if I remove the filter, the table below shows GBs of RAM per million rows from my tests:

Table
No filter
Filter status fail

tab_vol_1m
1.6
1.3

tab_vol_2m
3.7
1.4

tab_vol_5m
3.5
1.1

tab_vol_10m
4.8
1.3

/gcbrun

/gcbrun

/gcbrun


Require PIN for registering an  hotp credential

Motivation: https://github.com/Nitrokey/nitrokey-hotp-verification/issues/30

@sosthene-nitrokey can you link to parallel work for hotp-verification and Heads code needed to deal with this?

My testing show registering a TOTP secret still requires no PIN to be set.
Is this indented? WHy is there this asymmetry between HOTP and TOTP?

The use case is different and leads to different threat models.
HOTP is used so that the key can verify that the code sent to it is correct. Being able to create an HOTP credential could allow verification against a newly created credential instead of the real one.
On the other hand TOTP is used to authenticate the key to a service. Being able to create false credential would only mean that the key would not be able to be authenticated, leading to an explicit failure, instead of authentication working when it shouldn't.
You should not that it it also not possible to create a pin-protected without first presenting the PIN.

To be precise, HOTP and TOTP credentials are handled identically.  What is different and affected by this PR is Reverse HOTP, sometimes shortened to HOTP.


Unable to save and open topology and level 2 script in MiniEdit.

I used "Mininet 2.3.0" with VM files "mininet-2.3.0-210211-ubuntu-20.04.1-legacy-server-amd64-ovf.zip" latest release. I run it in Virtualbox platform. I faced a problem when I run Miniedit and save the .mn files. The .mn file that I saved is empty when open. Any solution can I fix this problem?

isn't this the exact same issue as #1086 ? i think this can be closed in that case @lantz

closed as dup of #1086


Erro when select a column with float with size 8

Where I ran the following sql, retun "thread 'main' panicked at 'unsupported fixed type decoding: Float8'"
SELECT c5_num, RTRIM(a1_nome), c5_emissao, (c5_valmerc - c5_valdesc) FROM sc5
The c5_valmerc and the c5_valdesc are float with length 8

What MSSQL version are you using?

Microsoft SQL Server 2017 (RTM) - 14.0.1000.169 (X64)   Aug 22 2017 17:04:49   Copyright (C) 2017 Microsoft Corporation  Express Edition (64-bit) on Windows Server 2012 R2 Datacenter 6.3  (Build 9600: ) (Hypervisor)
By the way, thx for sharing your knowledge.

Thanks, fixed.

The version 0.3.2 already has the correction for this problem???

0.3.2 is 10 days old, so no, 0.3.3 will have it

Ok. Thank you again. Nice work!


  This fixes the
windows build.

Replace this paragraph with a description of your changes and rationale. Provide links to external references/discussions if appropriate.

Resolves SR-NNNN.

@swift-ci please test and merge

@swift-ci please test and merge


blobContentDisposition ignored when uploading a file to blob storage

Package Name: @azure/storage-blob

Package Version: 12.1.1

Operating system: Arch Linux

[ ] nodejs 12.16.2

Is the bug related to documentation in

[ ] README.md
[ ] source code documentation
[x] SDK API docs on https://docs.microsoft.com

Describe the bug
When uploading files to the Azure blob storage using uploadStream, the http header for blobContentDisposition is being ignored.
Look at the following code:
const blobServiceClient = BlobServiceClient.fromConnectionString(process.env.AZURE_BLOB_STORAGE_CONN!);
const containerClient = blobServiceClient.getContainerClient(keyPrefix);

const keyName = path.join(keyPrefix, uniqueLinkId)
let blockBlobClient = containerClient.getBlockBlobClient(uniqueLinkId)

blockBlobClient.setHTTPHeaders({"blobContentType": contentType!, "blobContentDisposition": "attachment; filename=" + downloadFileName})

const uploadBlobResponse = await blockBlobClient.uploadStream(fileStream, undefined, undefined, {blobHTTPHeaders: {blobContentType: contentType!}})

After the file is uploaded and when looking at the properties of the file using "Storage explorer (preview)" on the Azure portal, the contentType is set successfully but the contentDisposition remains empty. I've tried different variants of contentDisposition buch all are ignored.
To Reproduce
Steps to reproduce the behavior:

Set the header using setHTTPHeaders and include the blobContentDisposition value
Upload the file using uploadStream

Expected behavior
The contentDisposition should hold the value assigned when looking at it on the Azure portal.

@flashtheman the Azure Portal might not display every header.  Have you tried to retrieve the headers of the uploaded blob and verify that it include the contentDisposition value that was set previously by setHTTPHeaders()
    const getPropertiesResponse = await blockBlobClient.getProperties();
    console.log("properties:")
    console.log(getPropertiesResponse)

@flashtheman the Azure Portal might not display every header. Have you tried to retrieve the headers of the uploaded blob and verify that it include the contentDisposition value that was set previously by setHTTPHeaders()
    const getPropertiesResponse = await blockBlobClient.getProperties();
    console.log("properties:")
    console.log(getPropertiesResponse)

Thanks but I tried it. It's definitely not set.

FYI: It works when I set it manually in the portal using the new "Storage explorer (preview)" explorer.

Interesting, in my test code it worked:
  const uploadBlobResponse = await blockBlobClient.uploadStream(stream, undefined, undefined
    );
    console.log("upload response")
    console.log(uploadBlobResponse);

    const setHttpResponse = await blockBlobClient.setHTTPHeaders({"blobContentType": "contentType1", "blobContentDisposition": "attachment; filename=package.json"})
    console.log("setHttpHeaders response")
    console.log(setHttpResponse);
    const getPropertiesResponse = await blockBlobClient.getProperties();
    console.log("properties:")
    console.log(getPropertiesResponse)

Here's part of the get properties response

I had to upload first then set http headers. Otherwise I got an error <?xml version="1.0" encoding="utf-8"?><Error>BlobNotFound<Message>The specified blob does not exist.
On a side note, you could set the blob http headers when uploading:
  const uploadBlobResponse = await blockBlobClient.uploadStream(
    stream,
    undefined,
    undefined,
    {
      blobHTTPHeaders: {
        blobContentType: "contentType2",
        blobContentDisposition: "attachment; filename=package.json",
      },
    }
  );

Expand details to see my full test code:

// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.
/*
Setup: Enter your storage account name and shared key in main()
*/
import {
BlobServiceClient,
StorageSharedKeyCredential,
} from "@azure/storage-blob";
import * as fs from "fs";
async function main() {
// Enter your storage account name and shared key
const account = process.env.ACCOUNT_NAME || "";
const accountKey = process.env.ACCOUNT_KEY || "";
// Use StorageSharedKeyCredential with storage account and account key
// StorageSharedKeyCredential is only avaiable in Node.js runtime, not in browsers
const sharedKeyCredential = new StorageSharedKeyCredential(
account,
accountKey
);
const blobServiceClient = new BlobServiceClient(
https://${account}.blob.core.windows.net,
sharedKeyCredential
//pipelineOptions
);
// Create a container
const containerName = newcontainer${new Date().getTime()};
const containerClient = blobServiceClient.getContainerClient(containerName);
const createContainerResponse = await containerClient.create();
console.log(
Create container ${containerName} successfully,
createContainerResponse.requestId
);
const blockBlobClient = containerClient.getBlockBlobClient(
newblob${new Date().getTime()}
);
const stream = fs.createReadStream("./package.json");
//blockBlobClient.setHTTPHeaders({ blobContentType: "contentType", blobContentDisposition: "attachment; filename=package.json"})
const uploadBlobResponse = await blockBlobClient.uploadStream(
stream,
undefined,
undefined,
{
blobHTTPHeaders: {
blobContentType: "contentType2",
blobContentDisposition: "attachment; filename=package.json",
},
}
);
console.log("upload response");
console.log(uploadBlobResponse);
// const setHttpResponse = await blockBlobClient.setHTTPHeaders({"blobContentType": "contentType1", "blobContentDisposition": "attachment; filename=package.json"})
// console.log("setHttpHeaders response")
// console.log(setHttpResponse);
const getPropertiesResponse = await blockBlobClient.getProperties();
console.log("properties:");
console.log(getPropertiesResponse);
// Delete container
//await containerClient.delete();
console.log("deleted container");
}
main().catch((err) => {
console.error("Error running sample:", err.message);
});

Can you try to await on the setHTTPHeaders() call? It could be throwing errors but you didn't handle it?

What if getProperties returns contentDisposition as it is defined but the response lacks that header? Are there any other settings that need to be set for this to work?

The getProperties() uses HEAD operation behind the scene. Here's an example
HEAD /newcontainer1588095956354/newblob1588095956939 HTTP/1.1

The response headers include Content-Disposition:

@abarisic86 does this answer your question? Or do you expect something else from the library?

Not exactly, retrieving the actual blob doesn't contain the header. Even
with specified Content-Disposition in Access-Control-Expose-Headers. Is
there any other property that needs to be set for this to work?
On Tue, Apr 28, 2020 at 7:59 PM Jeremy Meng<EMAIL_ADDRESS>wrote:

The getProperties() uses HEAD operation behind the scene. Here's an
example
HEAD /newcontainer1588095956354/newblob1588095956939 HTTP/1.1
The response headers include Content-Disposition:
[image: image]
https://user-images.githubusercontent.com/7583839/80520858-eb7c7000-893e-11ea-9ca8-c70e28b379a9.png
@abarisic86 https://github.com/abarisic86 does this answer your
question? Or do you expect something else from the library?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Azure/azure-sdk-for-js/issues/8553#issuecomment-620763901,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABFYME5HWEI5TM622DTLIH3RO4KPJANCNFSM4MRTSGZA
.

retrieving the actual blob doesn't contain the header.

Retrieving using this library, or other clients (browsers for example)? I just tested using curl to download a blob and the response headers contains the expected Content-Disposition.  Do you have a detailed steps that reproduce your issue (assuming it's different than in the original issue report)?

In the code, I noticed uploadStream didn't provide blobContentDisposition property:
blockBlobClient.setHTTPHeaders({"blobContentType": contentType!, "blobContentDisposition": "attachment; filename=" + downloadFileName})

const uploadBlobResponse = await blockBlobClient.uploadStream(fileStream, undefined, undefined, {blobHTTPHeaders: {blobContentType: contentType!}})

During upload, would you try to set blobContentDisposition explicitly?
When it's not set explicitly, service might cleaned up contentDisposition already set with setHTTPHeaders.
Thanks,
Jiachen

In the code, I noticed uploadStream didn't provide blobContentDisposition property:

You are right, this solves the issue. It seems I'm overwriting the header was setting in the previous line. Sorry for this.
Thanks!!!

Glad to know it helps. Closing the issue.


InterfaceRedundancyGroupAssociation CustomFiled

Environment

Nautobot version (Docker tag too if applicable): 1.6.4
Python version: 3.11.6
Database platform, version: PG13
Middleware(s):

Steps to Reproduce

create custom field: dcim | interface redundancy group association
retrieve CF from InterfaceRedundancyGroupAssociation object

Expected Behavior
>>>  InterfaceRedundancyGroupAssociation.objects.first().cf
{}

Observed Behavior
>>> InterfaceRedundancyGroupAssociation.objects.first().cf
Traceback (most recent call last):
  File "<console>", line 1, in <module>
AttributeError: 'InterfaceRedundancyGroupAssociation' object has no attribute 'cf'

Interesting. InterfaceRedundancyGroupAssociation model doesn't support custom fields, and I don't see dcim | interface redundancy group association as an option when defining a Custom Field in the UI. How did you create the custom field?

Ah thanks. I was testing in 2.0 rather than 1.6; it looks like the option to define them was removed as a side effect of some of the other changes we've made in 2.0.

A bit of context... these many-to-many association through models are a newer pattern for us and we haven't yet considered how to make them first class citizens, in terms of things like extensibility feature sets. This undoubtedly leads to some of the inconsistency, for example, being able to declare a field, despite _custom_field_data not actually existing on the model.
I think the configuration management use cases surrounding these models do provide a compelling reason to enable all of the extensibility features, plus the basic fact that the reason we wanted an explicit through model is because we need more context to be a part of the M2M association.
Unfortunately, making custom fields work in 1.6 is going to be a non-starter because it would entail a schema migration. We can however, look to do this in the mainline 2.X releases.
That said, we will continue to treat this issue as a bug, in that you should not see this model as an option in 1.6.

implemented for 1.6 by #4878.


System tray title can be too long on Windows

The system tray title can be too long, causing a ValueError: string too long (x, maximum length 128).
Code Reference
https://github.com/Windows200000/TwitchDropsMiner-updated/blob/c9cd4adadaf0357567f4d10885cd05421560ecad/gui.py#L1048-L1057
Suggested Solution
Truncate the title to 128 characters. Possible remove self.TITLE (TITLE = "Twitch Drops Miner") from the title?
Or add an try-except block to catch the error and just skip updating the title.
Example
Example system tray title (This is for a working drop, I have no idea what the title is for the drop that causes the error):
Twitch Drop Miner
Halo Infinite
Adrenal M40 AR 6.1% (0/3)

Stack Trace

Stack trace (click to expand)

10:36:26: Fatal error encountered:
10:36:26:
10:36:26: Traceback (most recent call last):
10:36:26:   File "C:\Code\TwitchDropsMiner_2024-5-fix\main.py", line 158, in main
10:36:26:     await client.run()
10:36:26:   File "C:\Code\TwitchDropsMiner_2024-5-fix\twitch.py", line 769, in run
10:36:26:     await self._run()
10:36:26:   File "C:\Code\TwitchDropsMiner_2024-5-fix\twitch.py", line 982, in _run
10:36:26:     active_drop.display(countdown=False, subone=True)
10:36:26:   File "C:\Code\TwitchDropsMiner_2024-5-fix\inventory.py", line 242, in display
10:36:26:     self._manager.display_drop(self, countdown=countdown, subone=subone)
10:36:26:   File "C:\Code\TwitchDropsMiner_2024-5-fix\gui.py", line 2149, in display_drop
10:36:26:     self.tray.update_title(drop)  # tray
10:36:26:     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
10:36:26:   File "C:\Code\TwitchDropsMiner_2024-5-fix\gui.py", line 1115, in update_title
10:36:26:     self.icon.title = self.get_title(drop)
10:36:26:     ^^^^^^^^^^^^^^^
10:36:26:   File "C:\Code\TwitchDropsMiner_2024-5-fix\env\Lib\site-packages\pystray\_base.py", line 150, in title
10:36:26:     self._update_title()
10:36:26:   File "C:\Code\TwitchDropsMiner_2024-5-fix\env\Lib\site-packages\pystray\_win32.py", line 81, in _update_title
10:36:26:     self._message(
10:36:26:   File "C:\Code\TwitchDropsMiner_2024-5-fix\env\Lib\site-packages\pystray\_win32.py", line 337, in _message
10:36:26:     win32.Shell_NotifyIcon(code, win32.NOTIFYICONDATAW(
10:36:26:                                  ^^^^^^^^^^^^^^^^^^^^^^
10:36:26: ValueError: string too long (131, maximum length 128)
10:36:26:
10:36:26: Exiting...
10:36:26:
10:36:26: Application Terminated.
10:36:26: Close the window to exit the application.

Additional Information
@woctezuma made an issue about this in the original repository:
https://github.com/DevilXD/TwitchDropsMiner/issues/460

Response from DevilXD (click to expand)

The error going away doesn't resolve the underlying issue, which in this case appears to be
a 128 characters limit on a tray icon title. This can apparently be exceeded, if a drop
that's currently being mined has a long name, or belongs to a game that has a long name,
or a combination of these two.

The simplest solution would be to just always trim the tray icon title down to <128 characters,
but that'd just make it ugly and weirdly cut-off in these cases. A proper solution would be to always
leave the main title and progress percentage up, while only trimming the drop's name.
If that's isn't enough, then trim the game's name as well. Maybe the application title could be removed as well,
solely to allow for more characters to be displayed. I'll try to put out a patch for this soon.

Truncate the title to 128 characters. Possible remove self.TITLE (TITLE = "Twitch Drops Miner") from the title?
Or add an try-except block to catch the error and just skip updating the title.

As @DevilXD said, the drop name should be trimmed. Should be fairly easy to do.

Fixed in https://github.com/Windows200000/TwitchDropsMiner-updated/commit/1c1a950ab4f5f06e79e71e58efae9f6f58e9cb88
https://github.com/Windows200000/TwitchDropsMiner-updated/blob/1c1a950ab4f5f06e79e71e58efae9f6f58e9cb88/gui.py#L1048-L1079

2024-06-01 16:31:06: Progress: 1/240 - Campaign(Vampire: The Masquerade - Bloodhunt, Bloodhunt June Drops, 0/1)
len after: 128
Twitch Drops Miner
Vampire: The Masquerade - Bloodhunt
Kill Time, Scribbled Away, Unworship, Burnout, Milk, Swift Vi… 0.4% (0/1)

Thanks, it works

2024-06-01 16:31:06: Progress: 1/240 - Campaign(Vampire: The Masquerade - Bloodhunt, Bloodhunt June Drops, 0/1)
len after: 128
Twitch Drops Miner
Vampire: The Masquerade - Bloodhunt
Kill Time, Scribbled Away, Unworship, Burnout, Milk, Swift Vi… 0.4% (0/1)

Thanks, it works

Interesting, well first of all I forgot to remove the print statement, but also it allows 128 but only prints 127. Will fix that.

Fixed in v15.6.0


Invalid "N more..." label

Hi,
I am facing this issue, where a day on calendar says "3 more..." where as actually that day only contains 2 events. Attaching SS here.

Also attaching sample dashboard JSON (convert .txt to .json).
Calendar Issue for ...more rows-1671445911766.txt
Plugin version: v1.0.0, v1.1.0

@yusuf-multhan Thank you for your contribution.
Please let us know if there is anything else.

You can download the artifact from the CI if you need it: https://github.com/VolkovLabs/volkovlabs-calendar-panel/actions/runs/3826757870

@yusuf-multhan I looked closer at this issue, and I understand the root cause why you see more events than displayed. It's a fillers for multi-day events. See on my screenshot.

We can't just filter out undefined events, because multi-days lines will be broken.
I am looking how Google Calendar implemented it and they have similar concept of empty spaces. Except, that next week have to realign, but we align elements for the whole range.
It is something to improve in future versions, but for now I keep undefined events.
Let me know if it's clear. I am planning to release a new version later today and you are welcome to test it to see how it works for your use case.


[Don't merge] Use MD instead of RST for the docs

because it makes writing docs more fun for me and it's consistent with the daily GitHub workflow.
I have no intention to break RTD the docs support, but soon I want a github.io page for the widgets which will be the "prefered" rendering location of the docs.  This will allow @SylvainCorlay and I to have control over rendering widget demos in the docs.
@SylvainCorlay in a later PR I think I'm going to try http://bruth.github.io/jekyll-docs-template/ to create a simple GitHub.io page, what do you think?

@jdfreder this is awesome! Thanks for tackling the github pages!
I like the jekyll docs examples. Which part of the pages would you like to use it for?

You can use markdown on RTD, and I think it makes sense to consolidate the docs in the same place as much as possible.
I really don't like including docs builds in the same repo as the code, since it adds a huge amount of unused bloat to clones. So if we decide gh-pages is preferable, it should be a dedicated repo.

I really don't like including docs builds in the same repo as the code, since it adds a huge amount of unused size to clones. So if we decide gh-pages is preferable, it should be a dedicated repo.

Agreed, I'm happy to go ahead and remove it in this PR.  I've been playing with http://ipython.org/ipywidgets/ (the gh-pages branch) and it seems like this will work well.  I'm a little worried about diverging from RTD because IPython lives there...  Maybe I'll try to figure out a way to redirect to http://ipython.org/ipywidgets/ nicely.
Thoughts?

Shoot, with jekyll + github.io, I'm not quite sure how to handle multiple versions of the docs.  I think we at the very least need master + stable.

We just had a quick chat about this with @jdfreder .
Regarding the choice of Jekyll, another popular static site generator is Pelican (which is Python-based instead of Ruby-based). We might be more comfortable troubleshooting it since it is part of our ecosystem.
I saw that @rossant was using Pelican for his personal blog and has some blog entries on how to use it with Github pages.
@rossant, do you think that Pelican is a good fit for narrative documentation of a project? We will also embed some JavaScript in the page to create live interactive widgets in the examples...

And what would be the benefits of those over using markdown in Sphinx? Which gets us automatic builds for all versions no hosting, no repos to deal with, common build tools, etc.?
If the goal is just to use markdown as the source format, I'm all for it, but there are a lot of benefits of sphinx+rtd that we don't have to lose if that's the only motivator.

For reference, the JupyterHub docs are markdown for the most part, but since it's sphinx I can still use a couple of rst files for the autodoc API bits that don't need much manual writing.

cc @willingc

Do you know http://www.mkdocs.org/ ?
Pelican is good but might be overkill for documentation. I would favor sphinx, mkdocs or another documentation library over pelican if possible.

@jdfreder @SylvainCorlay @minrk Let's begin discussing this tomorrow. I'm fine with Markdown as a source. I'm not ok, yet, with moving all ipywidgets docs off of RTD. While I think we should explore ideas for the best way to document the widgets, I'm -1 on having to maintain different documentation workflows for each project.
For now, let's begin determining what is needed for documenting the content, and worry less about whether to host on github.io/jekyll. Pelican, while I like it, would add significant overhead and maintenance of the docs.
While I see @minrk's point on docs adding to the github repo size, I think decoupling the doc content source from the repo would be a premature action and likely result in the docs falling out of date given the number of repos that we currently have.
Ideally, I would like to see us be able to support rst, md, ipynb, some of the cool js doc tools on a global Jupyter scope.

I would fully support helping with development of @rossant's @ipymd to support the current notebook version since it would give us better support of notebooks within the sphinx docs.

@minrk in response to your question
Benifits:

Live unconnected widget support
Control over page layout

Cons:

Multiple versions are hard
Automating doc releases isn't clear
Not consistent with other RTD docs

Whether we continue to deploy on RTD or in a github.io page, we still have the problem that this repo really hosts 3 projects, two of which need serious narrative doc content and will be targeted to mostly entirely different audiences.

@rossant I didn't know about mkdocs, but it looks great.  I particularly like this theme: https://github.com/chrissimpkins/cinder

@jdfreder mkdocs is cool. @minrk and I looked at it when we were thinking about jupyterhub docs. Yet, I'm not yet convinced that it provides more than Sphinx/RTD. I've been playing around with nicer themes than the RTD theme for our RTD docs.

I think you can use Markdown in RTD via mkdocs:
http://read-the-docs.readthedocs.org/en/latest/getting_started.html#in-markdown
http://read-the-docs.readthedocs.org/en/latest/builds.html?highlight=mkdocs#mkdocs

Thanks @rossant !  If we roll our own theme, even if it's based off of an existing theme, it looks like we may be able to inject some Javascript on RTD.  The ability to put Javascript on the page is the big motivator of this discussion

because it allows us to put widgets on the page

which will allow us to demo things
and showcase cool examples, like bqplot
and show dynamic layout traits of the widgets

and it allows the user to play with the APIs on the page

I totally agree that moving off of RTD has a cost, both in reader experience because of consistency and project doc contributor workflow.

@jdfreder Sounds like we are in sync. Havingmyou and @SylvainCorlay working through this with me will likely improve not just the ipywidget docs but all of the Jupyter docs. I have some ideas which I think with your knowledge will help us get there sooner :sunrise:
@rossant Thanks for sharing the links re: mkdocs. You can also use markdown in RTD if you use recommonmark with sphinx. https://github.com/jupyter/jupyterhub/blob/master/docs/source/conf.py

Okay, I've been able to get CDN content and embeded Javascript working.  Relative imports I don't have working yet.

@jdfreder There's a new version of Sphinx released today. You might want to see if that helps.

In any case, the version of sphinx that comes in anaconda is a bit outdated at the moment

So I've made a lot of progress here.  It's not quite as pretty as the jekyll-docs-template, but I was able to get local Javascript on the page and automagically replace .MD links with .HTML so that the docs will render on Github and on RTD as fancy HTML.  I'm working on cleaning things up aesthetically, then I'm going to work towards getting the widget.js in particular on the page.

Here's a work in progress preview, which will update as I go: http://jons-widgets-fork.readthedocs.org/en/nicedocs/index.html

@jdfreder Nice!
Here's a sample (very rough) of another doc theme (sphinx_bootstrap_theme) http://test-notebook.readthedocs.org/en/sphinx-alpha/

Okay, I'm going to stop fiddling with the style a bit.  I'll start working on getting the jupyter-js-widgets javascript on the page tomorrow...

@jdfreder You might find the theme preview on this helpful: https://pythonhosted.org/sphinxjp.themes.basicstrap/index.html Here's the repo: https://github.com/tell-k/sphinxjp.themes.basicstrap

ooh I like that one @willingc !  I ended up pulling bootstrap into the one I have applied now.
I'm currently trying to figure out the mechanics of sphinx with Markdown.  Specifically TOC and content structuring which Markdown lacks.  Also, still trying to figure out the best way to pull node content into the mix automagically.   I think a small Python script called by conf.py may do the trick...

@jdfreder You can do TOC easily with RST. The rest of the content structuring we should be fine with markdown. Perhaps an addition to conf.py will work. Let's talk tomorrow/later. See the jupyterhub docs for the combination of RST for TOC and md for the rest.

@minrk , @willingc filled me in on the JupyterHub docs.  Tomorrow we are going to work together to incorporate some of the stuff from there in addition to getting, building, and hosting the widget static JS and an example of a live widget.
We are also trying to figure out how/if notebook files play a role in this story.  Maybe docs -> notebooks, maybe notebooks -> docs, maybe notebooks !== docs, it's not clear.
We are also keeping our eyes open for clean, clear, crisp, easy on the eyes themes - but that's lower priority because we can change that fairly easily later down the line.

Sounds great!

@jdfreder We're going to want to pin the sphinx dependency to 1.3.6+ or the "Edit on GitHub" link in the upper right will 404. Not sure that the feature is still in our custom template. Yet, the link failure was a Sphinx bug fixed in the 1.4alpha and backported to 1.3.6.

Thanks @willingc !  I think that is actually not a part of our template.  IIRC it would need to be custom anyways because we are using MD and ipynb source files.

@SylvainCorlay here I moved the embed manager into jupyter-js-widgets and updated the docs to use that.  I also added an embed helper to generate the text/html data for given widget state.

@jdfreder I think that the moved embed widget manager has the same issue as before, which is that you display all widgets top-level. Instead, we should use the views list which contains the root-level list of views, like in the web2 and web4 examples.

We could move the web4 widget manager in jupyter-js-widgets and make jupyter-js-widgets export the w function.

@SylvainCorlay It doesn't make sense to export the w function without the manager because the manager has to be on the page for the w function to work.

@SylvainCorlay It doesn't make sense to export the w function without the manager because the manager has to be on the page for the w function to work.

Yes, I meant also export the w function.

@jdfreder do you have a minute to appear.in?

Okay @willingc and @SylvainCorlay , this is good to go now.  I'd like to follow it up with another PR where I add new content.  In its current state this PR uses the new embed widget manager, to allow the embedding of widgets into the docs, it also uses the new jupyter-sphinx-theme for both an improved style and the ability to integrate Javascript.

Arg, this comes with all the commits from 6.x.

Arg!  Good catch, I must have rebased against the 6.x branch instead of current master.

Note: I noticed that you are using ES6's const keyword in the embed manager.

Did you get a chance to do a rebase?

I totally did, I'm having some really weird GH problems.  The 6.x commits shown here don't show up in my local git log!!

I totally did, I'm having some really weird GH problems. The 6.x commits shown here don't show up in my local git log!!

Did you reset your local master to upstream/master?

I did, I'm trying another machine now...

Okay, it took some magic but I was able to repair the branch.  Before we merge, I'm going to squash some of these commits, there are too many!!!

So we are not using embedded widget yet right?

@SylvainCorlay okay, everything is squashed and rebased.  To answer your question, we are including it, but not using it yet in this PR.   I removed the code in the page that was using it because it shouldn't have belonged there.

btw, I pushed to master the use of addEventListener instead of onload.

btw, I pushed to master the use of addEventListener instead of onload.

You should not push to master for things like that, only bumping versions for releases.  It helps me when you open PRs, even if super small, because then I can be notified of your changes and keep track of them for my own info.

btw, I pushed to master the use of addEventListener instead of onload.

You should not push to master for things like that, only for bumping versions for releases. It helps me when you open PRs, even if super small, because then I can be notified of your changes and keep track of them for my own info.

Sorry, it was outstanding from #450 as per comments.

I think that it is worth merging now. It is going to be a pain to rebase again...

:shipit: :+1: :tada: :racehorse:

Yay!!!!! :cake: for @jdfreder, @SylvainCorlay, and the team


Why do volatile operations not suffice?

[std::ptr::write_volatile](file:///usr/local/share/doc/rust/html/std/ptr/fn.write_volatile.html) says "Volatile operations are intended to act on I/O memory, and are guaranteed to not be elided or reordered by the compiler across other volatile operations."  I just used those when I attempted to do do something similar, although lacking your nice stack trick.

I tried, but the code generated seems to be quite bad (see https://github.com/rust-lang/rust/issues/39001 which I found while writing the stack clearing code). Allowing the compiler to do its normal memset, followed by an empty inline assembly with the right inputs/outputs (or a call to a foreign function) to make it believe the results of the memset (and the memory address itself) are in use, seems to work much better. Once I had the "hide" trick working for the stack clearing, it was a no-brainer to also use it for ClearOnDrop and to prevent inlining.
Also, my main objective with this crate was to show an alternative solution. Others were already using write_volatile, and I had already done inline assembly tricks to fool the optimizer in a different context (the OPTIMIZER_HIDE_VAR Linux kernel macro, mentioned on the hide.rs comment, was my idea), so I wanted to go that way and see what would happen. The names came from an old Rust RFC of mine (https://github.com/rust-lang/rfcs/pull/1496).
I believe that the main problem with write_volatile (and read_volatile) is that it's designed for I/O. When writing an array of sixteen u8, it will do sixteen byte-sized writes, instead of a few larger writes or a memset. This makes sense for I/O memory (where access size and order is important, and the tricks memset might do can cause problems), but not so much for normal cacheable memory.


Tiff image

Hi there,
Thanks for uploading this good work. I downloaded the toy tiff image using the posted command line. But it seems the tiff image is all in black. Any tips?
suffix=images/2020_11_04_CPJUMP1/images/BR00117010__2020-11-08T18_18_00-Measurement1/Images/r01c01f01p01-ch1sk1fk1fl1.tiff

aws s3 cp \
  --no-sign-request \
  s3://cellpainting-gallery/cpg0000-jump-pilot/source_4/${suffix} \

.

Never mind. It seems this example is only black&white.


deployment with dynamic number of nic (range from 1 to 4)

allow for a vnet deployment with a dynamic set of subnet ranging from 1 (minimum requirement) to 4
NOTE: This feature alters the original network and load balancing design - the external load balancer routes traffic to all 4 subnets - it now only routes incoming traffic from the external load balancer to the subnet 1. None of the extra subnets (subnet 2 to subnet 4) will receive traffic from the external load balancer.

feature will be delivered in #14


[1571] Split course_enrichment validations

Context
We need to add validations for saving enrichments and for publishing them.
First PR of a few.
Changes proposed in this pull request
Splits up the course_enrichment validations into two scopes: :publish which is same as before, and :save which skips the presence validations.
Also some miscellaneous refactors.
Guidance to review
Best reviewed commit by commit.
Checklist

[x] Make sure all information from the Trello card is in here
[ ] Attach to Trello card
[x] Rebased master
[x] Cleaned commit history
[x] Tested by running locally

 Add validations to course update action


GC error, possibly related to fill operations

The recently-added 'mssp' Regent mini-app works ok as it is checked in, but if you apply the following
patch:
--- a/language/examples/mssp/mssp_base.rg
+++ b/language/examples/mssp/mssp_base.rg
@@ -66,3 +66,3 @@ do
   -- fill called by parent
-  --fill(rn.distance, 1e100) -- == infinity for floats
+  fill(rn.distance, 1e100) -- == infinity for floats
   root.distance = 0
@@ -158,3 +158,3 @@ task toplevel()
   for i = 0, graph.num_sources do
-    fill(rn.distance, 1e100) -- == infinity for floats
+    --fill(rn.distance, 1e100) -- == infinity for floats

and then run it like so:
(build Regent)
cd language/examples/mssp
../../regent.py ./mssp_base.rg small

you get:
graph metadata: nodes=10, edges=20, sources=1
[0 - 7f7590531700] {4}{tasks}: WARNING: REGION REQUIREMENT 0 OF TASK read_edge_data (ID 5) HAS NO PRIVILEGE FIELDS! DID YOU FORGET THEM?!?
terra: /local/home/sean/s3dfoo/legion/runtime/legion/garbage_collection.cc:89: void LegionRuntime::HighLevel::DistributedCollectable::add_gc_reference(unsigned int): Assertion `current_state != DELETED_STATE' failed.

Assigning to Mike for initial debug.

Fixed with https://github.com/StanfordLegion/legion/commit/1b75ec7ff283169b84aec6308bd07999d6171be7


SELinux apply doesn't work on Arch Linux WSL

Currently I'm using Arch Linux for WSL(2) using this distro: https://github.com/yuk7/ArchWSL
Seems like it's same as #66.
Logs: https://pastebin.com/7fTy0K0R
chcon: failed to get security context of '(directories)': Operation not supported

Install Ubuntu or another Debian based distro and try from that

For Arch, the solution is to install some SELinux packages:

add the following lines to your /etc/pacman.conf:

[selinux]
Server = https://github.com/archlinuxhardened/selinux/releases/download/ArchLinux-SELinux
SigLevel = Never

run pacman -Sy coreutils-selinux shadow-selinux cronie-selinux sudo-selinux
⚠ !!! After this, sudoer files may be reset. So make sure you can login as root before install.

Try build images again.

For Arch, the solution is to install some SELinux packages:

add the following lines to your /etc/pacman.conf:

[selinux]
Server = https://github.com/archlinuxhardened/selinux/releases/download/ArchLinux-SELinux
SigLevel = Never

run pacman -Sy coreutils-selinux shadow-selinux cronie-selinux sudo-selinux
⚠ !!! After this, sudoer files may be reset. So make sure you can login as root before install.
For WSL, you can login as root by wsl --user root
Try build images again.

I need someone to validate it. If it works, consider contribute to README.md

is work


AmbiverseNLU returning 404 errror after build

After a successful build application returning 404 error:
HTTP ERROR 404
Problem accessing /. Reason:

    Not Found
Powered by Jetty:// 9.4.4.v20170414

The log file is attached and it does not show errors.
ambiversenlu.log

How does your service call look like?
It should be something like this:
url --request POST \
  --url http://localhost:8080/entitylinking/analyze \
  --header 'accept: application/json' \
  --header 'content-type: application/json' \
  --data '{"docId": "doc1", "text": "Jack founded Alibaba with investments from SoftBank and Goldman.", "extractConcepts": "true" }'

There is nothign on localhost:8080 if this is what you are trying to access. The valid path is http://localhost:8080/entitylinking/analyze

Thank you,
It works from the command line but I thought that in addition to that application also has a web interface like one that is shown on the demo page https://ambiversenlu.mpi-inf.mpg.de/
Does this application have only a command line interface?

The system has two ways to run. Command Line described here and with a webservice call, described here. The one that you are referring is a webservice call. The interface shown on the demo page is available in another repository as a wordpress plugin. This plugin for now connects to the public API of our web service. If you want to use it, you probably have to change the source code to connect to your service.


WOQ is not giving any performance speedup in whisper

System Info

CPU: x86_64
CPU memory size: 96GB
GPU: A100
Tensorrt tag: latest one(https://github.com/NVIDIA/TensorRT-LLM/commit/5d8ca2faf74c494f220c8f71130340b513eea9a9)
TensorRT version: build from source for cc: 80-real
Nvidia-driver version: 525.125.06
OS: Debian 4.19.304-1 (2024-01-09) x86_64 GNU/Linux

Who can help?
@kaiyux
Information

[X] The official example scripts
[ ] My own modified scripts

Tasks

[X] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
[ ] My own task or dataset (give details below)

Reproduction
To reproduce this issue, build the Whisper medium model in two versions: one as normal and the other using weight-only quantization (WOQ).
Expected Behavior
The version built with WOQ is expected to have better performance.
actual behavior
********
engine_dir: /app/tensorrt_llm/examples/ts-whisper/ts_whisper/benchmark/whisper_models/whisper_medium_bs10_bw5_FP16
RTF: 0.0174
total_duration: 481.035 seconds
(0.13 hours)
processing time: 8.357 seconds (0.00 hours)
batch size: 10
num_beams: 5
total error rate: 4.62%
********
engine_dir: /app/tensorrt_llm/examples/ts-whisper/ts_whisper/benchmark/whisper_models/whisper_medium_woq_bs10_bw5_FP16
RTF: 0.0174
total_duration: 481.035 seconds
(0.13 hours)
processing time: 8.365 seconds (0.00 hours)
batch size: 10
num_beams: 5
total error rate: 4.70%
********

Additional Notes
I think this issue may be related to GPU architecture. Is there another method for using quantization with the Whisper model?

System Info

CPU: x86_64
CPU memory size: 96GB
GPU: A100
Tensorrt tag: latest one(5d8ca2f)
TensorRT version: build from source for cc: 80-real
Nvidia-driver version: 525.125.06
OS: Debian 4.19.304-1 (2024-01-09) x86_64 GNU/Linux

Who can help?
@kaiyux
Information

[x]  The official example scripts
[ ]  My own modified scripts

Tasks

[x]  An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
[ ]  My own task or dataset (give details below)

Reproduction
To reproduce this issue, build the Whisper medium model in two versions: one as normal and the other using weight-only quantization (WOQ).
Expected Behavior
The version built with WOQ is expected to have better performance.
actual behavior
********
engine_dir: /app/tensorrt_llm/examples/ts-whisper/ts_whisper/benchmark/whisper_models/whisper_medium_bs10_bw5_FP16
RTF: 0.0174
total_duration: 481.035 seconds
(0.13 hours)
processing time: 8.357 seconds (0.00 hours)
batch size: 10
num_beams: 5
total error rate: 4.62%
********
engine_dir: /app/tensorrt_llm/examples/ts-whisper/ts_whisper/benchmark/whisper_models/whisper_medium_woq_bs10_bw5_FP16
RTF: 0.0174
total_duration: 481.035 seconds
(0.13 hours)
processing time: 8.365 seconds (0.00 hours)
batch size: 10
num_beams: 5
total error rate: 4.70%
********

Additional Notes
I think this issue may be related to GPU architecture. Is there another method for using quantization with the Whisper model?

Yes, WOQ int8 cannot significantly improve inference speed, but it can reduce memory usage while maintaining the same speed. For example, the memory usage of Whisper Large can be reduced by 1.5 GB. In the future, we will support the FP8 quantization scheme, which can improve inference speed. For the Ampere architecture, SmoothQuant int8 can also speed up inference, but its support priority will be after FP8.
Additionally, you can try the inference of Whisper Large v3. On the A100, it has better accuracy and is not much slower than the Medium model.

@yuekaizhang Thanks for the answer, but please check the following information:
**********************************************************
running benchmark for whisper_medium_bs10_bw5_FP16
**********************************************************
engine_dir: /app/tensorrt_llm/examples/ts-whisper/ts_whisper/benchmark/whisper_models/whisper_medium_bs10_bw5_FP16
RTF: 0.0181
total_duration: 481.035 seconds
(0.13 hours)
processing time: 8.702 seconds (0.00 hours)
batch size: 10
num_beams: 5
total error rate: 4.62%
total 1.6G
-rw-r--r-- 1 root root 1.7K May 22 20:06 decoder_config.json
-rw-r--r-- 1 root root 1.4K May 22 20:06 encoder_config.json
-rw-r--r-- 1 root root 978M May 22 20:06 whisper_decoder_float16_tp1_rank0.engine
-rw-r--r-- 1 root root 589M May 22 20:06 whisper_encoder_float16_tp1_rank0.engine
**********************************************************
running benchmark for whisper_medium_woq_bs10_bw5_FP16
**********************************************************
engine_dir: /app/tensorrt_llm/examples/ts-whisper/ts_whisper/benchmark/whisper_models/whisper_medium_woq_bs10_bw5_FP16
RTF: 0.0178
total_duration: 481.035 seconds
(0.13 hours)
processing time: 8.539 seconds (0.00 hours)
batch size: 10
num_beams: 5
total error rate: 4.70%
total 897M
-rw-r--r-- 1 root root 1.7K May 22 20:16 decoder_config.json
-rw-r--r-- 1 root root 1.4K May 22 20:16 encoder_config.json
-rw-r--r-- 1 root root 595M May 22 20:16 whisper_decoder_float16_tp1_rank0.engine
-rw-r--r-- 1 root root 302M May 22 20:16 whisper_encoder_float16_tp1_rank0.engine

Based on the above information that the WOQ is using less disk space, I can be sure that it is indeed using WOQ. However, please check the memory consumption.

processing time: 8.697 seconds (0.00 hours)
batch size: 10
num_beams: 5
total error rate: 4.70%

@robosina Hi, thanks for your investigation. I have done some perf jobs. And indeed, found that whisper encoder fp16 is faster than int8 WOQ.  We would work on the issue and update here.
For Whisper Decoder, WOQ int8 should be faster than fp16.
For the VRAM usage, whisper medium has 0.7B parameters. In this way, WOQ int8 would decrease about 700M VRAM, and WOQ int4 would decrease another 350M VRAM usage.

Hi @robosina do u still have further issue or question now? If not, we'll close it soon.

Hi @nv-guomingz, Yes, this issue has been resolved in the recent releases, so I will do it myself, thanks for the support.


recv returns trash data if eth frame was expanded

If ethernet frame was less than minimum ethernet frame size and was extended, recv will return trash data.
In linux ethernet frames often bigger that minimum, as linux use timestamps by default. But in Windows packets often comes without timestamps, and could be situation when frame should be extended. For example, if someone decided to send 1byte (telnet?), we will have 14(eth)+20(ip)+20(tcp)+1(payload) = 55 bytes frame. It will be expanded, and ofp will receive 60bytes frame. And than ofp will just drop headers and recv will return rest of packet.

As possible solution:
diff --git a/src/ofp_tcp_input.c b/src/ofp_tcp_input.c
index 79466bb..26c648a 100644
--- a/src/ofp_tcp_input.c
+++ b/src/ofp_tcp_input.c
@@ -1733,6 +1733,11 @@ ofp_tcp_do_segment(odp_packet_t m, struct ofp_tcphdr *th, struct socket *so,
                                            newsize, so, NULL))
                                                so->so_rcv.sb_flags &= ~SB_AUTOSIZE;
                                odp_packet_pull_head(m, drop_hdrlen);   /* delayed header drop */
+
+                               todrop = odp_packet_len(m) - tlen;
+                               if (todrop > 0)
+                                       odp_packet_pull_tail(m, todrop);
+
                                ofp_sbappendstream_locked(&so->so_rcv, m);
                        }
                        /* NB: sorwakeup_locked() does an implicit unlock. */
@@ -2787,6 +2792,11 @@ dodata:                                                  /* XXX */
            TCPS_HAVERCVDFIN(tp->t_state) == 0) {
                tcp_seq save_start = th->th_seq;
                odp_packet_pull_head(m, drop_hdrlen);   /* delayed header drop */
+
+               todrop = odp_packet_len(m) - tlen;
+               if (todrop > 0)
+                       odp_packet_pull_tail(m, todrop);
+
                /*
                 * Insert segment which includes th into TCP reassembly queue
                 * with control block tp.  Set thflags to whether reassembly now

Hi,
Anyone who wants to contribute to the OpenFastPath Foundation needs to sign the contribution agreement. http://www.openfastpath.org/downloads/OpenFastPath_ContributionAgreement.pdf
See http://www.openfastpath.org/index.php/about-us/
To submit patches see:
http://www.openfastpath.org/index.php/service/getinvolved/
Mailing List and Patches Chapters.
BR,
Sorin
OpenFastPath Maintainer


[dapr] Event Hubs input binding crashes dapr container

I'm trying to define a dapr Event Hubs input binding for my Container App.
Here's what i came up with -
componentType: bindings.azure.eventhubs
version: v1
metadata:
  - name: connectionString.     # My Event Hub connection string
    value: Endpoint=sb://iothub-ns-poorlyfund-2415[.....]=;EntityPath=poorlyfundedskynet
  - name: storageAccountName    # Azure Storage Account Name
    value: "SomeStorageAccountNameHere"
  - name: storageAccountKey     # Azure Storage Account Key
    value: "nrkA[....]t2ahsCg=="
  - name: storageContainerName  # Azure Storage Container Name
    value: "SomeContainerNameHere"
scopes:
  - dapr-eventhub.   # the name of my Container App

Initially, i did not have storageAccountName and storageAccountKey in, and that made the sidecar container fail to start with storageAccountName is required, so i got the hint.
Now, the dapr sidecar container just crashes on startup and the system logs don't give me anything useful.
Is there an obvious mistake i'm making in the YAML definition?
This is what i use to apply it -
az containerapp env dapr-component set \
           --name env-dapr-eventhub \
           --resource-group dapr-eventhub \
           --dapr-component-name eventhub-input-binding \
           --yaml components/eventhubs_binding.yaml

Adding consumerGroup fixed the container startup problem, but now i'm simply not getting anything in the request body -
2023-01-27T10:51:43.30244  Connecting to the container 'dapr-eventhub'...
2023-01-27T10:51:43.32251  Successfully Connected to container: 'dapr-eventhub' [Revision: 'dapr-eventhub--new', Replica: 'dapr-eventhub--new-7b547d5d6c-4w8zl']
2023-01-27T10:50:26.146309473Z req.Body is: {}

func main() {
	http.HandleFunc("/eventhub-input-binding", func(rw http.ResponseWriter, req *http.Request) {
		var _time TheTime
        fmt.Printf("req.Body is: %s\n", req.Body)
        ...

@snobu  I believe that you are missing couple of important configurations for this binding component.
Please see all required fields here https://docs.dapr.io/reference/components-reference/supported-bindings/eventhubs/
I can quickly see that eventHub and eventHubNamespace are missing, that may be a reason for events are being subscripted

@BlackRider97 In the docs, it says eventHubNamespace and connectionString mutually exclude.


PowerToys Run doesn't reload environment variables.

ℹ Computer information

PowerToys version: 0.31.2
PowerToy Utility: PowerToys Run
Running PowerToys as Admin: no
Windows build number: 19042.804

📝 Provide detailed reproduction steps (if any)

Open PowerToys.
Modify environment variables, e.g PATH, from windows settings.
Open pwsh code or anything else.
echo $env:PATH.

✔ Expected result
The environment variables should be what you have modified.
❌ Actual result
The environment variables remain what they were when PowerToys was started.
📷 Screenshots
This is a simple scenario, I think no screenshot is needed.

Good catch! Deactivate and activate the module should fix this

I think it's a problem that opend apps get the environment (variables) from PT Run.

Duplicate of https://github.com/microsoft/PowerToys/issues/6847


ErrorException: file_put_contents(/meta/services.json): failed to open stream: No such file or directory

I'm a novice developer trying to get phpunit set up in an already developed app(Laravel 4.2) but I get the following error on all my tests when I try to run phpunit:
ErrorException: file_put_contents(/meta/services.json): failed to open stream: No such file or directory
I did not develop the app or set any of its configurations and I'm woefully inexperienced so I don't know where to go with this. I DO have a services.json file in app/storage/meta but phpunit seems to be looking for it in the root directory and I have no idea why or how to point it to the right place.
When tracing through the error(in particular, the compiled.php file), it looks like the error is coming from the writing the "manifest", specifically when creating the manifest path. However, I'm pretty lost as to what this implies or how to fix it.

Sorry, Laravel 4.2 is not supported.