fiona.transform.transform - different behavior based on GDAL version

fiona.transform.transform exhibits different behavior based on the underlying gdal library installed.
Expected behavior and actual behavior.
If I run fiona.transform.transform("epsg:4326", "epsg:3857", [-75.71], [38.06]) I expect to get: ([-8427998.647958742], [4587905.271362515]) regardless of library version, however with gdal 3.0.2 and fiona 1.8.13.post1 I get the incorrect result.
In particular, the xs, ys position are swapped in the arguments, i.e. if I run fiona.transform.transform("epsg:4326", "epsg:3857", [38.06], [-75.71]) then I do get the correct result.
@snowman2 suggested that this could be related to a PYPROJ issue, https://github.com/pyproj4/pyproj/issues/538 (see https://fiona.groups.io/g/main/topic/75506516#60 for comments).
Steps to reproduce the problem.
conda create --name test python gdal fiona # this installs gdal 3.0.2 and fiona 1.8.13.post1
conda activate test
python
>>> import fiona.transform
>>> fiona.transform.transform("epsg:4326", "epsg:3857", [-75.71], [38.06])
([4236819.819591993], [-13244913.18166699])

If I switch the lat/lon position then I get the correct result (http://epsg.io/transform#s_srs=4326&t_srs=3857&x=-75.7169053&y=38.0624956):
>>> fiona.transform.transform("epsg:4326", "epsg:3857", [38.06], [-75.71])
([-8427998.647958742], [4587905.271362515])

If I instead do the conda install with a gdal version less than 3 I get the expected result:
conda create --name test2 python "gdal<3" fiona # this installs gdal 2.3.3 and fiona 1.8.4
conda activate test2
python
>>> fiona.transform.transform("epsg:4326", "epsg:3857", [-75.71], [38.06])
([-8427998.647958742], [4587905.27136252])

Operating system
I have tested this with conda on Windows 10 and Ubuntu 18.04.
Fiona and GDAL version and provenance
@snorfalorpagus has reproduced with different versions on Travis - https://travis-ci.org/github/snorfalorpagus/Fiona/builds/708143904.

Thanks for the report @calebrob6 . In another I'm recommending that users switch, if they can, to "OGC:CRS84" with GDAL 3 instead of expecting the old defaults for "EPSG:4326". But that doesn't mean that we won't have to do something about this.

I also observed an odd behavior using fiona.transform.transform_geom() with either providing CRSes as string or as rasterio.crs.CRS:
#!/usr/bin/env python
from fiona.transform import transform_geom
from rasterio.crs import CRS
from shapely import wkt
from shapely.geometry import mapping

example_point = mapping(wkt.loads("POINT<PHONE_NUMBER>578609<PHONE_NUMBER>680807)"))
src_crs = "EPSG:31287"
dst_crs = "OGC:CRS84"

# output should be {'type': 'Point', 'coordinates': (11.868736780148138, 47.29350228983309)}

print(transform_geom(src_crs, dst_crs, example_point))

print(transform_geom(CRS.from_user_input(src_crs).to_dict(), CRS.from_user_input(dst_crs).to_dict(), example_point))

Output using GDAL 2.4.4, PROJ 5.2.0, pyproj 1.9.6 looks fine:
{'type': 'Point', 'coordinates': (11.868736780148138, 47.29350228983309)}
{'type': 'Point', 'coordinates': (11.868736780148138, 47.29350228983309)}

However, the output using GDAL 3.0.4, PROJ 6.3.2 and 7.1.0, pyproj 1.9.6 and 2.6.1.post1 swaps the axis on the fiona output not using rasterio.crs.CRS:
{'type': 'Point', 'coordinates': (13.048008632586706, 46.503227029929064)}
{'type': 'Point', 'coordinates': (11.868736773665933, 47.2935022723449)}

When setting dst_crs to EPSG:4326 on the latter the axes are swapped but still the coordinates are not correct:
{'type': 'Point', 'coordinates': (46.503227029929064, 13.048008632586706)}
{'type': 'Point', 'coordinates': (11.868736773665933, 47.2935022723449)}

I was using rasterio 1.1.4 and Fiona 1.8.13.post1 for this, both built from source.


Require login before seeing any data

Is there a way to make users sign in before they have access to any repositories?
Currently, if I want repositories to be browsable by other users, I make them public but it makes them also available to guests which I don't want to.

Fixed in #591

Great, thanks!


Fix a number of performance inefficiencies

Description
Fixes a number of performance-related issues, namely pass by value that should be const refs, avoiding unnecessary temporaries.
Tests
No further tests added.
Checklist:

[x] I have read the contribution guidelines.
[x] If this is more extensive than a small change to existing code, I
have previously submitted a Contributor License Agreement
(individual, and if there is any way my
employers might think my programming belongs to them, then also
corporate).
[x] I have updated the documentation, if applicable.
[x] I have ensured that the change is tested somewhere in the testsuite
(adding new test cases if necessary).
[x] My code follows the prevailing code style of this project.

Do you have benchmarks to prove that your changes measurably improve performance? Because these are not in fact inefficiencies at all, and the changes in this PR will likely slow things down.
BIG objects or objects that have nontrivial copy constructors should certainly be passed by const reference instead of copying the objects onto the stack, because it's less expensive to push an 8 byte pointer onto the stack rather than, say, the 200 byte structure it points to. But this general rule is not true for small objects that are themselves not much larger than a pointer, and that have trivial constructors.
Passing by value places the object you want right on the stack, in commonly-used memory and sharing a cache line with the other parameters or local variables of the called function. Also, because that makes it local to the function, the compiler knows it can't be modified by other threads or by other functions your function might call (in both cases, because if you don't explicitly pass its address, no other function or thread can possibly know where it is in the local storage of your function's stack frame), and can therefore do a variety of optimizations based on this knowledge. (I can explain examples if you have no idea what I'm talking about.)
On the other hand, anything you pass by reference costs an extra pointer indirection every time you access it, plus the penalty from the fact that the memory being accessed could be anywhere (and in particular not likely near the other things you will be using frequently, so may be a cache miss to access), and also the compiler has potentially no idea what other functions (or concurrent threads) might be modifying the memory your reference points to, and therefore will have to generate much more defensive and inefficient code in order to guarantee correctness.
TypeDesc is 64 bits, by design. Pointers are also 64 bits (on any CPU you would use if you care about performance). So there is no possible savings from passing a pointer to a TypeDesc instead of the TypeDesc itself. In fact, it's surely worse, because you then will have to pay for the extra pointer indirection every time you access it, and all the other problems outlined above.
string_view is a pointer and a length (total: 16 bytes). It, too, should be passed by value. It's true that passing by reference (i.e. a pointer, 8 bytes) saves slightly (pushing an extra 8 bytes on the stack), but you will come out behind because every single time you use it, you will incur an extra memory read because you've inserted an extra level of indirection.
And, sigh, also ustring! A ustring is, underneath, just a pointer to the characters. It's 64 bits. There is, I guarantee, no gain from passing a pointer to the pointer to the characters, instead of just the pointer to the characters, and that's exactly what you have done by changing ustring to const ustring&. You don't pass a reference to a ustring for exactly the same reason you wouldn't pass a reference to a char* -- the pointer itself is already the smallest and least expensive thing you can pass to convey where the characters are, there is nothing to gain by passing a pointer to the pointer.
I'm sorry about this, I feel bad that you made such extensive changes only to get it shot down. It really would have been better if you'd proposed on the mail list what you intended to do, or submitted a "preliminary" PR where you do the change to JUST ONE PLACE and say "hey, do you like this? Because if it's the efficiency gain I suspect, I can do it all over the code base." And with that tiny investment, we could have set you straight before you did any more of the work.

Thanks for the elaborate explanation. This is helpful.


Could not load the "sharp" module using the darwin-arm64 runtime

Describe the bug

When following the quickstart guide, once I get to running a character I get this error
Could not load the "sharp" module using the darwin-arm64 runtime
after running pnpm start --character="characters/trump.character.json"
To Reproduce

On M2 Macbook Air
nvm use 23.3.0
git clone https://github.com/ai16z/eliza.git
cd eliza
git checkout $(git describe --tags --abbrev=0)
pnpm install -w --include=optional sharp
pnpm build
pnpm start --character="characters/trump.character.json"

This produces the error
pnpm start --character="characters/trump.character.json"
 ℹ INFORMATIONS
   Loading embedding settings:
   {"USE_OPENAI_EMBEDDING":"","USE_OLLAMA_EMBEDDING":"","OLLAMA_EMBEDDING_MODEL":"mxbai-embed-large"}

 ℹ INFORMATIONS
   Loading character settings:
   {"ARGV":["/Users/nikita/.nvm/versions/node/v23.3.0/bin/node","/Users/nikita/Documents/chainpatrol/eliza/agent/src/index.ts","--isRoot","--character=characters/trump.character.json"],"CHARACTER_ARG":"--character=characters/trump.character.json","CWD":"/Users/nikita/Documents/chainpatrol/eliza/agent"}

Loaded .env file from: /Users/nikita/Documents/chainpatrol/eliza/.env
 ℹ INFORMATIONS
   Parsed settings:
   {"USE_OPENAI_EMBEDDING":"","USE_OPENAI_EMBEDDING_TYPE":"string","USE_OLLAMA_EMBEDDING":"","USE_OLLAMA_EMBEDDING_TYPE":"string","OLLAMA_EMBEDDING_MODEL":"mxbai-embed-large"}

/Users/nikita/Documents/chainpatrol/eliza/node_modules/sharp/lib/sharp.js:113
  throw new Error(help.join('\n'));
        ^

Error: Could not load the "sharp" module using the darwin-arm64 runtime
Possible solutions:
- Ensure optional dependencies can be installed:
    npm install --include=optional sharp
- Ensure your package manager supports multi-platform installation:
    See https://sharp.pixelplumbing.com/install#cross-platform
- Add platform-specific dependencies:
    npm install --os=darwin --cpu=arm64 sharp
- Consult the installation documentation:
    See https://sharp.pixelplumbing.com/install
    at Object.<anonymous> (/Users/nikita/Documents/chainpatrol/eliza/node_modules/sharp/lib/sharp.js:113:9)
    at Module._compile (node:internal/modules/cjs/loader:1546:14)
    at Object..js (node:internal/modules/cjs/loader:1698:10)
    at Module.load (node:internal/modules/cjs/loader:1303:32)
    at Function._load (node:internal/modules/cjs/loader:1117:12)
    at TracingChannel.traceSync (node:diagnostics_channel:322:14)
    at wrapModuleLoad (node:internal/modules/cjs/loader:218:24)
    at Module.require (node:internal/modules/cjs/loader:1325:12)
    at require (node:internal/modules/helpers:136:16)
    at Object.<anonymous> (/Users/nikita/Documents/chainpatrol/eliza/node_modules/sharp/lib/constructor.js:10:1)

Node.js v23.3.0
/Users/nikita/Documents/chainpatrol/eliza/agent:
 ERR_PNPM_RECURSIVE_RUN_FIRST_FAIL<EMAIL_ADDRESS>start: `node --loader ts-node/esm src/index.ts "--isRoot" "--character=characters/trump.character.json"`
Exit status 1
 ELIFECYCLE  Command failed with exit code 1.
 ```

**Expected behavior**

<!-- A clear and concise description of what you expected to happen. -->

- The character should run and I should be able to speak to it

**Screenshots**

<!-- If applicable, add screenshots to help explain your problem. -->

**Additional context**

<!-- Add any other context about the problem here. -->

same error happens on m1 macbook pro

+1

+1 same error on m1

Okay this took a really long time to solve but after 6 hours, I figured it out.
You cannot use libvips globally, and this is said specifically in the sharp documentation: https://sharp.pixelplumbing.com/install#custom-libvips
The use of a globally-installed libvips is unsupported on Windows and on macOS when running Node.js under Rosetta.
So in order for this to work you need to:

brew uninstall libvips
pnpm install sharp --force -w
pnpm build
pnpm start

I also added in package.json:
"supportedArchitectures": {
"os": [
"darwin",
"current"
],
"cpu": [
"arm64"
]
}
but unclear if this did anything.

Thanks for this 0xSmick, it just works (tm)


Partially migrate another user editing condition from AWX

Let me link the legacy code here:
https://github.com/ansible/awx/blob/918d5b3565e5997bdbd57117b92852ba13bb55d6/awx/main/access.py#L713-L726
When you look at that, first note that there is a condition for "orphan" users --> this means users who are not in any organizations.
The need for this is clear from the test. If a user is not in any organization, then we can't allow any other user to manage that user. That's why we need an extra condition in this case. That's not up for debate.
Here, I'm using is_superuser, because I'm reluctant to copy over all the rules from AWX to DAB. This one, in particular, seems dubious. The reason why the AWX criteria had extra stuff was so that organization admins could POST to /api/v2/users/ and then after that add the user as a member of their organization. I would like to punt this over to require an API solution, so that a user can be given an organization on creation @john-westcott-iv @slemrmartin

^ Ping @chadmf because there is already a sentiment that we need to take holistic view of platform-wide permissions and address the pain points. This ties into the larger UX flow, docs, and user recommendations.


Support iOS Target

@wingio any update on whether this will be added,
If i added this to a KMP app which targets iOS too, i get error when syncing the project.
Execution failed for task ':shared:transformNativeMainCInteropDependenciesMetadataForIde'.
> Could not resolve all files for configuration ':shared:iosArm64CompilationDependenciesMetadata'.
   > Could not resolve xyz.wingio.syntakts:syntakts-compose-material3:1.0.0-rc06.


chore: fallback to existing engine with docker

If we cannot contact the remote repository, try to first use an already existing docker dagger engine.
If there none failed
Signed-off-by: laurent Grondin<EMAIL_ADDRESS>

@laupse, can you please signoff your commit and rebase 🙏
LGTM, but I'll let Erik validate 😇

I'm just looking for a way to test this, if i do ./hack/dev i can test this code right ?
And i did signed the commit but it says i did'nt. Did i miss something ?

i did signed the commit but it says i did'nt. Did i miss something ?

Summary
Commit sha: 9a2c41d, Author: laupse, Committer: laupse; Expected "laupse<EMAIL_ADDRESS>but got "laurent Grondin<EMAIL_ADDRESS>
I'm just looking for a way to test this, if i do ./hack/dev i can test this code right ?
Mmmh, this will spawn a local version of Dagger with this code yes. To manually test it though, it might require some tweaking

Mmmh, this will spawn a local version of Dagger with this code yes. To manually test it though, it might require some tweaking

How can i build the cli locally ? Then i can use the EXPEREIMENTAL_CLI_BIN to target this and do some test ?

How can i build the cli locally ?

I was making some tests on your PR. I find the "failed to remove old container error a bit surprising, wdyt ?

Ok i'll test with this thx !

Ok i'll test with this thx !

By the way, once you feel confident on your manual tests, can you please make a recap comment with screenshots (if possible), to facilitate the tests on our side + get a track record of what you thought of 🙏 If we think about new ones, we'll add them as follow-up comments 😇

Here is tested dagger which latest release against .\dagger which is my branch. The docker ps output match the behaviour
Also fix the error you pinted out

Ok, on my side:

tested with wrong GHCR credentials (issue https://github.com/dagger/dagger/issues/4381).

Past engine running
There seems to be a a missing space or \n, which makes, on your example, the engine word stuck to the next engine id: engineeY... (on the logs).
It also appears for me, as the engine==> is also missing a \n or a  .

After removal of all engines

Summary
Works fine, apart the small missing space / \n 👏

Here we go !

Last commit fixes #4381 with improve log message


New version: Kubernetes.kwok version 0.5.2

Pull request has been created with komac v2.2.1 :rocket:
Microsoft Reviewers: Open in CodeFlow

For moderators
This account is automated by Github Actions and the source code was created by CoolPlayLin. If you have any questions about any pull request, don't hesitate to ping Exorcism0666.

[!warning]
With the new version of Komac, numerous Pull Requests may contain bugs or unwanted elements. If such issues arise, please tag russellbanks so that these bugs can be addressed in future updates.

[!important]
Please carefully review these Pull Request before merging. If it is a Pull Request for removing incorrect content and the URLs are issue, free upon manual checking, please close this Pull Request directly. (It is best to inform Exorcism0666 of the closure.)

良い一日をお過ごしください！

Automatic Validation ended with:

Executable C:\Users\validator\AppData\Local\Microsoft\WinGet\Packages\Kubernetes.kwok__DefaultSource\kwok.exe returned exit code: 1
Executable C:\Users\validator\AppData\Local\Microsoft\WinGet\Links\kwok.exe returned exit code: 1

(Automated response - build 877.)

It would be nice to upgrade Kubernetes.kwokctl as well. :pray:


Automated Package Upgrade

:tada: This PR is included in version 25.3.82 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


feat: nominate

[x] - nomination constants (nomination period, unbonding period)
[x] - expected rewards (just a global average for now)
[x] - fee, user input validations
[x] - sign, submit extrinsic
[x] - notification subscription

refactoring

[x] - move constants,  to context
[x] - reorg cart page components data flow
[x] - components and typography (bring them in from ui-components once it gets there)

I would update this with #302 once that's in.


Improve log and status

Codecov Report

Merging #12 (8d4269a) into integration (e5a340d) will decrease coverage by 2.63%.
The diff coverage is 44.82%.

@@               Coverage Diff               @@
##           integration      #12      +/-   ##
===============================================
- Coverage        87.67%   85.04%   -2.64%
===============================================
  Files               22       22
  Lines              430      468      +38
===============================================
+ Hits               377      398      +21
- Misses              53       70      +17

Impacted Files
Coverage Δ

lib/decision/api_manager.dart
80.95% <0.00%> (-4.05%)
:arrow_down:

lib/hits/hit.dart
87.50% <ø> (+1.78%)
:arrow_up:

lib/utils/constants.dart
100.00% <ø> (ø)

lib/api/service.dart
62.96% <33.33%> (-28.71%)
:arrow_down:

lib/api/tracking_manager.dart
42.50% <50.00%> (-7.50%)
:arrow_down:

lib/visitor/visitor_delegate.dart
93.54% <66.66%> (-6.46%)
:arrow_down:

lib/visitor.dart
86.00% <100.00%> (+0.89%)
:arrow_up:

lib/visitor/strategy/default_strategy.dart
80.30% <100.00%> (ø)

... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e5a340d...8d4269a. Read the comment docs.


Incorrect order for parameter check in uninstall release, Can we adjust it?

I read the code of uninstall release to study the code of this process.
Find that in UninstallRelease function, do these three steps:
1、 get release locker of release name
2、  check  whether the release name  is ValidName.MatchString
3、 check  whether len(req.Name)  is long than releaseNameMaxLen
But if the   len(req.Name) is long than releaseNameMaxLen, the req.Name is illegal. Surely the get relese locker is failed, as it certainty can't find an release which name is req.Name.
So can adjust the parameter check order ? To check  whether  the req.Name  is  illegal first , and then to get release locker.

This is a valid point, we probably should be checking the release name before trying to lock it

Closed by #2540
Thank you @yan234280533 !


Change the ID of a namespace resource

The docker-registry-cache namespace is defined in a kubernetes_namespace
resource which had the identifier "docker_registry_cache"
This was confusing the orphaned namespace checker, which reports that
the "docker-registry-cache" namespace is not defined in our source code.
Changing the identifier to match the namespace name should resolve this
problem (assuming terraform will allow an identifier with hyphens in
it).

According to terraform plan, this will delete the whole namespace, and then recreate it with the same name. This shouldn't matter - it just means the cache will go away briefly, which it does anyway when the underlying disk space it uses grows too large.


Fix: tf.contrib.distribute.CollectiveAllReduceStrategy can't load model from checkpoint #45839

About issue 45839
I analyzed and debug the source code and found the following problems:

The value of worker_context.should_checkpoint will change the basic path of checkpoint. But the restore action is related to

the base path of the checkpoint.
venv/lib/python3.6/site-packages/tensorflow_core/python/training/monitored_session.py line: 341
def _create_monitored_session_with_worker_context(...)
  ...
  if (((save_checkpoint_secs and save_checkpoint_secs > 0) or
       (save_checkpoint_steps and save_checkpoint_steps > 0)) and
      checkpoint_dir):
    if worker_context.should_checkpoint:
      all_hooks.append(
          basic_session_run_hooks.CheckpointSaverHook(
              checkpoint_dir,
              save_steps=save_checkpoint_steps,
              save_secs=save_checkpoint_secs,
              scaffold=scaffold))
    elif tmpdir:
      all_hooks.append(
          basic_session_run_hooks.CheckpointSaverHook(
              os.path.join(checkpoint_dir, tmpdir),
              save_steps=save_checkpoint_steps,
              save_secs=save_checkpoint_secs,
              scaffold=scaffold))
  ...

The value of worker_context.should_checkpoint depends on the _is_chief attribute of Class Collective AllReduce Extended

venv/lib/python3.6/sitepackages/tensorflow_core/python/distribute/collective_all_reduce_strategy.py line 231

def _initialize_multi_worker(self, cluster_resolver):
    """Initializes the object for multi-worker training."""
    ....
    self._is_chief = multi_worker_util.is_chief(
        cluster_spec, task_type,task_id)

multi_worker_util.py is_chief() is the key to the problem

/venv/lib/python3.6/site-packages/tensorflow_core/python/distribute/multi_worker_util.py
line 93: is_chief
def is_chief(cluster_spec=None, task_type=None, task_id=None):
  ...
  if task_type == "chief" or task_type == "evaluator":
    return True

  if ("chief" not in cluster_spec and task_type == "worker" and task_id == 0):
    return True
  return False
  ...

My sulotion as follow:
def is_chief_for_collective_all_reduce_strategy(cluster_spec=None, task_type=None, task_id=None):
  # Fix the bug that the model fails to load from checkpoint when using collective_all_reduce_strategy
  # if task_type == "chief" or task_type == "worker" or task_type == "evaluator":
  if task_type == "chief" or task_type == "evaluator":
    return True
  return False

I will use this solution in kubeflow in production environment.  Is there any risk in this?
Hope the PR can pass, if any questions, please give me some advice.
Thanks a lot!!!

This should be opened against master branch?

@mihaimaruseac

This should be opened against master branch?

Yes. I read the code of master branch and this problem also exists.
I ran the code on Tensorflow v2.3 and encountered the same problem.
Should I need to resubmit a new PR to the master branch?

Let's try to rebase this against master branch or close this and open one on the master branch.

Thanks! I have resubmitted a PR against the master branch on #45839, may I close this PR?


Added better support for whitespace/multi-line lists and multiple config files.

parseconf now returns the dictionary it parses, as well as assigning the global conf variable
Can now parse multi-line lists
strips white-space before the = character
String => AbstractString throughout

I extended the tests to parse a multi-line list, with a space before the equals sign.

This is great. Got bit by the whitespace before = the other day.
Any idea why this fails on 0.3.11? It says "AbstractString not defined"? WTF?

Looks like AbstractString wasn't defined until 0.4.0:
https://github.com/JuliaLang/Compat.jl/blob/master/src/Compat.jl#L34


Update README.md

fix #542

Why is there 1. everywhere? 🙈

It's just how markdown works :)
https://www.markdownguide.org/basic-syntax/#ordered-lists

Why is there 1. everywhere? 🙈

It's just how markdown works :)
https://www.markdownguide.org/basic-syntax/#ordered-lists

😮  I always thought numbers mattered, thanks for that !


Add basic duration comparison

Fix #39

Codecov Report

Merging #41 (c23e21c) into master (d1f24db) will increase coverage by 0.64%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master      #41      +/-   ##
==========================================
+ Coverage   84.82%   85.46%   +0.64%
==========================================
  Files          14       14
  Lines         514      516       +2
==========================================
+ Hits          436      441       +5
+ Misses         56       55       -1
+ Partials       22       20       -2

Impacted Files
Coverage Δ

core/condition.go
90.90% <100.00%> (+3.28%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d1f24db...c23e21c. Read the comment docs.

@TwinProduction I added a patch for days, maybe it should be in another PR though

I don't know if the two if could be made as a single one, to avoid using continue. Feel free to amend my commit.

@Exagone313 Would you mind creating a different PR for the implementation of days? 🙏
I prefer releasing v1.6.0 with just support for the default time.ParseDuration(...) and then looking at supporting days separately

@Exagone313 I did a soft revert of the commit you made for supporting days so that you can create a different PR for that feature

Released in v1.6.0

Ok, I should work on this during the week end.


test & document

test

[ ] test with digits toy data
[ ] test for data with <11 observations
[ ] test with scRNAseq data (with "ground truth") e.g., Lee2020NatGenet
[ ] document speed of each from start to finish

document

[ ] README

[ ] note in the docs: fix any pickle (knn graph) issue by fixing Python version to 3.9!
[ ] make section for scRNAseq analysis/bioinformatiocs related information like package defaults and linking there\

[ ] UMAP scanpy (euclidean, min_dist=0.5), Seurat (cosine,.,..) -> check & link
[ ] Both, Seurat and scanpy use Leiden with RBConfigurationVertexPartition
Seurat parameters
scanpy parameters

[ ] Features

[ ] Leiden clustering supporting all partition_type, resolution_parameter and n_iterations
[ ] Visualization of clustering results as separate 2D scatter plot panel across all available embeddings
[ ] Interactive visualization as categorical variables in 2D & 3D

[ ] Methods

[ ] config.yaml

add to docs
“The validation of clustering structures is the most difficult and frustrating part of cluster analysis. Without a strong effort in this direction, cluster analysis will remain a black art accessible only to those true believers who have experience and great courage.”
Algorithms for Clustering Data, Jain and Dubes, 1988

Caveat: internal cluster indices are linear, using Euclidean metric and PCA (0.9) results.

caveat: Silhouette score sometimes generates NA due to a known bug. Clusterings with NA scores are removed before TOPSIS is applied.

clustification's performance scales with available cores.
Lee2020NatGenet:

2 cores -> 1.5h
8 cores -> 22m


Feature request: dice notation

Asked by kmcgerald on the Snips App Store:

Any plans to add other sided dice like those use in classic RPGs? D4, D8, D10, D12, and D20. Default to 6 if they don't specify. I'll probably add your app to my assistant either way but it would be nice to have other classic dice even though the same could be done with the random number skill you already have.

See https://en.wikipedia.org/wiki/Dice_notation

Implemented in https://github.com/koenvervloesem/snips-app-pick-something-random/pull/10


Size Mismatch for Depth-Anything-V2-Small Model

Hello,
I have recently downloaded the Depth-Anything-V2-Small model from the provided link on GitHub. According to the documentation, the model size is listed as 24.8M. However, upon downloading, I noticed that the actual size of the model does not match this specified size.
Here are the details:
Expected Size: 24.8M
Actual Size: 99.2M
Could you please verify if there is an error in the size listing or if there is another issue that I might be unaware of?
Thank you for your assistance.

The 24.8M is the number of model parameters, while the 99.2M is the checkpoint storage size. Since our model is stored in float32 (4 bytes), the storage size correctly matches the parameter size.

The 24.8M is the number of model parameters, while the 99.2M is the checkpoint storage size. Since our model is stored in float32 (4 bytes), the storage size correctly matches the parameter size.

You can reply with:
"Thank you for the clarification, I understand now!"


chore: add example instructions

This PR adds instructions of how to deploy cymbal bank example using cloud build.
Also adds code to 5-appinfra be deployable by cloud build.

[x] Adds missing roles
[x] Adds missing outputs
[x] Adds missing files (terraform.tfvars, backend.tf)
[x] Adds APIs enabling
[x] Removes Extra cymbal-bank folder at example
[x] Fixes symbolic links
[ ] Adds integration tests
[ ] Adds instructions deployment instructions

/gcbrun

/gcbrun

/gcbrun

/gcbrun


Move directory creation to PANDORA.Databse

Update from Issue #30
Database directories should not be made during the installation. Each function needing those directories should check if they exist and, if not, should create them.

Related to #156 and #164

Directory creation is now part of steup.py since #201 .


Struggling to use a listener from Java in python

Heya! I saw Jpype and i was testing it out today, and for my first test i am trying to use Javacord! Everything works except for using the listener, any idea on how i can use a Java listener in Python? I tried using it as a decorator but i got an error so i'm assuming that wouldn't work, any help is appreciated, thanks!

If you can post an example of what you are trying to do I can likely give you some tips.

If you can post an example of what you are trying to do I can likely give you some tips.

Heya! Thanks for responding, here's my code:
from jpype import startJVM, JImplements, JOverride
import jpype.imports
from jpype.types import *

startJVM(classpath=['jars/*'])

from org.javacord.api import DiscordApi
from org.javacord.api import DiscordApiBuilder
from org.javacord.api.listener.message import MessageCreateListener as JMessageCreateListener

vapi = DiscordApiBuilder().setToken(JString("My Bot Token")).login().join()

@JImplements(JMessageCreateListener)
class MessageCreateListener(object):
    def __init__(self):
        pass

    @JOverride
    def onMessageCreate(self, event):
        if event.getMessageContent().lower() == "!ping":
            event.getChannel().sendMessage(JString("Pong!"))

vapi.addListener(MessageCreateListener())

And here's the error i'm getting:
2021-01-20 09:57:02.642+0000 ERROR org.javacord.core.util.event.EventDispatcherBase Unhandled exception in a listener thread for 3Lovers4Ever! {shard=0}
 org.jpype.PyExceptionProxy
        at org.jpype.JPypeContext.createException(Unknown Source)
        at org.jpype.proxy.JPypeProxy.hostInvoke(Native Method)
        at org.jpype.proxy.JPypeProxy.invoke(Unknown Source)
        at com.sun.proxy.$Proxy6.onMessageCreate(Unknown Source)
        at org.javacord.core.util.event.EventDispatcher.lambda$dispatchMessageCreateEvent$102(EventDispatcher.java:3636)
        at org.javacord.core.util.event.EventDispatcherBase.lambda$dispatchEvent$10(EventDispatcherBase.java:197)
        at org.javacord.core.util.event.EventDispatcherBase.lambda$checkRunningListenersAndStartIfPossible$17(EventDispatcherBase.java:265)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
2021-01-20 09:57:03.545+0000 ERROR org.javacord.core.util.event.EventDispatcherBase Unhandled exception in a listener thread for 3Lovers4Ever! {shard=0}
 org.jpype.PyExceptionProxy
        at org.jpype.JPypeContext.createException(Unknown Source)
        at org.jpype.proxy.JPypeProxy.hostInvoke(Native Method)
        at org.jpype.proxy.JPypeProxy.invoke(Unknown Source)
        at com.sun.proxy.$Proxy6.onMessageCreate(Unknown Source)
        at org.javacord.core.util.event.EventDispatcher.lambda$dispatchMessageCreateEvent$102(EventDispatcher.java:3636)
        at org.javacord.core.util.event.EventDispatcherBase.lambda$dispatchEvent$10(EventDispatcherBase.java:197)
        at org.javacord.core.util.event.EventDispatcherBase.lambda$checkRunningListenersAndStartIfPossible$17(EventDispatcherBase.java:265)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)```

Heya! Thanks for responding, here's my code:
from jpype import startJVM, JImplements, JOverride
import jpype.imports
from jpype.types import *

startJVM(classpath=['jars/*'])

from org.javacord.api import DiscordApi
from org.javacord.api import DiscordApiBuilder
from org.javacord.api.listener.message import MessageCreateListener as JMessageCreateListener

vapi = DiscordApiBuilder().setToken(JString("My Bot Token")).login().join()

@JImplements(JMessageCreateListener)
class MessageCreateListener(object):
    def __init__(self):
        pass

    @JOverride
    def onMessageCreate(self, event):
        if event.getMessageContent().lower() == "!ping":
            event.getChannel().sendMessage(JString("Pong!"))

vapi.addListener(MessageCreateListener())

And here's the error i'm getting:
2021-01-20 09:57:02.642+0000 ERROR org.javacord.core.util.event.EventDispatcherBase Unhandled exception in a listener thread for 3Lovers4Ever! {shard=0}
 org.jpype.PyExceptionProxy
        at org.jpype.JPypeContext.createException(Unknown Source)
        at org.jpype.proxy.JPypeProxy.hostInvoke(Native Method)
        at org.jpype.proxy.JPypeProxy.invoke(Unknown Source)
        at com.sun.proxy.$Proxy6.onMessageCreate(Unknown Source)
        at org.javacord.core.util.event.EventDispatcher.lambda$dispatchMessageCreateEvent$102(EventDispatcher.java:3636)
        at org.javacord.core.util.event.EventDispatcherBase.lambda$dispatchEvent$10(EventDispatcherBase.java:197)
        at org.javacord.core.util.event.EventDispatcherBase.lambda$checkRunningListenersAndStartIfPossible$17(EventDispatcherBase.java:265)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
2021-01-20 09:57:03.545+0000 ERROR org.javacord.core.util.event.EventDispatcherBase Unhandled exception in a listener thread for 3Lovers4Ever! {shard=0}
 org.jpype.PyExceptionProxy
        at org.jpype.JPypeContext.createException(Unknown Source)
        at org.jpype.proxy.JPypeProxy.hostInvoke(Native Method)
        at org.jpype.proxy.JPypeProxy.invoke(Unknown Source)
        at com.sun.proxy.$Proxy6.onMessageCreate(Unknown Source)
        at org.javacord.core.util.event.EventDispatcher.lambda$dispatchMessageCreateEvent$102(EventDispatcher.java:3636)
        at org.javacord.core.util.event.EventDispatcherBase.lambda$dispatchEvent$10(EventDispatcherBase.java:197)
        at org.javacord.core.util.event.EventDispatcherBase.lambda$checkRunningListenersAndStartIfPossible$17(EventDispatcherBase.java:265)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)```

The error message indicates that a unhandled exception is leaking into the proxy.   I would recommend catching it to see what is going on.
    @JOverride
    def onMessageCreate(self, event):
        try:
          if event.getMessageContent().lower() == "!ping":
              event.getChannel().sendMessage(JString("Pong!"))
         except Exception as ex:
             print(ex)
             raise ex

The error message indicates that a unhandled exception is leaking into the proxy.   I would recommend catching it to see what is going on.
    @JOverride
    def onMessageCreate(self, event):
        try:
          if event.getMessageContent().lower() == "!ping":
              event.getChannel().sendMessage(JString("Pong!"))
         except Exception as ex:
             print(ex)
             raise ex

Ah! That fixed the issue thanks! It seems like the problem we were having before was that we were trying to use .lower() on a Java object, thank you again!

Ah! That fixed the issue thanks! It seems like the problem we were having before was that we were trying to use .lower() on a Java object, thank you again!

No problem.  Though this is the place where I look for feedback to improve things.  I few quick questions help in resolving this sort of question in the future.

Did you look through the JPype documentation regarding Proxies?  (It is fine if you didn't, I just need to know where you looked before putting in an issue so I can judge the effort that should be placed on documentation.)
Was it clear from the documentation that Python exceptions would be transferred into Java?
Looking at it now, is the name of the exception (org.jpype.PyExceptionProxy) clear that this was Python generated exception?
Is there anything in the way of improved documentation that would have made it easier to have resolved your issue?

Your case was a bit unusual as the call did not originate from Python. Normally if Python had been in the call stack would have been unwrapped back into a Python exception.   But in your case a Java thread launched the proxy and thus its handlers got an exception that was wrapped from Python.
I do see one improvement that can be made.   The 3rd party code should have gotten a message with the contents of the Python exception so that you don't have to add the try block to get the attribute error "lower not found".  Though this may have be a problem in the 3rd party logger.  I haven't tried getting the message from a PyExceptionProxy to see how useful it is.

No problem.  Though this is the place where I look for feedback to improve things.  I few quick questions help in resolving this sort of question in the future.

Did you look through the JPype documentation regarding Proxies?  (It is fine if you didn't, I just need to know where you looked before putting in an issue so I can judge the effort that should be placed on documentation.)
Was it clear from the documentation that Python exceptions would be transferred into Java?
Looking at it now, is the name of the exception (org.jpype.PyExceptionProxy) clear that this was Python generated exception?
Is there anything in the way of improved documentation that would have made it easier to have resolved your issue?

Your case was a bit unusual as the call did not originate from Python. Normally if Python had been in the call stack would have been unwrapped back into a Python exception.   But in your case a Java thread launched the proxy and thus its handlers got an exception that was wrapped from Python.
I do see one improvement that can be made.   The 3rd party code should have gotten a message with the contents of the Python exception so that you don't have to add the try block to get the attribute error "lower not found".  Though this may have be a problem in the 3rd party logger.  I haven't tried getting the message from a PyExceptionProxy to see how useful it is.

For the first question, yeah i skimmed through the docs multiple times (though i probably wasn't paying full attention), for the second question, not really 😅, i thought that all Java exceptions would be converted to a python exception somehow, for the 3rd, yeah it was clear that python caused this issue, but it didn't really have any information that could help me figure out what the issue actually was. And i think the docs weren't the issue, but a note that some python methods might not work on Java objects (without some formatting/processing) would be pretty nice to remind anyone in the future that the error could be an issue from python

For the first question, yeah i skimmed through the docs multiple times (though i probably wasn't paying full attention), for the second question, not really 😅, i thought that all Java exceptions would be converted to a python exception somehow, for the 3rd, yeah it was clear that python caused this issue, but it didn't really have any information that could help me figure out what the issue actually was. And i think the docs weren't the issue, but a note that some python methods might not work on Java objects (without some formatting/processing) would be pretty nice to remind anyone in the future that the error could be an issue from python

Thank you for your comments.  And good luck with your project!

Thank you for your comments.  And good luck with your project!


Fix: switch in the source and target distribution allocation of the potentials

Quick edit: I believe my understanding was flawed, sorry. It could be correct as it currently is. I (may wrongly) assumed Y would denote the target distribution and we were looking for the map from X to Y (hence, source to target). However, it seems that Y is sampled from Q, which is stated as the source distribution in the paper (and not only in the pseudocode, but also at other locations). The goal is to learn the map from Q to P. I realized this after having a more detailed look at the terminology used there again. Now that I have thought about it, my assessment seems to have been overhasty. I would still be interested if someone could clarify my understanding, however, so I get it actually correct. :-)

Perhaps my understanding is also wrong (which I cannot rule out).
According to the original paper, and my understanding, the g potential should be applied to the target distribution and the f potential to the source (see, e.g., Eq. 5 or Eq. 9). In the current version, this is switched.
I suspect this mix-up comes from the fact that Algorithm 1 in the paper has the source and target also mixed up in the pseudocode. Here, the source distribution is indeed given to the g potential and the target distribution to f. However, following the equations, I think this is simply a mistake (although I am not 100% sure).
My suspicion is further backed by the original implementation of the authors, where the source samples are given to the f potential and the target to the g.
After changing this accordingly, I feel that this improved the convergence speed as well (from a few first test runs).

Thanks Philip, we'll take a look at this shortly.

Thanks for the clarification! After re-reading, I assumed the same. Sorry for the inconvenience, and thanks for the great work!

Thanks Philip for your PR and the comments. Do you see a fix, at least in the docs or pydocs, that could help people not make the same switch?

Thanks Philip for your PR and the comments. Do you see a fix, at least in the docs or pydocs, that could help people not get the same misunderstanding?

Thanks for asking, Marco!
I think my biggest confusion came from the use of the potentials. In the paper, $f$ is used for the target and $g$ for the source. Additionally, $Y$ is used for the source and $X$ is used for the target.
As a novice in the field of optimal transport, I naively assumed that the $f$ potential would always apply to the source and that $X$ would normally be used to denote the source (and $g$ and $Y$ in a corresponding way).
In the OTT-jax code, it is actually clearer, since the terms "source" and "target" are used instead of "X" and "Y" to denote the associations. So what remains is simply my confusion about the responsibilities of the potentials.
In this regard, I think it might help to add in the documentation that the $g$-potential is used for the source and the $f$-potential for the target distribution to clarify this.
However, I am not sure if there is a reason why the authors kept this change? Is it perhaps a common convention that I haven't quite figured out yet? Or it could be a mathematical/theoretical trick that escaped me when I read the paper. In which case, an explanation of it would be very helpful.
Many thanks already.

Dear @ppnaumann,
thanks for your feedback and indeed, it might be worth extending the documentation with your suggestion. As you can use source and target interchangeably, it does not matter for which direction you set f and g. So I will extend the documentation with the information that g maps source to target, and f target to source  (for both modules, NeuralDual and NeuralDualSolver).
Thanks,
Charlotte


forever stopbypid does not work

Forever version 0.13.0
Here's trying to use stopbypid
root@35747a7d2ba4:/home/user/scripto2# forever list
info:    No forever processes running
root@35747a7d2ba4:/home/user/scripto2# forever start -a -l forever.log -o /var/log/server.js.log -e /var/log/server.js.err.log --pidFile /run/scripto.pid -c node scripto/ep_rundown/node/server.js
warn:    --minUptime not set. Defaulting to: 1000ms
warn:    --spinSleepTime not set. Your script will exit if it does not stay up for at least 1000ms
info:    Forever processing file: scripto/ep_rundown/node/server.js
root@35747a7d2ba4:/home/user/scripto2# forever stopbypid $(cat /run/scripto.pid)
error:   Forever cannot find process with pid: 493
root@35747a7d2ba4:/home/user/scripto2# forever list
info:    Forever processes running
data:        uid  command script                            forever pid id logfile               uptime
data:    [0] 1cMD node    scripto/ep_rundown/node/server.js 487     493    /.forever/forever.log 0:0:0:12.665
root@35747a7d2ba4:/home/user/scripto2#

Reporting the issue is still here. With the rollback from 14, any chance of getting this fixed?

It's a confirmed bug which has been fixed via 68502f1, since 54194df. after v 0.13.0


Proof-of-concept for #113 (ensureGlobalSecondaryIndexExists method)

This is just a proof-of-concept for further feedback right now. If the idea is generally acceptable, I can finish it out.

@jeskew Sorry for the delay. I had a chance today to return to this and write some unit tests. Can you take a look and let me know whether it looks sufficient?
I can get all unit tests running clean on my machine (macos), but haven't run the integration tests. Are there other steps required to run the integration tests besides the following?
aws configure
# set defaults for access_key_id, secret_access_key, and region
npm install
npm run integ

@jeskew Any update on whether this can be merged? We are dealing with a fair bit of copied code in our consuming project until we can add this functionality

I see this is merged but not published to npm. Why is that?


Graham/margin change

These are the minimal changes for the margins to be fixed and the majority of the latex build warnings to be gone.

Using the twoside option on the document class inserts extra pages into the front matter and I was unable to fix it via the document class without deleting the majority of the bsu-cs.cls document class front matter section.

There are still two build warnings that come from \buildFrontPages but as that comes from the class file I am not sure theres a way to pinpoint where it comes from besides commenting out lines and then rebuilding.  Seems like it is from something on the title page and "committee or proposal approval page".

hyperref package is commented out as it produces ~35 warnings that are specific to bsu-cs.cls and the various header redefinitions.

The warnings from hyperref aren't significant as it works perfectly fine in the actual document. It just doesn't like some of the front matter stuff.  The main issue is the twoside option -- I will take a look at it again.


fix(app): rename resolveCredentialDisplay to parseCredentialDisplay.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (08361da) 89.41% compared to head (1564a13) 89.41%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #735   +/-   ##
=======================================
  Coverage   89.41%   89.41%
=======================================
  Files         108      108
  Lines        4514     4514
=======================================
  Hits         4036     4036
  Misses        320      320
  Partials      158      158

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


[Feature-2622][core] Local task supports cancel

Purpose of the pull request

Brief change log

Verify this pull request

This pull request is code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:

close #2622


Fix WasmSection::data

Regression introduced by #77

Coverage remained the same at 0.0% when pulling 84af9c99bea2a28ca8d9d1765c1db1fcb991e8a7 on philipc:wasm into 5b5eb03cfa5213fc37f9b63b4bd7904372a84082 on gimli-rs:master.

Also @fitzgen, I just wanted to check that you still want to do reviews for this crate. I know you're busy with lots of other stuff, so I don't want to impose on your time unnecessarily. I've got a few more PRs for this crate in the works.

Also @fitzgen, I just wanted to check that you still want to do reviews for this crate. I know you're busy with lots of other stuff, so I don't want to impose on your time unnecessarily. I've got a few more PRs for this crate in the works.

I'm happy to review PRs if you find it helpful! They don't tend to be that huge of a time suck. Thanks for checking :)


Favorite refactoring (wip)

Stuff to do

[x] Refactor ember favorite handling (polymorphic)
[x] Refactor user controller for anime and manga favorites
[x] Add global component for favorite box of any media type
[x] Implement drag and drop sort
[x] Delete favorite records via ember data
[x] Bulk-update fav_rank property of favorites
[x] Fuzzy search anime and manga input to add media from fav box
[x] Build new polymorphic relation in ember for new favorites (undocumented ember shizzle)
[ ] Add endpoint to effectively add media to your favorites
Anime and Manga is using id as identifier for their polymorphic relation on rails side, while the ember application is using slugs as ids. This can either be solved by a workaround or maybe a direct way with Rails tricks. (cc. @NuckChorris)
[ ] Make it look pretty
The whole UI needs lots of CSS love (cc. @JoshFabian)

=====

[ ] Add favorite button to manga pages
This is optional, since these buttons might get removed
[ ] Add tab switch or new box for manga favorites
This should be done pretty quickly. We need a second edit indicator then though. Maybe this PR can be merged without this for now to test the new favorite box for anime and add all the manga stuff a bit later.

Can we add some tests at least for the backend stuff?

Merged in all the new stuff from master to prevent further merge conflicts caused by my first, hacky ember-cli update in this branch as a temporary workaround.
This PR should now be able to be tested with updated ember-cli and merged without any conflicts.


Duplicate siteUrls in the xml

In my xml there are duplicate siteUrls:
<url> <loc>https://example.com/</loc> <lastmod>2023-08-27T23:12:21.058Z</lastmod> <changefreq>daily</changefreq> <priority>0.7</priority> </url>
<url> <loc>https://example.com/</loc> <lastmod>2023-08-27T23:12:21.058Z</lastmod> <changefreq>daily</changefreq> <priority>0.7</priority> </url>
Any idea? This may be another thing related to the v2 entry.server.js being different that I described in another issue.

Sorry i can't reproduce this, can you provide more context with a repo? Thanks

closed due to inactivity


Updating user assigned properties for 2023-06-01 version

ARM (Control Plane) API Specification Update Pull Request
PR review workflow diagram
Please understand this diagram before proceeding. It explains how to get your PR approved & merged.

[1] ARM review queue (for merge queues, see [4])
The PRs are processed by time opened, ascending. Your PR may show up on 2nd or later page.
If you addressed Step 1 from the diagram and your PR is not showing up in the queue, ensure the label ARMChangesRequested
is removed from your PR. This should cause the label WaitForARMFeedback to be added.
[2] https://aka.ms/azsdk/support/specreview-channel
[3] List of SDK breaking changes approvers in pinned Teams announcement
[4] public repo merge queue, private repo merge queue (for ARM review queue, [1])
If you need further help with anything, see Getting help section below.
Purpose of this PR
What's the purpose of this PR? Check all that apply. This is mandatory!

[ ] New API version. (Such PR should have been generated with OpenAPI Hub).
[ ] Update existing version for a new feature. (This is applicable only when you are revising a private preview API version.)
[ ] Update existing version to fix swagger quality issues in S360.
[ ] Other, please clarify:

edit this with your clarification

Due diligence checklist
To merge this PR, you must go through the following checklist and confirm you understood
and followed the instructions by checking all the boxes:

[ ] I confirm this PR is modifying Azure Resource Manager (ARM) related specifications, and not data plane related specifications.
[ ] I have reviewed following Resource Provider guidelines, including
ARM resource provider contract and
REST guidelines (estimated time: 4 hours).
I understand this is required before I can proceed to Step 2, "ARM Review", for this PR.

Breaking changes review (Step 1)

If the automation determines you have breaking changes, i.e. Step 1 from the diagram applies to you,
you must follow the breaking changes process.
IMPORTANT This applies even if:

The tool fails while it shouldn't, e.g. due to runtime exception, or incorrect detection of breaking changes.
You believe there is no need for you to request breaking change approval, for any reason.
Such claims must be reviewed, and the process is the same.

ARM API changes review (Step 2)

If this PR is in purview of ARM review then automation will add the ARMReview label.
If you want to force ARM review, add the label yourself.
Proceed according to the diagram at the top of this comment.

Viewing API changes
For convenient view of the API changes made by this PR, refer to the URLs provided in the table
in the Generated ApiView comment added to this PR. You can use ApiView to show API versions diff.
Suppressing failures
If one or multiple validation error/warning suppression(s) is detected in your PR, please follow the
Swagger-Suppression-Process
to get approval.
Getting help

For general PR approval workflow, see the diagram at the top of this comment.
For guidance on fixing this PR CI check failures, see the hyperlinks provided in given failure
and https://aka.ms/ci-fix.
If the help provided by the previous points is not enough, post to https://aka.ms/azsdk/support/specreview-channel and link to this PR.

Please check the pipeline errors and fix them

Please check the pipeline errors and fix them

Hi @zizw123 Can you please re-look at this PR now

There do seem to be some interesting linter errors there.
Do your resource types support privateEndpointConnections?
Is this an already shipped API?
It doesn't appear to follow proper resource path guidelines probably since backupstorageconfig should be a camelcased plural such as "backupStorageConfigs". Fixing it would be breaking obviously.
"/Subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.RecoveryServices/vaults/{vaultName}/backupstorageconfig/vaultstorageconfig": {
"get": {
Resource type naming must follow camel case. Path: '/Subscriptions/{subscriptionId}/providers/Microsoft.RecoveryServices/locations/{azureRegion}/backupPreValidateProtection'
And apparently you forgot to document the default error response for many operations. And a delete operation is missing an expected success response code? These sem like they are probably oversights worth fixing!
❌ MissingDefaultResponse	Operation is missing a default response.
Location: Microsoft.RecoveryServices/stable/2023-06-01/bms.json#L397
❌ RequiredDefaultResponse	The response is defined but without a default error response implementation.Consider adding it.'
Location: Microsoft.RecoveryServices/stable/2023-06-01/bms.json#L397
❌ DeleteOperationResponses	The delete operation is defined without a 200 or 204 error response implementation,please add it.'
Location: Microsoft.RecoveryServices/stable/2023-06-01/bms.json#L448
❌ MissingDefaultResponse	Operation is missing a default response.
Location: Microsoft.RecoveryServices/stable/2023-06-01/bms.json#L448
❌ RequiredDefaultResponse	The response is defined but without a default error response implementation.Consider adding it.'
Location: Microsoft.RecoveryServices/stable/2023-06-01/bms.json#L448

Hi @TimLovellSmith I am not aware about this change as service is managed by multiple people.
The current PR is just a correction PR. We were earlier referencing a common type for UserAssignedIdentity which was only having read-only properties, but API allows the properties to be set as well hence we needed to create our own type to resolve this.

@shivani-tyagii Please can you seek the support of your cocontributors to respond as needed.

/azp run


Make spaces after commas optional

In Wireguard configs the addresses may be listed with or without spaces after commas (e.g., "<IP_ADDRESS>/24, <IP_ADDRESS>/24" or "<IP_ADDRESS>/24,<IP_ADDRESS>/24"). Therefore, using split(",") and strip() is appropriate and ensures all formats are handled.
Fixes #14

Looks good to me, pretty much how I would have fixed that issue myself.

@jaharkes Thanks for merging this, it's an issue I ran into a lot when using configs from providers. They all had inconsistencies in spaces 😂

@jaharkes Any plans for doing a new release? Once again, Thanks for this awesome lib

Yes, just adding some tests and fixing the ipv6 issue reported in #13 (have to strip [ ] to correctly parse an ipv6 endpoint_host)

Awesome! Thanks 💪


zksync-cli dev start command failed to build the Docker image due to a bug in the zkcli-in-memory-node Dockerfile

🐛 Bug Report for zkSync CLI
📝 Description
zksync-cli dev start command failed to build the docker image due to a bug in the zkcli-in-memory-node Dockerfile. I have submitted a PR to the zkcli-in-memory-node repo to fix this issue.
https://github.com/matter-labs/zkcli-in-memory-node/pull/3
Using curl -s  echo may cause the jq -r command to fail, but removing the echo command allows it to execute correctly. The current version of the Dockerfile will cause the build to fail; after modification, it can be built successfully.
# query faild with echo
curl -s echo "https://api.github.com/repos/matter-labs/era-test-node/releases/tags/v0.1.0-alpha.19" | jq -r
jq: parse error: Invalid numeric literal at line 1, column 10

# query successfully without echo
curl -s "https://api.github.com/repos/matter-labs/era-test-node/releases/tags/v0.1.0-alpha.19" | jq -r
{
  "url": "https://api.github.com/repos/matter-labs/era-test-node/releases/147603455",
  "assets_url": "https://api.github.com/repos/matter-labs/era-test-node/releases/147603455/assets",
  "upload_url": "https://uploads.github.com/repos/matter-labs/era-test-node/releases/147603455/assets{?name,label}",
  ...
}

🔄 Reproduction Steps

npx zksync-cli dev start

🤔 Expected Behavior
npx zksync-cli dev start command execute successfully.
😯 Current Behavior
npx zksync-cli dev start command execute faild.
🖥 Environment

Operating System: macOs Ventura 13.3
Node.js Version: 20.11.0
npm Version: 10.5.0
Docker Version: 24.0.2
curl Version: 7.87.0
jq Version: jq-1.7.1

📋 Additional Context
📎 Log Output

> npx zksync-cli dev start

Installing "In memory node"...
[+] Building 3.5s (10/11)
 => [zksync internal] load build definition from Dockerfile                                                                                0.0s
 => => transferring dockerfile: 787B                                                                                0.0s
 => [zksync internal] load .dockerignore                                                                                0.0s
 => => transferring context: 2B                                                                                0.0s
 => [zksync internal] load metadata for docker.io/library/ubuntu:latest                                                                    3.2s
 => [zksync auth] library/ubuntu:pull token for registry-1.docker.io                                                                       0.0s
 => [zksync 1/7] FROM docker.io/library/ubuntu:latest@sha256:77906da86b60585ce12215807090eb327e7386c8fafb5402369e421f44eff17e              0.0s
 => CACHED [zksync 2/7] WORKDIR /app                                                                                0.0s
 => CACHED [zksync 3/7] RUN apt-get update   && apt-get install -y curl jq   && apt-get clean   && rm -rf /var/lib/apt/lists/*             0.0s
 => CACHED [zksync 4/7] RUN echo "In memory node version: v0.1.0-alpha.19"                                                                 0.0s
 => CACHED [zksync 5/7] RUN curl -s echo "https://api.github.com/repos/matter-labs/era-test-node/releases/tags/v0.1.0-alpha.19" |   jq -r  0.0s
 => ERROR [zksync 6/7] RUN tar xz -f era_test_node.tar.gz -C /usr/local/bin/                                                               0.2s
------
 > [zksync 6/7] RUN tar xz -f era_test_node.tar.gz -C /usr/local/bin/:
#0 0.208 tar (child): era_test_node.tar.gz: Cannot open: No such file or directory
#0 0.208 tar (child): Error is not recoverable: exiting now
#0 0.210 tar: Child returned status 2
#0 0.211 tar: Error is not recoverable: exiting now
------
failed to solve: process "/bin/sh -c tar xz -f era_test_node.tar.gz -C /usr/local/bin/" did not complete successfully: exit code: 2
ⓘ There was an error while starting the testing environment:
ⓘ Command exited with code 17:
file:///Users/stan/.npm/_npx/ed2d5270b29717fa/node_modules/zksync-cli/bin/utils/helpers.js:52
                reject(new Error(`Command exited with code ${code}: ${errorOutput}`));
                       ^

Error: Command exited with code 17:
    at ChildProcess.<anonymous> (file:///Users/stan/.npm/_npx/ed2d5270b29717fa/node_modules/zksync-cli/bin/utils/helpers.js:52:24)
    at ChildProcess.emit (node:events:518:28)
    at maybeClose (node:internal/child_process:1105:16)
    at ChildProcess._handle.onexit (node:internal/child_process:305:5) {
  level: 'error',
  [Symbol(level)]: 'error',
  [Symbol(message)]: '\x1B[91mⓘ Command exited with code 17: \x1B[39m'
}

Node.js v20.11.0

I am also getting this error


Core carbon chemistry package

This issue is for discussing Core carbon chemistry package that goes beyond #135 work on gas exchange function. A Core carbon chemistry package came up in  #135 discussions with @seamanticscience, @stephdut, @mmazloff, @jmc and @raphaeldussin. Right now its

not clear if this makes sense

not clear what would be in scope and what be out of scope for such a package

@jm-c @mjlosch As you mentioned in PR #841 , there is duplication of code between pkg/bling, pkg/dic, and pkg/darwin. Carbon chemistry is one example, and forcing fields load is another. Darwin has a set of routines for loading external fields that seems like it could easily be ported to another package. Could this go in pkg/gchem?

I think that pkg/gchem would a good place. pkg/gchem was intended to be a common driver for difference bgc-models, and collecting common code here makes a lot of sense to me.


feat: Pass Theme to IconRow and introduce Cell Alignment

IconRow has previously inherited default theme for Img and Link without a way to update it, this fixes it by passing a theme to IconRow component.
Cells could not be aligned, added a align prop to the component.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Can we merge this? @jakubschenk

Can we merge this? @jakubschenk

Yes @iamhectorsosa


python3Packages.hydrus-api: init at 4.0.0

Description of changes
This is a python library for use with the HTTP API exposed by hydrus.
There aren't any unit tests.

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 198187 run on x86_64-linux 1

2 packages built:

python310Packages.hydrus-api
python39Packages.hydrus-api


Hyundai Tucson 2023 poor tuning

Waiting on more stock routes from user.
8426de28ebdc65a2|2023-01-27--20-07-53

[ ] Fix tuning/steer scaling
[ ] Look into other HKG models that are potentially like this

Looks like it's just one FW version that needs different tuning. Temporarily removed it in #27340 until we better understand the difference.

@adeebshihadeh looks like this camera FW version was added again under SANTA_CRUZ_1ST_GEN. I wonder if it also has bad tuning.

@nuwandavek by the way, it looks like we have 4 active Tucson users in the last 90 days with these different parts: {'1966303b764e7142', '8426de28ebdc65a2', 'bf1a6b0ba293c37b', 'a0191364609ee493'}

I've been running LAT_ACCEL = 2.9 - 3.1 and FRICTION = 0.095 for the past few days with the 2022 Hyundai Tucson Hybrid, and it is significantly better than the current values.

https://github.com/commaai/openpilot/pull/30513

I would like to combine the Santa Cruz and Tucson. According to the wikipedia page, it is "closely related" to the Tucson as it shares the platform code with an extension of OB meaning Open Bed. The mass and wheel base are a bit different though.
@nuwandavek can you look at these two vehicles in the fleet and determine if using a median value is safe, letting torqued learn the difference?


give_velocity works off global directions rather than relative to the player

The velocity entered into give_velocity should be relative to the player, not global directions.

Fixed in ec1705d.


⚠ Bofrost.be has degraded performance

In a2aef8e, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1073 ms

Resolved: Bofrost.be performance has improved in a1df020.


not launching

Could you please give some more specifics? Like, did you have Discord open when you launched the app??

Discord was launched before starting the app, I can try any of the 4 branches of discord if that fits the fancy, only 1 instance was launched at the time of the program launched, AM was open in bg

Discord was launched before starting the app, I can try any of the 4 branches of discord if that fits the fancy, only 1 instance was launched at the time of the program launched, AM was open in bg

try restarting discord and if that doesn't work try restarting your computer... sometimes discord rich presence has issues with apple silicon

if that doesn't work, in terminal run:
/Applications/applemusicrp.app/Contents/macOS/applemusicrp and tell me what it outputs

noticing a trend here and I believe it's a MacOS bug, it seems to be the problem that brings the Siri to halt, mic not working in a vm and other weird little things, it worked after a hard reboot

Yeah, not much I can do about that :(


Zerofox/fix connector timeout

Required items, please complete
Change(s):

removed the identity breach connector due to being incompatible with time-based trigger
Added async logic to Sentinel connector to improve performance
added logging entries for observability
changed the ZeroFox connector to generate entry batches instead of single entries

Reason for Change(s):

There was a case of missing logs, combined with missing entries of the ZeroFox feeds
Identity Breach connector required additional data which was incompatible with user needs, and thus was never usable

Version Updated:

Yes

Testing Completed:

Yes, Local tests were done on personal Sentinel workspace

Checked that the validations are passing and have addressed any issues that are present:

Yes

Hi @v-prasadboke , are there any comments regarding our PR?
here is a list of our connectors functioning, we implemented some fixes so that our data connector matched the ZeroFox API requirements better. Here's a list of active connectors

Can you share the invocation logs screenshot
It will expedite the merging process

Can you share the invocation logs screenshot It will expedite the merging process
Hi @v-prasadboke
Here are a few invocations, there are some errors because I was trying to improve the code and broke it by mistake

So the current changes which this PR has, is it breaking or working

So the current changes which this PR has, is it breaking or working

@v-prasadboke it is working

Ok Thanks for the confirmation

Hi @v-prasadboke  after discussing with our security team, you were given write access to our forked repo, feel free to include the commits you require
on a separate line, I saw that our release notes were included into the main branch before the code is that expected?

Thanks for the access @DNRRomero, Are there any more changes in your queue to the function app.

@v-prasadboke  none at all


Metadata Update

Hello,
Your dataset was added to CoronaWhy (https://www.coronawhy.org/) Data Lake on Dataverse as a piece of common COVID-19 data frame https://datasets.coronawhy.org/dataset.xhtml?persistentId=doi:10.5072/FK2/LWDW8F
Would you be willing to help with the maintenance of your dataset in Dataverse, e.g. adding the relevant metadata and keeping the dataset up-to-date? That will help to make the dataset findable and accessible for the medical science community.

Greetings, @reeshmabijosh.
I'd be glad to help maintain the appropriate information there to help make this repository more useful. How can I get started?

Hello,
Thanks for your willingness to contribute.
Please register in CoronaWhy Dataverse https://datasets.coronawhy.org/ and
you will get the necessary permissions for editing your dataset.
Then please add relevant metadata - description and keywords to your
dataset.
Thanks
Reeshma K
On Fri, Jun 19, 2020 at 2:44 AM Gabriel Alfonso Carranco-Sapiéns <
<EMAIL_ADDRESS>wrote:

Greetings, @reeshmabijosh https://github.com/reeshmabijosh.
I'd be glad to help maintain the appropriate information there to help
make this repository more useful. How can I get started?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/carranco-sga/Mexico-COVID-19/issues/15#issuecomment-646466481,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ALHC2U7MJDYYVDYHMZ5VEU3RXMCLPANCNFSM4OCHFOVQ
.


XSS problem

since we store all the params in preloadedData：
<script id="preloadedData" type="application/json">
...
</script>

any user can construct a url like  domain.com/a=%27%20</sCript><scRipt>8976</sCript>d=%27 to inject evil script when rendering.
Is it possible to validate all query params in isomorphic-relay-router ?

In order for the exploit to qualify as an XSS, isn't the injected data supposed to come from a different user? Maybe I do not fully understand the use case, could you explain the example in more detail?

see this url for detail:
view-source:http://shengupiao.com/earning/top-mover?agg=quarter' </sCript><scRipt>8976</sCript>d='&sort=gain

see this url for detail:
view-source:http://shengupiao.com/earning/top-mover?agg=quarter' </sCript><scRipt>8976</sCript>d='&sort=gain

In Chrome I get this:

When I look at the elements, I see two consecutive script tags:

In Firefox I get this:

Should I use a specific browser to be able to get the result as you are showing it above?

I was able to get the script in Firefox, similar to what you showed before:

i believe chrome/firefox is secure under this case, but :

it is vulnerable when some browser does not provide xss Auditor which chrome have
it will broke the whole page

I am sure Relay (with server rendering) is not the only library/application using Script tag to store JSON data within the page, that is subsequently uses. Are there any widely accepted practices for dealing with such an use case?

don't know yet  :disappointed:

Good catch. Probably, replacing <%- with <%= in <script id="preloadedData" type="application/json"><%- preloadedData %></script> should be enough.
Could any of you submit PRs to fix the examples?

I will try to do this today

Sorry for some reason having trouble with running the ToDo locally, I know I have done it before but right now getting all these weird errors, must be something with my setup. I will test as soon as I get this running.

@denvned Inside a script tag, standard HTML encoding doesn't apply. > becomes >, not &.
The solution would be to sanitise the JSON by replacing <\s*/ with <\/. The JSON spec allows for \/ for this purpose I believe. It's the only unnecessary escape (read: a backslash, and then a character that doesn't need it) it allows, IIRC.

The solution would be to sanitise the JSON by replacing <\s*/ with <\/.

I agree, escaping the forward slash is probably the cleanest solution here. And actually it is exactly what I did in isomorphic-relay-boilerplate: https://github.com/denvned/isomorphic-relay-boilerplate/blob/9ff4df83a9c2a8ae9068098204001618e960f31c/frontend/src/server/components/Page.js#L17


Caravan Gifts to Hostile Factions broken

When delivering gifts via caravans to hostile factions the trading menu just briefly opens and then immediately closes right after so that a trade/gift cannot be made. The issue is resolved when all other players leave the session.
This https://github.com/Parexy/Multiplayer/issues/1 looks related (so just mark this as duplicate in case all caravan related things are turned off).

Fixed by https://github.com/Parexy/Multiplayer/commit/5cc688caa70527aba66417b4f5d4acf3e7f73792
Use steam version or wait for <IP_ADDRESS>


Roll Flutter Engine from ba8e0d3e2f23 to<PHONE_NUMBER>2e (3 revisions)

https://github.com/flutter/engine/compare/ba8e0d3e2f23...28242309332e
2024-05-10<EMAIL_ADDRESS>[Impeller] treat glyph atlas texture as source of truth, remove copy of SkBitmap. (flutter/engine#52567)
2024-05-10<EMAIL_ADDRESS>Roll Dart SDK from 01121c008f4d to 13fc7db956c7 (2 revisions) (flutter/engine#52716)
2024-05-10<EMAIL_ADDRESS>Roll Skia from 11d892ce49b6 to d1118d56853c (1 revision) (flutter/engine#52717)
If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC rmistry<EMAIL_ADDRESS>on the revert to ensure that a human
is aware of the problem.
To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose
To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

Commit queue failed; closing this roll.


i've never seen a dragon ball

i just finished a web dev project so this is all you're gonna get from me today

:package:


Does an executed command wait for triggers to complete

It would be useful if the document confirmed whether when a command is executed by an external application (e.g. an EF Core call) the response is only sent once trigger execution has completed.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 2d8add50-414a-a033-6fc0-84de1c7735f2
Version Independent ID: 8f2f3149-7035-b3d4-6f09-a6eadd015e24
Content: CREATE TRIGGER (Transact-SQL) - SQL Server
Content Source: docs/t-sql/statements/create-trigger-transact-sql.md
Product: sql
Technology: t-sql
GitHub Login: @CarlRabeler
Microsoft Alias: carlrab

@haymansfield -- thank you for your question.
@CarlRabeler -- please look into this issue #4295.

@haymansfield Can you please provide me an example scenario? What type of trigger are you asking about?

@haymansfield (and @CarlRabeler ): Triggers in SQL Server execute synchronously in the operation that triggered them. Triggers are part of the transaction, whether explicit or auto-commit, that is the operation being performed. This should include DDL and LOGON triggers as well as DML triggers. This is how/why you are able to cancel an operation (even a log in) by calling ROLLBACK in the trigger.
Take care,
Solomon...
https://SqlQuantumLift.com/
https://SqlQuantumLeap.com/
https://SQLsharp.com/

@srutzky Thanks very much for the answer.

@srutzky Thanks very much for the answer.

You're quite welcome. Were you able to test any scenarios, or did you just dangerously take my word for it? 😉  🙀 Just curious 😺


[topgen] --top_ral switch also dumps a large number of RTL sources

topgen is written with the assumption that all sources are generated and checked into the repo. For DV (and likely SW as well) there are good reasons to generate sources in the scratch space only when the simulation is run, rather than checking in.
The problem is in the _process_top() method, which not only creates and returns the data-structures that are used for the generation of RTL/DV/SW/doc sources, but it also unconditionally generates several top-specific RTL, HJSon and other collateral. Consequently, when topgen is invoked with the intent to only create the top level RAL model, these additional sources are also created in the scratch space. This is benign, but not ideal.
I tried to mask the generation of these sources (i.e. all generate_*() function calls) by putting them inside if not args.top_ral clause, but seems like there is some kind of dependency on those functions too (especially, generate_clkmgr()) to successfully return from the _process_top() invocation. The following errors get thrown:
ERROR:  CLKMGR mismatched number of reset ports and nets
ERROR:  CLKMGR Following reset ports do not exist:
ERROR: rst_main_ni
ERROR: rst_io_ni
ERROR: rst_usb_ni
ERROR: rst_io_div2_ni
ERROR: rst_io_div4_ni
ERROR: module clock/reset checking failed
ERROR: xbar clock/reset checking failed
Error occured while validating top.hjson

Filing an issue to get this properly refactored later.

It will also be great to be able to invoke topgen just to query some information (which can be used in other scripts), without having a bunch of files being generated.

Looks like the issue is having to generate HJson data itself from the template on which the top depends on - perhaps for these IPs we only render the mako template (for the HJSon) in _process_top as a string first? We can convert that string into HJSon data directly without extracting from the generated file (and we maintain these IP specific HJSon data in the topcfg data structure so that we can come back to it later when we want to generate those files). This will allow us to move all of the generate_*() calls to outside _process_top().

see #2083, this isn't quite so straightforward.
It isn't just that we need to generate hjson data that the top depends on, it's also that generation of hjson between certain modules have a dependency.  So process_top needs to be called a couple of times, so that the first generation creates the proper hjson (for some modules) and the second pass creates the other module hjsons for those that have a dependency.  It's all very very messy and we need to figure it out how to better handle it one day (even this multi-pass doesn't completely work).
I THINK what you're getting at though, is to not generate the rtl (and other) collateral if we don't need it? If that's the case, i think it would be fine for _process_top to generate only the hjson first (if nothing else...this keeps the parsing method consistent between the 1st and 2nd runs), but hold off on generating everything else later.
Unless, is an hjson in string format can be parsed just like a normal hjson?

Yes - mako can return a string of what it processed without actually having to write the file. And you can convert that string to HJson data structure. So essentially we could completely postpone the generation of all sources.

is there any reason to postpone the hjson generation also?
the others i can see a rationale, but i'm not sure what the reason to
skipping hjson also.  Since you need that for the DV collateral anyways.
On Fri, Aug 20, 2021 at 12:12 AM Srikrishna Iyer @.***>
wrote:

Yes - mako can return a string of what it processed without actually
having to write the file. And you can convert that string to HJson data
structure. So essentially we could completely postpone the generation of
all sources.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/lowRISC/opentitan/issues/7857#issuecomment-902485750,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAH2RSSYQRNTVWHKDGIE5BDT5X57BANCNFSM5CPVVG2A
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email
.

I provide the scratch path to generate the RAL sources, and not the repo_top path. So what happens is it re-creates the whole directory structure in the scratch area to generate the HJSon file at the right place. But this is a duplicate of what is already checked-in. topgen has no idea that this is duplicate, which is not used by anything else.
One solution would be to make topgen understand where the repo_top is (which would be separate from --outdir), so that it always creates these sources relative to repo_top rather the --outdir. But this may make things even more confusing.

o i see. so in this case, you are assuming that all the collateral is
already in place, and we just want to go ahead and directly generate the dv
collateral, is that right?
On Fri, Aug 20, 2021 at 12:23 AM Srikrishna Iyer @.***>
wrote:

I provide the scratch path to generate the RAL sources, and not the
repo_top path. So what happens is it re-creates the whole directory
structure in the scratch area to generate the HJSon file at the right
place. But this is a duplicate of what is already checked-in. topgen has
no idea that this is duplicate, which is not used by anything else.
One solution would be to make topgen understand where the repo_top is
(which would be separate from --outdir), so that it always creates these
sources relative to repo_top rather the --outdir. But this may make
things even more confusing.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/lowRISC/opentitan/issues/7857#issuecomment-902491129,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAH2RSU2N5DQOHQA7OJPQRTT5X7HHANCNFSM5CPVVG2A
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email
.

Right, because we are not checking in the DV collateral.

yeah that makes sense.
So when we pass in the ral_switch, i guess really we should skip all the
data structure generation and skip directly ahead?
That seems like a very reasonable thing to do.  We'd probably just need to
refactor the code a bit.
On Fri, Aug 20, 2021 at 12:27 AM Srikrishna Iyer @.***>
wrote:

Right, because we are not checking in the DV collateral.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/lowRISC/opentitan/issues/7857#issuecomment-902493241,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAH2RSXEVHMTVH3OPAPBP43T5X7WZANCNFSM5CPVVG2A
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email
.

Hmm, to be precise, when we pass the ral switch, we need _process_top to generate the full topcfg data structure (which our code heavily relies on), but not create any of the files.

i can't remember now, but isn't topcfg mostly top_earlgrey.gen.hjson?
maybe if we can try to make those two equivalent, that would solve a lot of
problems.
On Fri, Aug 20, 2021 at 12:32 AM Srikrishna Iyer @.***>
wrote:

Hmm, to be precise, when we pass the ral switch, we need _process_top to
generate the full topcfg data structure (which our code heavily relies on),
but not create any of the files.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/lowRISC/opentitan/issues/7857#issuecomment-902495608,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAH2RSRITID4INCDEUHJ4KTT5YAHPANCNFSM5CPVVG2A
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email
.


Allow specifying custom string in value of usage

(current output edited, actual: -C, -- string, https://github.com/spf13/pflag/issues/139)
Flags:
  -C string   Act on path instead of working directory.
  -h, --help      help for git-id

^ right now it says 'string', it'd be neet if I could say UsageValueName: "path", so it'd display like so:
Flags:
  -C path   Act on path instead of working directory.
  -h, --help      help for git-id

Name for flag types is determined with the following function
// flag.go:566

// UnquoteUsage extracts a back-quoted name from the usage
// string for a flag and returns it and the un-quoted usage.
// Given "a `name` to show" it returns ("name", "a name to show").
// If there are no back quotes, the name is an educated guess of the
// type of the flag's value, or the empty string if the flag is boolean.
func UnquoteUsage(flag *Flag) (name string, usage string) {
	// Look for a back-quoted name, but avoid the strings package.
	usage = flag.Usage
	for i := 0; i < len(usage); i++ {
		if usage[i] == '`' {
			for j := i + 1; j < len(usage); j++ {
				if usage[j] == '`' {
					name = usage[i+1 : j]
					usage = usage[:i] + name + usage[j+1:]
					return name, usage
				}
			}
			break // Only one back quote; use type name.
		}
	}

	name = flag.Value.Type()
	switch name {
	case "bool":
		name = ""
	case "float64":
		name = "float"
	case "int64":
		name = "int"
	case "uint64":
		name = "uint"
	case "stringSlice":
		name = "strings"
	case "intSlice":
		name = "ints"
	case "uintSlice":
		name = "uints"
	case "boolSlice":
		name = "bools"
	}

	return
}

Meaning if you quote your desired usage value name/type with ` inside your usage string it will display correctly i.e.,:
flags.String("C", "", "Act on `path` instead of working directory.")

should turn to
Flags:
  -C path   Act on path instead of working directory.
  -h, --help      help for git-id

indeed, leaving open for missing docs


Ring of endurance RC boost

Remove the boost to runecrafting for ring of endurance (uncharged)
Currently it checks if you have either charged or uncharged and applies a boost, this should only be for the charged version
see here: https://github.com/oldschoolgg/oldschoolbot/blob/master/src/commands/Minion/rc.ts#L85

@6enny6
they both register as 1% though in the boosts after sending

fix hasnt been merged yet see #3351

fix hasnt been merged yet see #3351

I'm just saying I brought it up in the past and i forget to who... but they said they both got the same boost % for that reason


remove duplicate menu

this fixes the issue reported in https://github.com/Laravel-Backpack/community-forum/issues/849

Please revert these changes, as the menu is now gone on 'lg' and larger screens. I believe it can be fixed instead by adding 'd-none d-lg-block' classes to the account dropdown menu, the one that was removed.
This proposal works on my end, but please test to be sure:
<li class="nav-item dropdown d-none d-lg-block">

thanks, sorry for that.
fixed in 1.2.3


Hide/Disable AppViewBackButtonVisibility on Constructor

Is there any way to hide/disable the Built-In Back button of UWP in the Page or ViewModel Constructor? I'm using the following code in the constructor
SystemNavigationManager.GetForCurrentView().AppViewBackButtonVisibility = AppViewBackButtonVisibility.Collapsed;
But it seems that T10 overrides the visibility that I've set in the AppViewBackButtonVisibility.

Try Bootstrapper.ShowShellBackButton;

I used BootStrapper.Current.ShowShellBackButton = false;, it works, though it now shows the BackButton in T10 PageHeader. I tried to set the BackButtonVisibility of PageHeader to Collapsed but it still shows the BackButton.
I'll try to look for a solution to this one as well.

I don't know the way to hide it in constructor, but you can simply remove Frame binding for each header which loose back button
<controls:PageHeader x:Name="pageHeader" BackButtonVisibility="Collapsed" Content="Header" Frame="{x:Bind Frame}">

To
<controls:PageHeader x:Name="pageHeader" BackButtonVisibility="Collapsed" Content="Header">

Cool, thanks


Support babel's resolveModuleSource custom resolvers

Here is my attempt at #5

Thanks for the PR! I made some adjustments. If those work for you I'll go ahead and merge this. :)

@real34 Whoops, I committed too soon! Don't pull it if you haven't yet. I'm going to amend the commit with some needed changes in a few minutes!

Ok now it should be good. Let me know if that works for you.

It looks much more cleaner indeed! It works fine for me, thank you very much work your help.

I went ahead and published this(2.0.1-rc.0) so you can go ahead and start using it. Over the next few days I'm going to get the tests, dependencies, and a few other things in order and then do a proper release. Thanks again @real34 for all your help with this release!

Awesome! Thank you.


CRYPTO-40:Remove the full qualified package name for shadowed classes

CRYPTO-40
Since we have resolved the class name conflicts, the full qualified name for shadowing class such as Cipher and SecureRandom is longer needed.

LGTM

Also javax.crypto.Cipher.getInstance should changed to Cipher.getInstance

@jerrychenhf thank you for review.
"javax.crypto" & "java.security" are removed for Cipher & SecureRandom.
PR is updated.

+1

@kexianda Thanks for the PR.
The following code in CryptoCipher  should also be updated:
int ENCRYPT_MODE = javax.crypto.Cipher.ENCRYPT_MODE;
int DECRYPT_MODE = javax.crypto.Cipher.DECRYPT_MODE;

@dianfu thanks for your review. updated.

+1


Is there any way to implement this plugin to the ShopAPI?

I've been trying to implement the CaptureAction in an endpoint in the ShopAPI but no chance. What is a best-practice to add the payment redirection when reaching complete_checkout endpoint?
Actually, is this possible (with the Mollie plugin)?

@tudorradubarbu  Did you ever get this to work?

Not really. BUT. Based on this plugin, I've used the $this->mollieApiClient  and used it in my actions/handlers. Super easy to understand if you check the code.

@tudorradubarbu  Great thanks!


Feature/341 social share

#341 Social media Share for Surah.

Deployed to: http://staging.quran.com:32787

Deployed to: http://staging.quran.com:32788

Linter is failing on something I didn't touch

/home/travis/build/quran/quran.com-frontend/src/helpers/Html.js
  21:17  error  Property should be placed on a new line  react/jsx-first-prop-new-line

dw about this: https://travis-ci.org/quran/quran.com-frontend/builds/141903708#L415

Deployed to: http://staging.quran.com:32790

Deployed to: http://staging.quran.com:32791

Deployed to: http://staging.quran.com:32792

Deployed to: http://staging.quran.com:32793

Deployed to: http://staging.quran.com:32795

@mmahalwy reverted back to images.

Thank you @sabeurthabti

@mmahalwy are we using paid version of https://symbolset.com/icons/standard?
see Usage


Managed image sort columns

I added sort functionality to the managed image table. To accomplish this I had to move the rows into state so I can set the sorted rows and update the DOM.

@matoval can you also update in package.json this package to version 3.2.3 so you can use toUpperCase by Karel in here

@matoval we are so close to merge this - just this small changes and we're done.


When adding a category, the product page is not reloaded properly

The Nutri-Score doesn't appear on the main tab immediately
It appears on pull to refresh (or immediately, not sure about that) in the nutrition tab, but the button to add the category is still there.

Will be fixed with https://github.com/openfoodfacts/openfoodfacts-androidapp/pull/2494


Add .idea folder to .gitignore

Folder .idea is similar to .vscode, but for InteliJ IDE.

Thank you.


break out minibatch training lp

Changes

Stochastic EM algorithm returns expected training log probability for each minibatch, with shape (num_epochs, num_minibatches)
Added inner progress bar over minibatches (optional parameter)

Motivation
When the stochastic EM algorithm only returns expected log probability for each epoch, it can be hard to evaluate the rate of training (particularly when the dataset is very large and the algorithm is run over many minibatches). The old returned value can be recovered by retrieving the last expected log probability for each epoch, expected_log_probs[:,-1]

I'm afraid this PR is stale now. We can revisit later @ezhang94!


Loadtest can run different load programs

I noticed the PR is aimed to lwaftr_strafruit. Is that OK? Perhaps you meant lwaftr branch.
LGTM, if test pass.


318 - Removed upper bound protocol validation

It will automatically be negotiated to a valid value using the control
connection and it's one more place in the code that needs to be updated.

+1


Fixing mandatory CA requirements for insecureVerify Flag

[x] All tests passed. If this feature is not already covered by the tests, I added new tests.
[x] This pull request is on the main branch.
[x] I used gofmt for formatting the code before submitting the pull request.

I have read the CLA Document and I hereby sign the CLA


Laura/182 initial hosted se2 docs

Adding docs for hosted SE2 and generally optimizing.

Closing in favor of PR #186


KBFS does not start (and can't reinstall)

Context:  Problem began to occur following a hard-shutdown.
Platform:  Windows 10
Symptoms:

KBFS drive is not available.
GUI fails to load.
Installer fails with "The specified account already exists"

Rebooting does not help.
my log id: 1e48bc10c2043a64ad978c1c

Fixed by uninstalling before attempting to reinstall.

I was unable to fix it by uninstalling it first. Now it just says "Gathering required information...", the meter fills up, then it quits.

CC @zanderz installer broken at least?

@kode54 What does keybase version say? Can you do a keybase log send?

I got it to install successfully by deleting the HKCU\Software\Keybase key, then reinstalling. It also managed to change the KBFS mount to L: instead of K:.