Backport "HBASE-28917 ColumnFamilyMismatchException mixes IA public and private" to branch-2.6

… (#6369)

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 46s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
hbaseanti
0m  0s

Patch does not have any anti-patterns.

_ branch-2.6 Compile Tests _

+1 :green_heart:
mvninstall
3m  2s

branch-2.6 passed

+1 :green_heart:
compile
0m 29s

branch-2.6 passed

+1 :green_heart:
checkstyle
0m 12s

branch-2.6 passed

+1 :green_heart:
spotbugs
0m 33s

branch-2.6 passed

+1 :green_heart:
spotless
0m 46s

branch has no errors when running spotless:check.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
2m 47s

the patch passed

+1 :green_heart:
compile
0m 28s

the patch passed

+1 :green_heart:
javac
0m 28s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 10s

the patch passed

+1 :green_heart:
spotbugs
0m 40s

the patch passed

+1 :green_heart:
hadoopcheck
14m 52s

Patch does not cause any errors with Hadoop 2.10.2 or 3.3.6 3.4.0.

+1 :green_heart:
spotless
0m 45s

patch has no errors when running spotless:check.

_ Other Tests _

+1 :green_heart:
asflicense
0m 11s

The patch does not generate ASF License warnings.

27m 25s

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6412/1/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/6412

Optional Tests
dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless

uname
Linux c33e19801610 5.4.0-195-generic #215-Ubuntu SMP Fri Aug 2 18:28:05 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
branch-2.6 / fc48956d3b9ec278d524c6d7395d6f5c75ddf3cb

Default Java
Eclipse Adoptium-11.0.23+9

Max. process+thread count
79 (vs. ulimit of 30000)

modules
C: hbase-backup U: hbase-backup

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6412/1/console

versions
git=2.34.1 maven=3.9.8 spotbugs=4.7.3

Powered by
Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 25s

Docker mode activated.

-0 :warning:
yetus
0m  4s

Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck

_ Prechecks _

_ branch-2.6 Compile Tests _

+1 :green_heart:
mvninstall
2m 28s

branch-2.6 passed

+1 :green_heart:
compile
0m 18s

branch-2.6 passed

+1 :green_heart:
javadoc
0m 15s

branch-2.6 passed

+1 :green_heart:
shadedjars
4m 48s

branch has no errors when building our shaded downstream artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
2m 21s

the patch passed

+1 :green_heart:
compile
0m 18s

the patch passed

+1 :green_heart:
javac
0m 18s

the patch passed

+1 :green_heart:
javadoc
0m 14s

the patch passed

+1 :green_heart:
shadedjars
4m 43s

patch has no errors when building our shaded downstream artifacts.

_ Other Tests _

+1 :green_heart:
unit
11m 34s

hbase-backup in the patch passed.

28m 30s

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6412/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/6412

Optional Tests
javac javadoc unit compile shadedjars

uname
Linux 73a874178e8e 5.4.0-192-generic #212-Ubuntu SMP Fri Jul 5 09:47:39 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
branch-2.6 / fc48956d3b9ec278d524c6d7395d6f5c75ddf3cb

Default Java
Temurin-1.8.0_412-b08

Test Results
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6412/1/testReport/

Max. process+thread count
2908 (vs. ulimit of 30000)

modules
C: hbase-backup U: hbase-backup

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6412/1/console

versions
git=2.34.1 maven=3.9.8

Powered by
Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m 16s

Docker mode activated.

-0 :warning:
yetus
0m  6s

Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck

_ Prechecks _

_ branch-2.6 Compile Tests _

+1 :green_heart:
mvninstall
3m 52s

branch-2.6 passed

+1 :green_heart:
compile
0m 25s

branch-2.6 passed

+1 :green_heart:
javadoc
0m 19s

branch-2.6 passed

+1 :green_heart:
shadedjars
6m 46s

branch has no errors when building our shaded downstream artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
3m 52s

the patch passed

+1 :green_heart:
compile
0m 26s

the patch passed

+1 :green_heart:
javac
0m 26s

the patch passed

+1 :green_heart:
javadoc
0m 19s

the patch passed

+1 :green_heart:
shadedjars
7m 15s

patch has no errors when building our shaded downstream artifacts.

_ Other Tests _

+1 :green_heart:
unit
15m 28s

hbase-backup in the patch passed.

41m 12s

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6412/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/6412

Optional Tests
javac javadoc unit compile shadedjars

uname
Linux aa15db73679c 5.4.0-195-generic #215-Ubuntu SMP Fri Aug 2 18:28:05 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
branch-2.6 / fc48956d3b9ec278d524c6d7395d6f5c75ddf3cb

Default Java
Eclipse Adoptium-11.0.23+9

Test Results
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6412/1/testReport/

Max. process+thread count
3407 (vs. ulimit of 30000)

modules
C: hbase-backup U: hbase-backup

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6412/1/console

versions
git=2.34.1 maven=3.9.8

Powered by
Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.


Request for adding CNAME as ja.crystal.org

Hello, everyone. We (crystal-jp) are the activities that spread the Crystal in Japan.
We are translating official Crystal document now, so It is soon seems to end.
There are Crystal document in Japanese:

Web: http://gh-pages.crystal.pine.moe/docs/
GitHub: https://github.com/crystal-jp/gh-pages
Cite:

Google Group: https://groups.google.com/forum/?fromgroups#!topic/crystal-lang/YxvtmERLQOs
Issue: #1673

We would like to publish this translated document as subdomain in crystal.org.
We'll maintenance Japanese document, will track to original document,
and plan to support Japanese developer of Crystal. And, we'll host and publish this document in GitHub pages.
Would you add CNAME record (= ja.crystal.org) in crystal.org domain ? @waj  @asterite
Example of DNS record:

ja.crystal.org.  IN  CNAME  crystal-jp.github.io.

https://help.github.com/articles/setting-up-a-custom-domain-with-github-pages/
Thank you.

As described in above and also in #1673, crystal-jp project is about to finish the translation of the document into japansese. Then, we want to publish it as soon as possible because (sorry but it's just because of our circumstances) we are going to hold a meetup for about 80 crystal users in japan next week, and we would like to introduce that there are docs available in japanese to participants.
Could you please consider to make ja.crystal.org. to be a CNAME for crystal-jp.github.io. ?

I just added the CNAME. Note that the correct host name is ja.crystal-lang.org. Should we also set jp.crystal-lang.org?

BTW... crystal-lang.jp seems to be available :wink:

Thank you!!!

Note that the correct host name is ja.crystal-lang.org. Should we also set jp.crystal-lang.org?

oh... That's right! I had been mistaken.

BTW... crystal-lang.jp seems to be available

I'll think about it :smiley:

Juan, could you create a br. or pt-br subdomain to point to
http://alanwillms.github.io/crystal-pt-br-docs ?
I'm working on the Brazilian Portuguese translations of the docs.
2015-10-13 14:25 GMT-03:00 Pine Mizune<EMAIL_ADDRESS>
Thank you!!!
Note that the correct host name is ja.crystal-lang.org. Should we also
set jp.crystal-lang.org?
oh... That's right! I had been mistaken.
BTW... crystal-lang.jp seems to be available
I'll think about it [image: :smiley:]
—
Reply to this email directly or view it on GitHub
https://github.com/manastech/crystal/issues/1719#issuecomment-147784132.

I just added the CNAME. Note that the correct host name is ja.crystal-lang.org. Should we also set jp.crystal-lang.org?

Thank you! we appreciate your kindness, and yes, please set jp.crystal-lang.org to point to our website as well.

I'm closing this since ja.crystal-lang.org already works. Please reopen if you need jp or create other issues if you need other subdomains.


ENH: fixed window length error and updated pipeline calls

Pipeline improvements
Window length sorted for month
Month forward sorted out (now actually forward returns)

Can you figure out the conflict here? You might have branched from an older version of master without pulling the latest.

Closing because I think this is now outdated.


Deleting of created project blocked (rmtree and permission denied)

During last week this was working, but I think that now this cookie-cutter is creating a directory and then deleting it, failing in case user if not executing via sudo (to be confirmed).
(test) rio@NB085081:cookiecutter-scverse$ cookiecutter https://github.com/scverse/cookiecutter-scverse
project_name [project-name]: test
package_name [test]: test
project_description [A very interesting piece of code]: test
author_full_name [Your Name]: Ignacio Ibarra
author_email<EMAIL_ADDRESS><EMAIL_ADDRESS>github_user [your_github_username]: ilibarra
project_repo [https://github.com/ilibarra/test]: https://github.com/theislab/test
Select license:
1 - MIT License
2 - BSD 2-Clause License
3 - BSD 3-Clause License
4 - Apache License Version 2.0
5 - GNU General Public License Version 3
6 - Unlicense
Choose from 1, 2, 3, 4, 5, 6 [1]: 1
error: unknown option `initial-branch=main'
usage: git init [-q | --quiet] [--bare] [--template=<template-directory>] [--shared[=<permissions>]] [<directory>]

    --template <template-directory>
                          directory from which templates will be used
    --bare                create a bare repository
    --shared[=<permissions>]
                          specify that the git repository is to be shared amongst several users
    -q, --quiet           be quiet
    --separate-git-dir <gitdir>
                          separate git dir from working tree

Traceback (most recent call last):
  File "/home/rio/miniconda3/envs/test/lib/python3.7/site-packages/cookiecutter/generate.py", line 237, in _run_hook_from_repo_dir
    run_hook(hook_name, project_dir, context)
  File "/home/rio/miniconda3/envs/test/lib/python3.7/site-packages/cookiecutter/hooks.py", line 131, in run_hook
    run_script_with_context(script, project_dir, context)
  File "/home/rio/miniconda3/envs/test/lib/python3.7/site-packages/cookiecutter/hooks.py", line 115, in run_script_with_context
    run_script(temp.name, cwd)
  File "/home/rio/miniconda3/envs/test/lib/python3.7/site-packages/cookiecutter/hooks.py", line 87, in run_script
    'Hook script failed (exit status: {})'.format(exit_status)
cookiecutter.exceptions.FailedHookException: Hook script failed (exit status: 129)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/rio/miniconda3/envs/test/lib/python3.7/shutil.py", line 496, in rmtree
    os.rmdir(path)
PermissionError: [Errno 13] Permission denied: '/mnt/c/Users/ignacio.ibarra/Dropbox/workspace/cookiecutter-scverse/test'
During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/rio/miniconda3/envs/test/bin/cookiecutter", line 8, in <module>
    sys.exit(main())
  File "/home/rio/miniconda3/envs/test/lib/python3.7/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/rio/miniconda3/envs/test/lib/python3.7/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/rio/miniconda3/envs/test/lib/python3.7/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/rio/miniconda3/envs/test/lib/python3.7/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/rio/miniconda3/envs/test/lib/python3.7/site-packages/cookiecutter/cli.py", line 152, in main
    skip_if_file_exists=skip_if_file_exists,
  File "/home/rio/miniconda3/envs/test/lib/python3.7/site-packages/cookiecutter/main.py", line 106, in cookiecutter
    output_dir=output_dir,
  File "/home/rio/miniconda3/envs/test/lib/python3.7/site-packages/cookiecutter/generate.py", line 294, in generate_files
    repo_dir, 'pre_gen_project', project_dir, context, delete_project_on_failure
  File "/home/rio/miniconda3/envs/test/lib/python3.7/site-packages/cookiecutter/generate.py", line 240, in _run_hook_from_repo_dir
    rmtree(project_dir)
  File "/home/rio/miniconda3/envs/test/lib/python3.7/site-packages/cookiecutter/utils.py", line 34, in rmtree
    shutil.rmtree(path, onerror=force_delete)
  File "/home/rio/miniconda3/envs/test/lib/python3.7/shutil.py", line 498, in rmtree
    onerror(os.rmdir, path, sys.exc_info())
  File "/home/rio/miniconda3/envs/test/lib/python3.7/site-packages/cookiecutter/utils.py", line 26, in force_delete
    func(path)
PermissionError: [Errno 13] Permission denied: '/mnt/c/Users/ignacio.ibarra/Dropbox/workspace/cookiecutter-scverse/test'

It seems that error: unknown option initial-branch=main'is due togitversion. I usegit version 2.25.1`. What's the minimum one should consider? Thanks.
https://github.com/scverse/cookiecutter-scverse/blob/main/hooks/pre_gen_project.sh

This is solved by having the following two package versions:
* [ ]  `git >=2.38`

* [ ]  `python >=3.8.`

Indicating / checking the versions of those two and warnings would be ideal. Which file could I modify?

I think that it could be checked here: https://github.com/scverse/cookiecutter-scverse/blob/main/hooks/pre_gen_project.sh
Basically, before the git init command is run -> check versions -> error out with helpful error messages if your conditions that you defined are not met.

I am adding this as minimal dependencies to the README in #17.
Is that good enough or do we want checks?

This hasn't come up anymore... closing for now.


Added raml-ruby-client-generator

raml-ruby-client-generator can generate API client from a RAML file.

Please be kind to the ones who built this list before you and add your resource at the end of the current list.

Oops. It's moved.

Thanks! :)


Fragement name instead of mutation name displayed

When going through this site: https://demo.saleor.io/ there is a slight mistake in that the name of fragment is displayed instead of the name of the mutation
Bug Behavior:
Mutation name displayed as "Price"
Price is a fragment used by the mutation not he main mutation.
Expected Behavior:
Mutation name displayed as "UpdateCheckoutShippingMethod"

@vkhansen thanks for raising this. I can recreate the issue and have a PR ready to go. I'll release to dev store this week.
https://github.com/warrenday/graphql-network-inspector/pull/8


Add Error Check for TempAuthServer.wait_for_request

e.g. incorrect state token or redirect URI mismatch

Could pass validate_query_params={"state": "XXX"} as arg to wait_for_request


Caption of ToggleSwitchForm is given too little space

Steps to reproduce

Go to https://primer-lookbook.github.com/view-components/lookbook/inspect/primer/forms/example_toggle_switch_form
Decrease the window size, so that the label is wrapped

Buggy behaviour

The caption is limited to width of the label

Expected
Taken from the docs, the caption should span the whole witdh of the toggle switch form and not be limited to the width of the label

Nice catch! We'll take a look, I think we could update the container to not get squashed like that


Text embedding code failing for single prompt

When I try to use clap_model.get_text_embedding() on an array with a single prompt in it, the call fails with an error in the Roberta tokenizer. It seems that it's confused about the shape of the array unless there's more than one element in it.
File ".../transformers/models/roberta/modeling_roberta.py", line 802, in forward
batch_size, seq_length = input_shape
ValueError: not enough values to unpack (expected 2, got 1)

Hi, are you inputting the string to the function? In that function, we expect a list. So you could make that string as a text (length 1). Please let us know how you do it.

I'm inputting an array of length 1.
prompt = "Text prompt"
prompts = [prompt] * args.batch_size
text_embeddings = latent_diffusion_model.embedder.get_text_embedding(prompts)

with args.batch_size is set to 1, this code fails. It works with larger batch sizes.

I can confirm this. I am facing the same issue. Thanks

#105 should fix this.  Are there plans to merge it?

need more than 2 text to get_text_embedding

need more than 2 text to get_text_embedding

already made it a list (all_text_list[0:1]) but not work at all


feat: add refs support for various fields

Issue #, if available:
Closes aws-controllers-k8s/community#1513
Closes aws-controllers-k8s/community#1860
Description of changes:
The following fields have had refs support added:

CacheSubnetGroup.SubnetIDs: ec2.Subnet.Status.SubnetID
ReplicationGroup.CacheParameterGroupName: CacheParameterGroup.Spec.CacheParameterGroupName
ReplicationGroup.CacheSubnetGroupName: CacheSubnetGroup.Spec.CacheSubnetGroupName
ReplicationGroup.SecurityGroupIDs: ec2.SecurityGroup.Status.ID

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

/ok-to-test

Ideally we want to add e2e tests for references that are managed by the same controller (elasticache-controller). But this is non-blocking since the controller is not GA.

/lgtm


[WIP] migrate backup hooks tests

Signed-off-by: Steve Kriss<EMAIL_ADDRESS>@nrb @carlisia @prydonius in migrating the tests related to backup hooks, I didn't really see a better way to structure these than using a mock for the PodCommandExecutor. The interaction between pkg/backup and the PodCommandExecutor is really simple - just a single method call - and this test is just verifying that the method is called when a hook is configured. WDYT about this?

added a couple more test cases

If the mock usage seems reasonable, then this can be reviewed since I added a couple more test cases. There's probably an opportunity to add a few more test cases if someone works with the hooks code later, but TBH I think moving on to other things (e.g. pkg/restore) is a better use of time right now.

merge conflict, 1 sec


doc: second return value of syscall.Syscall needs to be documented

I'm using Go 1.11.2 on 32 bit Windows.
Current documentation of Syscall* doesn't give any sense about second return value r2. And in most cases, this value seems not needed at all. But Go codebase internally uses it:
https://github.com/golang/go/blob/5538a9a34fc4e395967c0233aab5bdde0cebbf9b/src/runtime/syscall_windows_test.go#L103-L118
And if I understood correctly, the second value is needed only when dealing with APIs that use 64 bit values(e.g. VerSetConditionMask). Since uintptr is 32 bit integer and can't hold those APIs' result, the second return value r2 is used. But there isn't any documentation about return values from syscall.Syscall* that tells which one holds highest 32 bits, etc.
Also the arguments for those APIs seems to be merged(see code above) when dealing with 64 bit values. Without knowing that, I met a horrible panic that doesn't tell any meaningful reason. So it'd be good if these behaviors are well documented.

I guess it is already fixed. please check current documentation

@Piyushhbhutoria Still I don't see any documentation about r2. What do you mean by 'it is already fixed'?

how do i get started on this? can you explain how to recreate your panic and where to edit the docs?

@Piyushhbhutoria I don't have 32 bit Windows machine now so I'm afraid I cannot show you my code that created panic. Could you try yourself passing only one uintptr argument for VerSetConditionMask's first parameter on 32 bit Windows?

I don't have a 32-bit operating system either, I have a mac.


Sway doesn't render colors properly with 10bit depth

I am testing a new patch series for mesa0. I can see in the logs wlc telling me it choose RGB101010, but sway doesn't render colors correctly.
Testing with Sway 7553103 and Archlinux 4.12.10.

What do you mean "Doesn't render colors correctly"? Are all colors off, sway config colors or something else?

This is a WLC issue, and WLC doesn't support 10 bit colours.


model and queries inconsistent

main and queries do not align between the query file path and how it is called in the template. Aligning these would make it easier to understand and remember.
lebot@aquarius:/var/www/prov-wg/prov-lodspeakr/components/services/provrdf-owl-coverage$ grep "select" queries/main.query
select distinct ?construct ?yes

lebot@aquarius:/var/www/prov-wg/prov-lodspeakr/components/services/provrdf-owl-coverage$ grep "row in" html.template
      <ul> {% for row in models.main %}

I don't understand what you mean:
Assuming you have a query called filename.query, all the data is in models.filename while the first values are in first.filename.

(oh, first works again?)
queries/main.query  does not resemble models.main, making it more difficult for beginners to use the tool (because they have to remember to "switch" queries to models when they want to access it in the view template. On the other hand, if model/main.query  and models.main or queries/main.query  and queries.main were used for the filename and view template access variable, this would be easier to learn.


Docker: Conditionally start ssh agent for SSH downloads

We want to run ORT within a Kubernetes pod, but we also need to be able to resolve repositories by SSH since some packages have a download link as SSH-link.
Previously, we would start an agent on the host and then just share the socket with the container, but this is not as simple in a Kubernetes setup. Therefore we need a way to open an ssh agent within the container, which is what this PR is about.
The PR will only change anything if an identity file is mounted into the Docker container.

I'm not sure if this is useful default behavior for other users for the Docker image, did you consider using a custom entrypoint script in your setup instead?

Commit message:

This allows setting up SSH downloads in containers where SSH socket can
be shared from the host system.

Shouldn't this say "cannot be shared"? Also, naming Kubernetes pod as a concrete example would be nice, IMO. Maybe even with a link to docs explaining why this is not possible / supported in a Kubernetes setup.

@mnonnenmacher That's what we were wondering. I'm also not sure, so I'll close this for now.


fix/rollover-not-working: Don't compare with slash at the end

Fixes #94

Hey, I believe the root cause here is that Obsidian allows multiple definitions of the pathPrefix for daily notes. To tackle this, we have to think about the path to a daily file, which consists of two components:

The path prefix coming from the daily settings
The filename for the daily file that might be a folder or a path name → Always a moment timestamp

This gives us three options we need to cover:

pathPrefix/ + 2022/Feb/01.md
pathPrefix + 2022/Feb/01.md
3./pathPrefix/ + 2022/Feb/01.md

In order to get the moment timestamp from the path, we have to remove the pathPrefix from the complete path and cover all three cases, since they're all valid settings in Obsidian.
I reckon that's also why this sort of issue surfaces once we address it for one of the other cases. What do you think about this? Am I missing something?

Now that I think of those issues again, I think the discussion should actually be a different one: path-handling is becoming too confusing. We should refactor the settings-parsing, so that we can rely on the daily-notes-path being of the form path/to/ with a slash at the end.
this way we don’t have to parse the notes-path when the rollover-function runs. Paths will be handled the same way everytime then.

Sorry for taking so long. I ran some tests locally and it still works as expected :+1:
Going to implement the path handling you proposed, in v2, in the summer when there is more time available.


The query must be of type ObjectQuery or DbQuery. Parameter name: source.

I'm getting the error above despite a really simple query:
using (DataContext db = new DataContext())
{
db.Products.Where(x => x.PRODUCT_ID == "05501").Update(x => new Product() { CurrentState = "TEST STATE" });
}
How is the source query null when we have this segment
"db.Products.Where(x => x.PRODUCT_ID == "05501")"?
It appears in BatchExtensions.cs it's hitting this code path:
if (sourceQuery == null)
throw new ArgumentException("The query must be of type ObjectQuery or DbQuery.", "source");
Is there a bug in this EF.Extended or can anyone assist?
Much appreciated!

Hi @neoswarm
You can check the type of your db.Products.Where(x => x.PRODUCT_ID == "05501"), is your db.Products is a dbSet<T>?
check the value returned by this statement db.Products.Where(x => x.PRODUCT_ID == "05501").ToObjectQuery(), if it is null, there is someting wrong when you define your Products.,.


change(deps): Initializes cargo vet in Zebra

Motivation
We want to use cargo vet to audit Zebra's dependencies.
Solution

Ran cargo vet init

Added zcash/zcashd, Google, and Mozilla's audit.tomls as trusted audit imports

Follow-up Work

Use cargo vet to review updates in https://github.com/ZcashFoundation/zebra/pull/8640
Use cargo vet when reviewing Zebra's future dependency updates

PR Author's Checklist

[x] The PR name will make sense to users.
[x] The PR provides a CHANGELOG summary.
[x] The solution is tested.
[x] The documentation is up to date.
[x] The PR has a priority label.

PR Reviewer's Checklist

[x] The PR Author's checklist is complete.
[x] The PR resolves the issue.

@conradoplg added you as a reviewer on this one as you might be interested in this also for FROST

Cool! I'm interested in how you will incorporate it in your workflow


Startup validation

There are two areas where startup validation can be improved:

Parsing the options that were passed in (mainly for JS)
Validation the executable schema (no resolvers missing etc.)

This prevents errors on first query execution.

From what I got from the codebase —at least at the lambda side — all we can do is throw new('Bad arguments, no 🍩 for you') into the constructors. All the graph resolving happens inside apollographql/apollo-server lambda, we have no control into that and I am sure no one want to rewrite how to manage lambda code here.
https://github.com/prismagraphql/graphql-yoga/blob/9fc308a0ea4968831fc091f26ce10d422bf6af94/src/lambda.ts#L18-L59


Add registry upload manifest command

I recommend using gofmt to format your code before creating PRs. It's likely your editor has some kind of plugin that can be set up to do this automatically.

Done.


chore(node): drop support for 4.4.0, run travis job on node 7

chore(node): drop support for 4.4.0, run travis job on node 7

@cjhowald @dami-gg This would fix #344 failing build when merged

👍

👍


Create xcm_btp_integration.md

XCM BTP W3F open grant proposal
Project Abstract
A proposal to integrate BTP with XCM for extended cross-chain interoperability beyond Polkadot.
The aim is to connect ICE parachain and ICON’s BTP network hub with other parachains on Kusama/Polkadot by implementing an XCM/BTP services contracts and core interfaces that adapt a logical representation of XCM instructions on BTP-enabled chains.
Short-term scope is to base implementation needed to enable arbitrary messaging and asset transfers between ICON <> ICE, as well as concept validation for multi-chain messaging.

If your application is a follow-up to a previous grant, please mention which one in the first line of the abstract and include a link to previous pull requests if applicable.

For which grant level are you applying?

[ ] Level 1:  Up to $10,000, 2 approvals
[x] Level 2:  Up to $50,000, 3 approvals
[ ] Level 3:  Unlimited, 5 approvals (for > $100k Web3 Foundation Council approval)

Application Checklist

[x] The application template has been copied, renamed ( project_name.md) and updated.
[x] I have read and understood the FAQs, application guidelines and announcement guidelines.
[x] A BTC, Ethereum (USDT/USDC/DAI) or Polkadot/Kusama (aUSD) address for the payment of the milestones is provided inside the application.
[x] I have read and acknowledge the terms and conditions.
[x] The software delivered for this grant will be released under an open-source license specified in the application.
[x] The initial PR contains only one commit (squash and force-push if needed).
[x] The grant will only be announced once the first milestone has been accepted.
[ ] I prefer the discussion of this application to be in a private Element/Matrix channel. My username is: @_______

How Did You Hear About our grants program?

[ ] Social Media
[ ] Hackathon
[x] Personal Recommendation
[ ] Substrate Builders Program
[ ] Investor/VC
[ ] Online Search
[ ] Other: _______

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Thanks, @Noc2 for reviewing and the feedback. It sounds like I may need to rewrite some parts of the grant to make it more clear that the primary focus is use cases of XCM over BTP, these are some examples from the project details.

Cross-chain messaging between Parachains and other chains outside Polkadot
Multi-assets transfers between blockchains on ICON BTP hub and Polkadot parachains
Ability of any BTP enabled chain to control sovereign account holding a reserve for its native asset

As for Moonbeam, it's one of the many layer1 networks that's been deployed on ICON's BTP networks Testnet hub, and so the integration of other networks I've mentioned in the overview section including Near, BSC, and Harmony currently in-flight -- we anticipate many layer1 networks to connect via BTP. Happy to emphasize a bit more in the development status section to clarify that Moonbeam is an example.
We are building ICE network to become a parachain on Polkadot. It supports both Wasm (ink!) and EVM. EVM is necessary for legacy applications and the wide spectrum of developers who would develop cross-chain applications on ICE and Polkadot. In our future plans, we would like to rewrite most parts of BTP as pallets so new BTP use cases and services can be implemented in ink!.
As for milestone 2, we can reduce the scope of the demo/workshop, and remove the deployment from the roadmap.
Let me know your thoughts and further elaborate on why we think this is an interesting proposal.

Thanks, @Noc2. This sounds good, we have a few ideas on how could approach this integration with ink! - but will require a redesign of some of the relay components.
I understand why this BTP might seem like it contends snowbrige and IBC -- but BTP is not focused on Ethereum, there are other non-Ethereum-based blockchain integrations on the roadman for ICON's BTP.

@mohamedelshami note that we try to keep only active applications open and usually close them if there's been no activity in over 2 weeks. Are you planning to update your current one, or do you prefer to apply again at a later date?

I’m closing this PR due to inactivity. But I’m happy to reopen it as soon as you are ready.

hey @alxs @Noc2 apologies for the late response, we haven't incorporated the discussed changes above in this proposal yet. It makes sense to close it now, and we will reopen once we've made the changes and updated the scope.


⚠ Acceso a Mercados has degraded performance

In ce9e8a5, Acceso a Mercados ($URL_AAM) experienced degraded performance:

HTTP code: 503
Response time: 10366 ms

Resolved: Acceso a Mercados performance has improved in 295edf6 after 9 minutes.


Add Cloud Redis generated client

Will need to wait for 0.38.0 release of proto-client

Merging, as the build error is related to: https://github.com/GoogleCloudPlatform/google-cloud-php/issues/1048


Created a static image folder.

👷 Deploy Preview for poisoned-sweets processing.

Name
Link

🔨 Latest commit
ab072323dad7692be44949e0bd93b3f0f94b36d4

🔍 Latest deploy log
https://app.netlify.com/sites/poisoned-sweets/deploys/6527e9b1f9a9800008736c2a


Item Categories

Create a categories feature for the items.
It should include: Displaying the item category on the items list Filter items by category on the items list * Chose a category dropdown on the new item view

First we will probably need to change the way we are constructing the items list. It's getting slow since it's loading all items at the same time.


Don't overwrite old refresh token if STS does not return a refresh token in response

see https://github.com/IdentityModel/IdentityModel.OidcClient2/issues/79

seems to be fixed.


Adds action to web

Sonner already supports action - adding parity on web.

hey @nandorojo - anything else you need from me here to get this merged?

I think you need to push your changes

Sorry - I might just not understand merging into someone else's repo from my forked repo well enough. I have a 2nd commit in this PR that adds the comment. Are you unable to see that on your end?

Oh, you didn’t put the comment in the right place. It should be where I left my review and in the same format as the other comments.

@nandorojo ohhh I'm sorry about that - got confused between the two files since the keys were the same keys. Just opened a PR with a cleaner commit https://github.com/nandorojo/burnt/pull/41


How to use with Webpack?

Hi,
how do I use your library with webpack?
thanks

Hi, sorry but I don´t have much experience with webpack so not sure what are the specific steps.
If you find the answer please share it here, it could be useful for someone else.


operator project-quay (3.9.0-nightly.20230606)

Add 3.9.0-nightly.20230606 to community operators

/merge possible

/merge possible


Support for websocket

Need websocket support

can you provide more detail on what exactly is missing now? The actual proxy used by reporxy is stdlib's httputil.ReverseProxy and it supports WS for many years (since v1.2)
As far as I recall @bobuk actually used reproxy with SW and may provide some inside

I use reproxy for public hasura api (graphql server). Hasura user websocket for subscrubtion query.
Message in console when i make request to hasura endpoint:
reproxy           | 2021/05/22 15:21:06.701 [WARN]  {lgr/adaptor.go:16 lgr.(*Writer).Write} http: proxy error: can't switch protocols using non-Hijacker ResponseWriter type struct { http.ResponseWriter }
reproxy           | 2021/05/22 15:22:09.156 [DEBUG] {proxy/proxy.go:226 proxy.(*Http).proxyHandler.func2} proxy to http://<IP_ADDRESS>:8080/v1/graphql
reproxy           | 2021/05/22 15:22:09.163 [WARN]  {lgr/adaptor.go:16 lgr.(*Writer).Write} http: proxy error: can't switch protocols using non-Hijacker ResponseWriter type struct { http.ResponseWriter }
reproxy           | 2021/05/22 15:23:09.300 [DEBUG] {proxy/proxy.go:226 proxy.(*Http).proxyHandler.func2} proxy to http://<IP_ADDRESS>:8080/v1/graphql

My docker-compose file:
version: "3"
services:
  reproxy:
    image: umputun/reproxy
    container_name: reproxy
    hostname: reproxy
    ports:
      - "80:8080"
      - "443:8443"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    environment:
      - DEBUG=true
      - TZ=Europe/Moscow
      - DOCKER_ENABLED=true
      - MAX_SIZE=0
      - GZIP=true

......

  graphql-engine:
    image: hasura/graphql-engine:v2.0.0-alpha.10
    depends_on:
      - "db"
    restart: always
    ports:
     - <IP_ADDRESS>:8080:8080
    env_file:
      - .env_main
    labels:
      - "reproxy.route=^/api/hasura/(.*)"

thx for the details. Looks like some of the handlers redefine ResponseWriter without implementing Hijacker interface.
I have two suspects - gzip handler or the metrics handler. Most likely this is due to metrics, which are partially enabled (but not exposed) by default. I have changed the default to eliminate them completely. The master image built and you can test it by changing image: umputun/reproxy to image: umputun/reproxy:master
Let me know if helped

thx, master version is working.
P.S>I checked before with gzip=off/on and it,s no helped.

cool, I'll add a fix to metrics middleware

FYI: just released a hotfix v0.6.2


feat: folder output

Correlate this with https://github.com/lucide-icons/lucide/pull/1223#pullrequestreview-1532425055

For file-code to work, we would have to move <> up by 1px.

Is this one ready @jguddas ?


Tests for ICAT 5 Mappings

Description:
While writing the endpoint tests, I've skipped some of them that rely on ICAT 5 mappings. When ICAT 5 is released, we should unskip these tests, but only if the ICAT version is less than 5. We can get the version number through the client class. This might also be relevant for the tests on the mappings too.
Acceptance criteria:

[ ] Detect ICAT Server version and skip certain tests if they require ICAT 5 to pass

For the tests to work, there will need to be test data added to the generator script for the new entities meaning the new entities will need to be added to the DB backend. Is it easier to rewrite the generator script using Python ICAT at this point?


Updated CloudAux for ELB/ALB Fix

Fixed a bug with the Python 3 support in CloudAux for ALB/ELBs

Closes #1242

Coverage remained the same at 68.64% when pulling 9ffb5c8f2b23dd1faf3bf53d7abf071c29c015fb on py3fix into d97c15508b14692a1b781f36007bee51c9245581 on develop.


Support for Indian Rupee Sign ₹

Need support for Indian Rupee Sign ₹ please.

@CreativeWolf could you please elaborate on issue when you use this sign?

You'd need to add localization for Indian: https://github.com/allcount/allcountjs/blob/master/docs/configuring.md
Please see localization section.

Thank you for that clarification.


Missing new line () in MDX

Description
In Markdown (and MDX), it should be possible to make HTML
 by adding multiple space characters ( ) before a new line (\n) but it doesn't seem possible to do so with Tina.
Steps to reproduce
In a Markdown / MDX file, add something like
first line
second line

Expected result
The resulting AST should be something like:
{
  "type": "root",
  "children": [
    // ...

    {
      "type": "p",
      "children": [
        {
          "type": "text",
          "text": "first line"
        },
        {
          "type": "break",
        },
        {
          "type": "text",
          "text": "second line"
        },
      ]
    },
   // ...

The rendered HTML should contain:

Actual result
It's impossible to have a line break from Markdown / MDX content.
Environment
Tinacms Package Versions

├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>

Tools

Tool
Version

browser
N/R

os
linux

node
v16.6.2

npm
8.4.0

In this case, the markdown is correct, however, the editor is not properly handling.

In that case, the markdown was "edited" with the editor (adding a few "spaces" and "SHIFT + RETURN" - as I imagine the most common), saved and its result in the markdown file is... "strange"


When a note has more than 3 arguments, the program crashes.

Describe the bug
Describe what happened.
When i was waiting for it to convert, when it got to 42%... It crashed.
Error message
If you got an error message, post it here.
Traceback (most recent call last):
File "fnf_converter.py", line 528, in exportAsOsz
File "fnf_converter.py", line 153, in exportOsuFile
File "fnf_converter.py", line 249, in getBPMList
File "fnf_converter.py", line 269, in getNotesAll8K
File "fnf_converter.py", line 306, in getNotesPlayer
TypeError: '<' not supported between instances of 'str' and 'int'
Screenshots
If applicable, add screenshots to help explain your problem.
(this screenshot was taken recently)

OS and version:
Tell us on what you ran the program (Window, Linux, ...) and which version you used.
Windows 10 Insider Preview Pro 21h1
Python version: 3.10
(oh wait i get what version means now) 1.1.1 (latest)
Used files:
If you input files in this program, post them here (just drag the files in this box), it would be useful to understand how the bug happened.
they're here since i can't really attach jsons
screwed.zip
Additional context
Something to add?
no

Found in v1.2.0


VFL with Safe Area

Hi, been using your VFL functions a lot and they are very helpful. Having to adapt most of my apps to the iPhone X, I realized that your functions do not take the newly introduced Safe Area into account. Any plans on updating?

Now compatible with Swift 5. However, I am deprecating this library in favor of LBTATools.


[BUG] Random ] get added to GraphQL mutation input

I hope this is no duplicate, but I couldn't find any similar issue.
My graphql input gets altered by bruno resulting in a badly formatted graphql mutation.
This just started occuring since I updated to OSX Sonoma (but I can't explain why that should be the reason).
I'm on 1.3.2, but it also occured on 1.3.0.
This is my mutation:
mutation {
  authenticate(input: { native: { username: "superadmin", password: "superadmin"}}) {
    __typename
    ... on CurrentUser {
      id
      identifier
    }
  }
}

Resulting query:
{"query":"mutation {\n authenticate(input: { native: { username: \"superadmin\", password: \"superadmin\"]}}) {\n __typename\n ... on CurrentUser {\n id\n identifier\n }\n }\n}","variables":{}}

You can see that after the password value a "]" is added. The schema does not accept an array as input.
"superadmin"]}
Any ideas are appreciated.

Yes, I can confirm that adding spaces between the brackets fixes the issue.

Thanks for checking @Its-treason, it's indeed caused by that change (was able to reproduce it). I'll fix it 👍


Migrate table definitions from Hive to Mimir Metadata

Presently, Mimir's Spark backend uses Hive to store table definitions.  This means that:

Tables need to be explicitly loaded into spark via HDFS/S3.  They can't be used in-situ on the local file-system. Apart from the usability concerns, this additional step slows down a user's access to data (and makes dynamic loading strategies a'la NoDB harder).
Hive manages schema metadata, requiring type translations, and causing a metadata split between Mimir's internal store and itself.
Mimir poops out a dozen directories, files in the working directory whenever files are created.

It would be useful to create an infrastructure within Mimir for manually managing table definitions / table schemas, etc...  (with the option of reading from Hive as well if configured to do so).  Specifically:

[ ] CREATE TABLE should no longer invoke Spark (purely meta-data based)
[ ] Table mutators (UPDATE, INSERT, etc...) should no longer invoke Spark
[ ] LOAD should no longer trigger a write to Spark (though the schema detector pipeline will still read from Spark).  Notably, LOAD should store the source URL, source format, and any other information needed to create a Spark DataFrame
[ ] CREATE MATERIALIZED VIEW and ALTER VIEW MATERIALIZE should materialize data to a configurable location (e.g., local filesystem / hive / s3) in a configurable format (e.g., scala-native binary / csv / json / etc...)
[ ] Classes in the compilation pipeline (e.g., mimir.compiler.Compiler) should dynamically create the appropriate type of Spark DataFrame as needed.
[ ] Add a caching layer?
[ ] Add a facility to connect external data sources with their own schema definitions (e.g., Hive, JDBC, etc...).  Perhaps use the same schema field used by Adaptive Schemas to hide these imports behind a distinct namespace.

A good question for discussion is how to implement table mutators in Mimir.  Clearly, updates to tables should be registered from Mimir's side.  However, it's worth asking whether such updates should be propagated back to the source.  Tagging @mrb24 and @lordpretzel for feedback.
There's really only one major argument for propagating: It keeps the view of the data from the backend in-sync with the view of the data from Mimir.
Conversely...

Spark/Hive isn't really update-friendly.  We can make updates waaaay faster.
Handling updates in Mimir gives us the ability to track and query provenance of updates, a'la GProM, eventually.
Handling updates in Mimir gives us the ability to add MAYBE { UPDATE | INSERT | DELETE } operations in the future (i.e., a non-deterministic update/insert/delete).
We don't have any active use cases that would benefit from keeping the backend in-sync with the frontend.  Even those cases that might would probably benefit more from an UNLOAD operation.

Which leaves the question of how to implement this simulation of updates.  My own proposal would be to adopt a GProM-like versioning scheme, where each version is defined as a view over the previous version of the table.  This means that we'd need to keep a log of updates to the table in the metadata backend.  This could become a performance bottleneck in the longer term, but we could both keep a log and cache the most recent version of the table.

virtualizing the updates we can batch them before we apply them. Also we can combine this with other lossless versioning at the storage level, e.g., we can make the storage versioned by adding a version column. Updates are kept in the log forever, but every X updates (e.g., X = 1000) we append the new row versions to the table stored in HDFS. This would work like a DB with time-travel + an audit log which GProM uses in Oracle to replay history with or w/o provenance tracking.

After exploring the code a bit, it seems like this should be doable easily if broken up into a read-only step and a read-write step. For the first read-only step, we would support only LOAD.  CREATE TABLE, UPDATE, INSERT, and DELETE would error.
At present mimir.backend.SparkBackend serves multiple roles

Managing Spark-related context (which, due to Spark limitations has to be global).
Providing a wrapper around mimir.algebra.spark.OperatorTransforms, including several utility methods like zipWithIndex and an entry point to the compiler logic.
Implementing a data interface (readDataSource, materializeView).

I propose the following changes

Create a new object (e.g., mimir.exec.spark.MimirSpark) to take on managing global Spark-related state.  If we need to have a single global spark context, we may as well have it in one place rather than having this strange hybrid global/local object SparkBackend.
Integrate compilation-related methods into mimir.exec.Compiler.  At this point, we're tightly enough coupled with Spark that it doesn't make sense t abstract these behind a layer of indirection.
Define a new interface: SchemaProvider, replacing QueryBackend, as a wrapper around components that can define sets of tables.  SchemaProviders would be registered with Database at startup (i.e., in Mimir or MimirVizier) or possibly at runtime.

Examples of SchemaProviders include:

HiveSchemaProvider: Tables already defined in the Spark runtime.
LoadedTableSchemaProvider: LOADed tables.
ViewSchemaProvider: LOADed tables.
AdaptiveSchema

A schema provider would be registered with

A name (e.g., for a provider named foo providing barTable, we'd write  SELECT * FROM foo.barTable
Whether or not the provider should be searched for tables queried without a schema (i.e., SELECT * FROM barTable).

sealed trait TableImplementation;
case class DataFrameTable(df: DataFrame) extends TableImplementation
case class ViewTable(query: Operator) extends TableImplementation
case class IteratorTable(iter: mimir.exec.result.ResultIterator) extends TableImplementation

trait SchemaProvider {
  // Implementations define these...
  def listTables: Iterable[ID]
  def tableSchema(table: ID): Option[Seq[(ID, Type)]]
  def implementation(table: ID): Option[TableImplementation]

  // These come for free but may be overridden
  def listTablesQuery: Operator =
    HardTable(
      Seq(ID("TABLE"), TString()),
      listTables.map { Seq(_) }.toSeq
    )
  def listAttributesQuery: Operator = ???
}

For step two we can define traits that indicate that the provider can accept bulk writes / view materialization, or updates/inserts/deletes.

A quick chat with @mrb24 led to a revision to the SchemaProvider API.  The limiting factor is that depending on situation, different interfaces might be appropriate: In some cases we may want the full query provenance (i.e., we want a view), while in others we want just the dataframe.  The idea would be to expose each access path explicitly:
trait SchemaProvider {
  // Implementations define these...
  def listTables: Iterable[ID]
  def tableSchema(table: ID): Option[Seq[(ID, Type)]]
  def dataframe(table: ID): Option[Dataframe]
  def view(table: ID): Option[Operator]
  def iterator(table: ID): Option[ResultIterator]

  // These come for free but may be overridden
  def listTablesQuery: Operator =
    HardTable(
      Seq(ID("TABLE"), TString()),
      listTables.map { Seq(_) }.toSeq
    )
  def listAttributesQuery: Operator = ???
}

For any caller that just wants one particular format, we can provide translation utilities.

I would like to decouple loading data in Mimir from the process of data staging  My proposal is as follows:
First, the LOAD command (and related mechanisms within Mimir) do nothing more than creating links.  You give it the command URI, and the corresponding dataset becomes visible within Mimir (analogous to Spark's load() command.
As I see it, the primary use-cases for staging is Vizier.  For uploaded files, at least, it makes a lot more sense to have Vizier handle staging directly (i.e., files get streamed directly into S3/HDFS rather than going through Mimir).  This makes it possible to avoid the redundant copy as the data goes through the local filesystem.
For URLs and other network resources like google sheets, spark already seems to do some caching internally.  If necessary, we could materialize one of the views used in data loading, transparent to the user.
What do you think @mrb24 ?

As a point of curiosity: With Mimir handling table definitions, is it safe to drop the alias field from Table and related Operator subclasses?

Most of the immutable data access functionality is alive and kicking.  In particular see the commit notes for:

369536f90bfe974c5a3f7b079f38232cb9181bbb
3996a1a227c6b46f2ce5c02dc759d2c43da28ab7
df9c53778b0e839b2de6d8870707e5c7ebdb70b3
937a0b8257a48a121afa25715dc6b384707051f8
0c6448d484a5347ada1cffd5aa720ebd0a4d4ef3

I'm moving on to implementing mutability.  In particular

Where do materialized views live?
Where does bulk model-related metadata live?
(How) Do we allow write access to S3/Hive?
Support for CREATE/INSERT/UPDATE/DELETE

Where do materialized views live?
I see several places where materialized views can live.

Hive (the current behaviour in master).
The local filesystem (preferred behaviour for the command-line).
The metadata backend?

Hive makes sense when Mimir is connected to a remote Spark instance (e.g., when it's used in Vizier), but carries many of the same limitations that this ticket is trying to address when Mimir is run locally.  The metadata backend is a second potential option, but materialized views are bulk data that rather than simple key-value associations.  This will hurt if, for example, we ever want to use GIT or Ground for metadata.  The local filesystem has the advantage that it also makes Mimir's views accessible from outside of Mimir once they're materialized (the same goes for Hive).
In other words, we want the materialized view target to be configurable (either LoadedTables, Hive, or eventually HDFS).  Moreover, it should just be a slightly more powerful form of a schema provider, so I plan to make LoadedTablesSchemaProvider and HiveSchemaProvider instances of the following trait:
trait BulkStorageProvider extends SchemaProvider
{
  def createStoredTableAs(query: Operator, name: ID)
  def dropStoredTable(name: ID)
}

The database (view manager?) would then be configured at start-up to use one of these providers.
Part of the configuration would also involve deciding on a format for these files.  Hive already defaults to parquet, and I think I'm going to do the same with LoaedTables
Where does bulk model-related metadata live?
A related problem arises with some lenses and adaptive schemas. For example the (now defunct) DiscalaAbadi adaptive schema needed to materialize several tables of derived state (the FD graph).  I see the solution as being similar: The database has a pre-configured bulk storage target, and we just use that.
How/Do we allow write-through access to Hive/HDFS/S3?
Since we're no longer staging, a reasonable question is whether we want to allow users to dump data back in to Hive or similar data stores (i.e., SELECT INTO, CREATE TABLE AS, some staged variant of LOAD, or Vizier-specific functionality).  This could, in principle, work exactly as view materialization... the BulkStorageProvider trait defined above would be enough to support it.
Support for Updates
Punting on this for now, but when it gets implemented it'll be along the lines of an Updatable trait, likely implemented along the lines of @lordpretzel and my discussion above.

The final question is how to implement staging: locally caching remote resources.  Specifically, we'd add an option to the LOAD command
LOAD 'url' WITH STAGING

when WITH STAGING is specified, we create a local copy of the specified url and treat that as the primary URL.  We then back-up the original URL somewhere, allowing it to be re-loaded, e.g. by adding a new command
RELOAD table

There seem to be two ways to accomplish this:

Create (in effect) a materialized view over the data frame obtained by loading the original URL

Pro: Easy to implement / natively supported by BulkStorageProvider
Pro: Universally supported: Every format can be used to create a DataFrame
Con: The original data file can't be re-parsed.
Con: DataFrames don't track provenance, so we'd need to implement support infrastructure to keep track of where it came from (e.g., #319 )

Download the specified file and redirect the URL to point locally

Con: Needs additional API support (added to BulkSchemaProvider?)
Con: Hard (parquet) or impossible (google sheets) to implement for all providers.
Con: Getting this to work with Hive/HDFS will be messy
Pro: The original file can be re-parsed if we need to change load options.
Pro: Implementing RELOAD is easy.

I'm honestly not sure how the latter approach would be implemented for SparkSchemaProvider, since Spark kind of assumes that everything is a Dataset or RDD.  We'd have to go around Spark to whatever datastore (Hive, Derby, etc...) it's using on the backend -- and that assumes that the backend even supports raw file access.
It might be more appropriate to define something like:
trait StagingProvider {
  def stage(url: URL, nameHint: Option[String]): URL
  def stage(input: InputStream, fileExtension: String, nameHint: Option[String]): URL
  def drop(local: URL)
}

stage would download a remote URL to a local staging area (e.g., LocalFS or S3) and return a URL to the local copy.  During database initialization, we pick a suitable StagingProvider and rely on that for staging.
I'm a little worried that this overlaps in purpose with BulkStorageProvider.  For example, we could add the following method:
  def stage(input: DataFrame, nameHint: Option[String]): URL

However, I can't quite square the two.  There's a major distinction (and maybe this should be reflected in the names) in that StagingProvider is for raw-file storage (staged files are accessed by URL), while BulkStorageProvider is specifically for storing tables (stored tables are immediately visible within Mimir).
I'm going to rename them:

BulkStorageProvider -> MaterializedTableProvider
StagingProvider -> RawFileProvider

Action items:

[ ] Rename providers
[ ] Implement a LocalFS RawFileProvider (S3 and HDFS can come later)
[ ] Modify the CREATE TABLE AS workflow to use RawFileProvider
[ ] Add WITH STAGING to the LOAD command
[ ] LOAD WITH STAGING called using raw-staging-friendly format ('csv', 'json', 'excel', 'xml', or 'text') should download the specified URL using the Database's preferred RawFileProvider.
[ ] Add and implement RELOAD table

Of course, we can't just use raw staging.  Parquet files don't download easily (multiple files), and some formats (e.g., Google Sheets) just straight up don't download locally.  For these, the only practical approach is to materialize the corresponding dataframe.  Doing this through ViewManager is tempting.  However, ViewManager also tries to materialize supporting metadata (e.g., taint columns) which will not exist here.  Worse, if metadata columns are requested that do not exist in the materialized view, ViewManager defaults back to running the source query.  This is not behavior that we want.  We need a stand-alone thingie for materializing the dataframe.
The natural approach would be to use the database's preferred MaterializedTableProvider.  This has the advantage of being simpler to implement, but creates a behavioral fork based on the table format.  Some staged tables go down one path, while other staged tables go down another.  It also places some loaded tables in the IMPORTED schema, and others in whatever happens to be the currently live MaterializedTableProvider.
A more stable way to implement this would be to use the CREATE TABLE AS workflow instead of the CREATE VIEW workflow.  That is, we instantiate the DataFrame as normal, but dump it out to a raw file the way LoadedTables implements MaterializedTableProvider.  File formats like Parquet can do this losslessly.  Then, we point at the newly materialized Parquet file and set the format to Parquet rather than the original.  We would need a new method on RawFileProvider:
  def stage(input: DataFrame, format: String, nameHint: Option[String]): URL

Action Items:

[ ] Implement DataFrame staging
[ ] LOAD WITH STAGING called using a non-raw-friendly format (everything else) should instantiate a DataFrame using the specified URL, feed it to the RawFileProvider, and continue the workflow as before. Save the original format along with the url in the MimirMetadata.

Modulo making sure test cases pass, this branch seems to be feature complete.

closed with #338 merge


Check if we've used a day pass, before declaring we own the iotm

Issue
If the user uses a day pass, then logs out. On login, mafia will declare that they now own the iotm itself.
Fix
Check if the "today" preference was set, indicating they used a day pass to get access to the iotm.

I'm confused about why this is needed.  @gausie changed a number of preferences so that they were set by whether KoL thought the account owned the cool item, or not.  That is clearly more reliable than trying to guess or just believing what the user claimed.
If a user doesn't have a cool item, uses a day pass, logs out and then KoLmafia claims the user has the cool item it seem there might be a better fix.  My first question is just to ask how you know that "mafia will declare that they now own the iotm itself"?  Mafia should only make such a declaration if KoL tells KoLmafia via an API call that the item is owned.  So the mafia code or script that makes the declaration could be incorrect and a better (IMO) fix would be at that point.  If the code making the declaration is correct then perhaps when the API is called and the cool items updated needs to be moved?  Logically, it seems to me that if the item is owned then the day preference is not relevant.
I note there is a reduction in test coverage.  A failing test that shows what doesn't work without this fix would certainly be helpful.

The problem is that http://<IP_ADDRESS>:60080/api.php?what=status&for=me returns a coolitems array that contains whether or not you have access to the item at the moment, but the Mafia implementation assumes that coolitems showed whether you have the item unlocked permanently.

The problem is that http://<IP_ADDRESS>:60080/api.php?what=status&for=me returns a coolitems array that contains whether or not you have access to the item at the moment, but the Mafia implementation assumes that coolitems showed whether you have the item unlocked permanently.

But #474 says about cool items that

It indicates the fact that a given item has been bound to the account

Before we change the code perhaps we should come to an agreement on what the preferences are supposed to mean?
To me, "bound to the account" is not the same as "accessible".
The behavior as described - unbound, use day pass, logout, bound, does seem buggy but that is because the only things that logically should change the "bound status" are a call to API or doing something that makes the IOTM show up in inventory (and thus KoL will eventually bind it to the account).

Yeah, I think coolitems was supposed to be "bound to the account" but actually is "can access at the moment". So the API is less useful than thought.

Yeah, I think coolitems was supposed to be "bound to the account" but actually is "can access at the moment". So the API is less useful than thought.

So what we have is a fix to KoLmafia to deal with a KoL bug or a lack of communication on KoL's part?
So if cool items means accessible NOW...
I use a day pass.  If that is successful it means the area is available.  Presumably if path or other restrictions applied mafia would not let you use the pass.  I log out.  I log back in.  Something says the area is available.  That is correct since the pass was used.  There is no error mafia to be fixed unless KoLmafia overrides what it read from the API.  If for some reason KoL returned different cool item values then maybe KoL should be petitioned to fix their bug?
So is the proposed fix for something that is not a bug?

I'm in a run which is under Standard restrictions, although it is Grey Goo, not Standard.
Here is an extract from my api.php status:
"pathname":"Grey You",
"coolitems":"airport1,airport2,airport3,airport4,airport5,gingerbreadcity,spacegate,fantasyrealm,piraterealm,neverendingparty,voterregistered,boxingdaycare,hascosmicball"
On the Wrong Side of the Tracks, the Boxing Daycare is not present - but the Neverending Party is.
If I try to enter, "That party is so last year. You don't want to visit on this path."
(The other "old" locations - Detective Precinct, L.O.V., etc. - are not present. Nor are the Voting Booth and LT&T and Horsery on the Right Side of the Track.)
Spacegate and Chateau Montegna are visible in the Mountains. Gingerbread City is not.
Lyle's Monorail can go to FantasyReal and PirateRealm.
I can go to the Welcome Center and get an item - which is too old to use.
I can step into Port Beginning and get the eyepatch - which is too old.
I can go to the airport.
I can go to Barf Mountain
I can enter the Maintenance Tunnels and the Kiosk and the Store
If I try to adventure, "This zone is too old to visit on this path."


Notification Issues

I am getting unreadable characters in the output of second notification , somehow utf-8 isnt decoding the output properly.
Also It will be a lot better, if we can simply send notification like "Device_name Disconnected/Connected" instead of the current notification which is just a information mess.

Fixed. Please update this extension from Ulauncher settings and check now =)

yeah, working now. thanks


Detect variables in iproj.json and .ibmi.json

When the project is opened and the extension is connected, scan the local workspace to find the iproj.json and the many possible .ibmi.json files.
Find all the possible variables used inside of those files. Usually a variable starts with &.
If no .env exists, automatically create a .env with those variables found so the user can provide values. Set the initial value to *CURLIB for each variable

excited about this functionality.

Will be part of the Project Mode extension being developed by IBM.


Plotting

Atom doesn't use all space available for plotting.
image
using Gadfly

I've answered this over at https://github.com/JuliaLang/julia/issues/16367#issuecomment-219272525.


conda-build-all no longer determines correctly whether a destination exists on a channel

The issue is that anaconda.org now returns package names qualified by channel names so the current logic at https://github.com/SciTools/conda-build-all/blob/master/conda_build_all/builder.py#L166 fails.
For example, new list of keys from a call to index is: [u'astropy-ci-extras::obvious-ci-0.6.0-py35_0.tar.bz2', u'astropy-ci-extras::obvious-ci-0.6.0-py27_0.tar.bz2', ...].
I'll get a fix in later today, wanted to record the cause before I forget. It would be nice to get #39 fixed so that integration tests worked again 😄

Ah, it is a change in conda, not anaconda.org. Relevant code is at https://github.com/conda/conda/blame/master/conda/api.py#L33.

Released in conda 4.0.6.

To continue the conversation with myself, the specific line above is incorrect. The relevant line is actually: https://github.com/conda/conda/blame/master/conda/fetch.py#L308 (which was released in 4.0.6).

A temporary solution was added in #43 (the unit test already failed appropriately). Specifically https://github.com/SciTools/conda-build-all/pull/43/files#diff-9421a72ec0f69a877cefa04fb8dda2abR166.
The temporary workaround is consistent with previous behaviour, but does mis-represent the channel priority that is now part of conda since 4.0.6 so I'm still keen to implement the fix correctly.
In the meantime, I'm going to release conda-build-all v0.13.1 and would happily release a v0.14 as soon as we have a more appropriate solution (help definitely sought for that @mwcraig).

It is on my list... Worst case it'll get done at scipy.
On Jun 26, 2016, 3:46 AM -0500, Phil<EMAIL_ADDRESS>wrote:

A temporary solution was added in#43(https://github.com/SciTools/conda-build-all/pull/43)(the unit test already failed appropriately). Specificallyhttps://github.com/SciTools/conda-build-all/pull/43/files#diff-9421a72ec0f69a877cefa04fb8dda2abR166.
The temporary workaround is consistent with previous behaviour, but does mis-represent the channel priority that is now part of conda since 4.0.6 so I'm still keen to implement the fix correctly.
In the meantime, I'm going to release conda-build-all v0.13.1 and would happily release a v0.14 as soon as we have a more appropriate solution (help definitely sought for that@mwcraig(https://github.com/mwcraig)).
—
You are receiving this because you were mentioned.
Reply to this email directly,view it on GitHub(https://github.com/SciTools/conda-build-all/issues/42#issuecomment-228590896), ormute the thread(https://github.com/notifications/unsubscribe/ABGBH7xHrsCNvCUgYuIBW9cyzsPICI0Rks5qPjxqgaJpZM4I9GIt).


query string params

Hi, you have planned in the roadmap when the feature will be available to collect the parameters through the query string?
Thanks!

Same as #56

Close in favor of #56? 😄

Closing as a duplicate of #56.


[Helm Charts] Diff new default values against existing values

What would you like to be added:
A way to diff new default values against the existing values, when upgrading a helm chart.
Why is this needed:
When upgrading a chart, all existing values will be shown (not only user changed values), which in most cases also contain version numbers. Even when setting up a new chart with just changed values, as soon as you upgrade a chart, everything will be user changed values, if you don't sort out the values you don't want to change.
This way, when just "upgrading" the chart, nothing will actually change, the versions will stay the same as pinned and maybe the chart structure will have changed. But I actually want to upgrade the Application Version with the Chart Version.
Another Problem is that my AppVersion shows 1.9 while my actual pinned image is still 1.8, because my values have overwritten the new default values.
Even when just showing the user-changed values in the upgrade window, it would be nice to see a diff for the values I changed.
Environment you are Lens application on:

Kubernetes distribution: K3S
Desktop OS: MacOS

This problem is the reason I never use Lens to upgrade my Helm releases.


Add a cron recipe dashboard page

I think I got timezone stuff sorted finally.  Time issues in programming are a real headache!

Job python3 Selenium Test on 5e7721d : invalidated by @rwcarlsen


First-class combinational components

Calyx supports comb groups (introduced in #635) for encapsulating combinational (sub-cycle) logic, and primitive components (ones implemented externally in RTL) are allowed to be declared as combinational. The standard library relies heavily on these primitives—for example, std_add is a combinational primitive.
However, Calyx-implemented component definitions must take at least one cycle. (As a side note, it would be great to document this fact, even before we change it…) As an aspiration, it would be great to be able to implement combinational components in Calyx so you are not forced to switch to Verilog if you want to build up combinational abstractions. For example, you might like to implement an add3 component that uses two std_adds to add together 3 inputs instead of 2.
Some open questions here include:

Can these components have a non-empty control program? (Probably not, at least in the first version. Our control statements are sequential by nature.)
How do we enforce combinational-ness? (You can only instantiate combinational subcomponents, maybe?) Or do we just assume it?
Are there any complexities in the compilation process that are not present for combinational primitives?

As background, this came up most recently in https://github.com/cucapra/calyx/discussions/1050#discussioncomment-3027264, i.e., in the context of the resource sharing pass (and inferring shareability). We previously implemented combinational groups and primitives in #635. This is also related to the discussions in #788 about how you're allowed to use done, when we sort of figured out that we wanted all components to be sequential in that their done condition is held by a register.

Bumping this as it would significantly simplify code generation for the memory compiler. @matth2k or I might try to tackle this soon.


[SPARK-15994] [MESOS] Allow enabling Mesos fetch cache in coarse executor backend

Mesos 0.23.0 introduces a Fetch Cache feature http://mesos.apache.org/documentation/latest/fetcher/ which allows caching of resources specified in command URIs.
This patch:

Updates the Mesos shaded protobuf dependency to 0.23.0
Allows setting spark.mesos.fetchCache.enable to enable the fetch cache for all specified URIs. (URIs must be specified for the setting to have any affect)
Updates documentation for Mesos configuration with the new setting.

This patch does NOT:

Allow for per-URI caching configuration. The cache setting is global to ALL URIs for the command.

Can one of the admins verify this patch?

ok to test

Test build #60998 has started for PR 13713 at commit 1ebfaf6.

Test build #60998 has finished for PR 13713 at commit 1ebfaf6.

This patch fails build dependency tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60998/
Test FAILed.

Merged build finished. Test FAILed.

@tnachen any comments?

Test build #62931 has started for PR 13713 at commit a9b735c.

Test build #62931 has finished for PR 13713 at commit a9b735c.

This patch fails build dependency tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62931/
Test FAILed.

We also fetch URIs for running drivers in cluster mode (MesosClusterScheduler.scala). I'm thinking we should also allow this configuration to effect that too.

Test build #63032 has started for PR 13713 at commit 974245f.

Test build #63032 has finished for PR 13713 at commit 974245f.

This patch fails build dependency tests.
This patch does not merge cleanly.
This patch adds no public classes.

Build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63032/
Test FAILed.

OoooOOO master updated to 1.0.0
Fixed merge conflicts

@tnachen this test is a little larger than I originally anticipated. Let me see if I can add some unit tests

Test build #63046 has started for PR 13713 at commit bad9d69.

Test build #63046 has finished for PR 13713 at commit bad9d69.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
class SparkConf(loadDefaults: Boolean) extends Cloneable with Logging with Serializable

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63046/
Test PASSed.

@drcrallen Are you still planning to update this? It's quite a useful feature, so hoping this can get in. Also since Fine grain mode is depcreated I don't think we need to update it too.

I can update this today.

Test build #66688 has started for PR 13713 at commit 456d28f.

I'm still adding a basic test for this.

Test build #66688 has finished for PR 13713 at commit 456d28f.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66688/
Test PASSed.

Tested with mvn test -Pmesos -pl mesos -Dtest=MesosCoarseGrainedSchedulerBackendSuite

@tnachen let me know if there's any outstanding issues here.

Test build #66690 has started for PR 13713 at commit 97c0d30.

Other than the 2 comments, the changes LGTM. @mgummelt @srowen

Test build #66690 has finished for PR 13713 at commit 97c0d30.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66690/
Test PASSed.

Merged build finished. Test PASSed.

Test build #66703 has started for PR 13713 at commit 54ae86b.

Test build #66703 has finished for PR 13713 at commit 54ae86b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66703/
Test PASSed.

Merged build finished. Test PASSed.

Test build #66850 has started for PR 13713 at commit cc3901e.

Test build #66852 has started for PR 13713 at commit fc0802c.

Test build #66850 has finished for PR 13713 at commit cc3901e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test build #66852 has finished for PR 13713 at commit fc0802c.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66850/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66852/
Test PASSed.

LGTM @srowen

Test build #66909 has started for PR 13713 at commit 7195c6e.

Test build #66909 has finished for PR 13713 at commit 7195c6e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66909/
Test PASSed.

Merged build finished. Test PASSed.

@mgummelt do you know what still needs to be done to get this in?

We need to get @srowen or one of the other committers to merge it.

Merged to master


change names of sentinel connectivity limits

I confirm that this contribution is made under the terms of the license found in the root directory of this repository's source tree and that I have the authority necessary to make this contribution on behalf of its copyright owner.
@havardpe please review
@kkraune note new names / semantics in progress...

please take another look now


Move links from bids-validator to legacy-validator

Links were broken by repo migration.

Following advice in https://github.com/gaurav-nelson/github-action-markdown-link-check, switched to https://github.com/UmbrellaDocs/action-linkspector.

Not sure that linkspector is actually finding things... I'll move that to a separate PR.


VS2022 always prompts to update package and then throws exception at the end of model training!

System Information (please complete the following information):

Model Builder Version (available in Manage Extensions dialog): <IP_ADDRESS>2701
Visual Studio Version: 17.0.0

WebsiteSentiment-TSS0MD.txt
This happens even when there is no update shown in nuget package manager.
Describe the bug

On which step of the process did you run into an issue:
Clear description of the problem:

To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Add any other context about the problem here.

Thanks for reporting! Per the log file we're trying to update to 1.6.0
Installing nuget package, package ID: Microsoft.ML, package Version: 1.6.0
@khteh What type of project is this (.NET Core, framework...) and what version? Can you check the csproj?

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net6.0</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.ML" Version="1.7.0" />
  </ItemGroup>
  <ItemGroup Label="WebsiteSentiment">
    <None Include="WebsiteSentiment.consumption.cs">
      <DependentUpon>WebsiteSentiment.mbconfig</DependentUpon>
    </None>
    <None Include="WebsiteSentiment.training.cs">
      <DependentUpon>WebsiteSentiment.mbconfig</DependentUpon>
    </None>
    <None Include="WebsiteSentiment.zip">
      <DependentUpon>WebsiteSentiment.mbconfig</DependentUpon>
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
    </None>
  </ItemGroup>
</Project>

Thank you @khteh! This helps a lot.
The code checks to see if the current version and target version match, and if not it tries to upgrade. The problem here is that we're actually trying to downgrade from 1.7.0 to 1.6.0. @LittleLittleCloud Are we good to ignore the nuget package install if we'd actually be going backwards?

@khteh let us know if this error blocks you from model builder but you should be able to just ignore the error window since the downgrade error is a recoverable one.
@beccamc  We can ignore downgrade ml.net if there's no api breaking change for the newer version and the newer ml.net can always load old model. Otherwise it's better to try downgrading. However, I feel like it's always unsafe to try downgrading a package. Maybe a better way is just upgrade model builder which uses latest ml.net
The downgrading should be an uncommon case though since model builder should always stick to the latest ml.net. Since ml.net 1.7.0 is released, we should release a newer model builder asap.

Not a show-stopper but

if there's no api breaking change for the newer version and the newer ml.net can always load old model
Can you verify this and provide a better VS2022 experience with ML.Net?

Can you verify this and provide a better VS2022 experience with ML.Net?

Unfortunately under the current technology, there's no way to verify things that do not happen. even though mlnet will try their best to maintain backward compatibility. However we will push a fix and provides clearer information on such exception to avoid downgrading packages and  reduce confusions.

This is completed and shipped! If you're still seeing a problem please open a new issue.


Cancel commit with  key

Instead of getting out, when I use <Esc> in telescope prompts, it goes to the next question.

Hi @alr86, thanks for opening an issue. I've made the changes in #15 and you should be able to cancel the current action on <Esc> from now on after pulling the latest version.


Uint (s) are equal to int (s).

Expected
False
Actual
true
Steps To Reproduce
Live Bridge
Dot Net Fiddle
/// <summary>
/// Creates a simple class
/// </summary>
public class App
{
    // Use the [Ready] attribute to automatically
    // run this method when the page is ready.
    [Ready]
    public static void Main()
    {
        Console.WriteLine(3.Equals(3u));
    }
}

Related #1301, #1347

Fixed in https://github.com/bridgedotnet/Bridge/issues/2343


[SE-0258] Clarify that property delegates cannot start with a lowercase letter

Alright, since this was always intended to be a baseline for the proposal, I'll merge it in.


Packaging: make mac package require 10.12 or newer

PR Summary

Fixes #4290
Make mac package require 10.12 or newer.
Required that the package is installed to a disk with macOS installed.
The Apple example XML had a background image added as well, so I added a background image:

PR Checklist
Note: Please mark anything not applicable to this PR NA.

[x] PR has a meaningful title

[x] Use the present tense and imperative mood when describing your changes

[x] Summarized changes
[NA] User facing Documentation needed

[NA] Issue filed - Issue link:

[x] Change is not breaking
[NA] Make sure you've added a new test if existing tests do not effectively test the code changed

[NA] Add [feature] if the change is significant or affectes feature tests

@thezim If you are familiar with this area, would you review this.  Look at the issue for the main reason for the change.

@TravisEz13 The package does not contain the launcher when I build your branch. Not sure if this is product of #5625 or I'm missing something in the build. Can you please verify if the launcher is there?

@thezim Thanks for the review.  The issue was caused by the previous PR.  I have updated this PR.  Thanks.


Revert "Add quay.io as second registry"

Reverts weaveworks/kured#439
:sob:

Hello folks!
What was the reason for the revert? I fail to see that in the commit message.

Weaveworks's quay.io account is still not working. Maybe we should move to GHCR? I'll ask again. :-(


Adds C++ guards for dirent.h fixing linker errors when included from cpp file.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.2 out of 15 committers have signed the CLA.:white_check_mark: projectgus:white_check_mark: dobairoland:x: blueMoodBHD:x: jack0c:x: heyinling:x: MaxwellAlan:x: shenjun7:x: igrr:x: Isl2017:x: chhajedji:x: antmak:x: Sam Washburn:x: abarangan:x: Aditi-Lonkar:x: xiewenxiangSam Washburn seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Implement Registration backend service API

As a backend developer
I need to Implement Registration backend service API
So that users can create new accounts on the platform.
Details and Assumptions
* The API should support user registration by collecting necessary details such as username, email and password.
* The API should validate input data for correctness and uniqueness (e.g., unique email).
* The API should securely store user credentials using encryption.
* The API should send a confirmation email to the user upon successfully registration.
* Proper error handling and responses should be implemented.

Acceptance Criteria
gherkin
Given the requirements for the Registration backend service API
When I implement the API
Then the API should accept user details for registration
And the API should validate and store user information securely
And the API should send a confirmation email to the user
And the API should provide proper error handling and responses

I'm working on it.


Clean up factory girl association build / create chains

See #1030 spec/factories/classifications.rb
I guess this is really an argument where we should use factory_girl specifically mocking and stubbing vs persisting data to the DB. However if/when we do persist data we need to ensure that only the required factory associations we need for the test are built and we don't fire off costly persistence operations.

Good idea but I think this needs to be boyscouted rather than something we'll do as a concrete issue.


Public exports of generated models can be improved

Description
When generating a model, the created file (e.g., src/models/persons.rs) is missing public reexports for ActiveModel and Model. Only Entity is publicly exported.
The default src/models/users.rs file, however, reexports everything through pub use super::_entities::users::{self, ActiveModel, Entity, Model};. This allows convenient (and clean) usage of these types in tests (see import section of tests/models/users.rs.
Tests for newly generated models (e.g., tests/models/persons.rs) need to use the full path to access the corresponding ActiveModel.
I do not consider this a bug - as the user can easily work around it - but unifying this would streamline the developer experience.
To Reproduce

Generate a new SaaS project
cargo loco generate scaffold persons uuid:uuid first_name:string! last_name:string! --api
Import section of tests/models/persons.rs and src/controllers/persons.rs is not ideal.

Expected Behavior
src/models/persons.rs should look like this:
pub use super::_entities::persons::{ActiveModel, Entity as Persons, Model};
use sea_orm::entity::prelude::*;

impl ActiveModelBehavior for ActiveModel {
    // extend activemodel below (keep comment for generators)
}

tests/models/persons then can use a clean and simple import like use foobar::models::persons::{self, Persons};
Environment:

loco-cli 0.12.0
rustc 1.82.0 (f6e511eec 2024-10-15) with target stable-x86_64-unknown-linux-gnu (default)

Additional Context
None

implemented

great thanks!


NaNs appear in ORES output

For an example that returns NaNs, see https://ores.wmflabs.org/scores/enwiki/wp10 which currently contains
[...]
    "roc": {
      "B": {
        "auc": 0.153
      },
      "C": {
        "auc": NaN
      },
      "FA": {
        "auc": NaN
      },
      "GA": {
        "auc": NaN
      },
      "Start": {
        "auc": NaN
      },
      "Stub": {
        "auc": NaN
      }
    },
[...]

This is not up to code with the JSON spec.  See https://tools.ietf.org/html/rfc7159
It happens with python's json.dumps() method.  See https://docs.python.org/3/library/json.html#json.dump  Specifically, there's an allow_nan argument that is set to True by default.

Addressed upstream.

Should we close this?


Total quota of personal space is set to 0

Describe the bug
A clear and concise description of what the bug is.
Steps to reproduce
Steps to reproduce the behavior:

As user Moss send curl request to list the drives

curl -u moss:vista -k -X GET https://host.docker.internal:9200/graph/v1.0/me/drives -v | jq

Expected behavior
The values of the total quota should be equal to the remaining quota as nothing is used
Actual behavior
Total quota is displayed as 0
{
  "value": [
    {
      "driveAlias": "personal/moss",
      "driveType": "personal",
      "id": "1284d238-aa92-42ce-bdc4-0b0000009157$058bff95-6708-4fe5-91e4-9ea3d377588b",
      "lastModifiedDateTime": "2022-08-22T06:28:41.915990441Z",
      "name": "Maurice Moss",
      "owner": {
        "user": {
          "id": "058bff95-6708-4fe5-91e4-9ea3d377588b"
        }
      },
      "quota": {
        "remaining":<PHONE_NUMBER>8,
        "state": "normal",
        "total": 0,
        "used": 0
      },
      "root": {
        "eTag": "\"a5efa2445179bbc3ebba96ede5096d21\"",
        "id": "1284d238-aa92-42ce-bdc4-0b0000009157$058bff95-6708-4fe5-91e4-9ea3d377588b",
        "webDavUrl": "https://host.docker.internal:9200/dav/spaces/1284d238-aa92-42ce-bdc4-0b0000009157$058bff95-6708-4fe5-91e4-9ea3d377588b"
      }
    }
  ]
}

Even if I upload certain content the total quota is still 0
{
  "value": [
    {
      "driveAlias": "personal/moss",
      "driveType": "personal",
      "id": "1284d238-aa92-42ce-bdc4-0b0000009157$058bff95-6708-4fe5-91e4-9ea3d377588b",
      "lastModifiedDateTime": "2022-08-22T06:38:56.419772283Z",
      "name": "Maurice Moss",
      "owner": {
        "user": {
          "id": "058bff95-6708-4fe5-91e4-9ea3d377588b"
        }
      },
      "quota": {
        "remaining":<PHONE_NUMBER>8,
        "state": "normal",
        "total": 0,
        "used": 174
      },
      "root": {
        "eTag": "\"c8238b82fcb0c596b9060e90a6380440\"",
        "id": "1284d238-aa92-42ce-bdc4-0b0000009157$058bff95-6708-4fe5-91e4-9ea3d377588b",
        "webDavUrl": "https://host.docker.internal:9200/dav/spaces/1284d238-aa92-42ce-bdc4-0b0000009157$058bff95-6708-4fe5-91e4-9ea3d377588b"
      }
    }
  ]
}

If I update the quota of the user
curl -X PATCH 'https://localhost:9200/graph/v1.0/drives/1284d238-aa92-42ce-bdc4-0b0000009157$058bff95-6708-4fe5-91e4-9ea3d377588b' -H "Content-Type: application/json" --data '{"quota":{"total":1000000000}}' -uadmin:admin -vk

Then it gets updated
{
  "value": [
    {
      "driveAlias": "personal/moss",
      "driveType": "personal",
      "id": "1284d238-aa92-42ce-bdc4-0b0000009157$058bff95-6708-4fe5-91e4-9ea3d377588b",
      "lastModifiedDateTime": "2022-08-22T06:38:56.419772283Z",
      "name": "Maurice Moss",
      "owner": {
        "user": {
          "id": "058bff95-6708-4fe5-91e4-9ea3d377588b"
        }
      },
      "quota": {
        "remaining": 999999826,
        "state": "normal",
        "total":<PHONE_NUMBER>,
        "used": 174
      },
      "root": {
        "eTag": "\"c8238b82fcb0c596b9060e90a6380440\"",
        "id": "1284d238-aa92-42ce-bdc4-0b0000009157$058bff95-6708-4fe5-91e4-9ea3d377588b",
        "webDavUrl": "https://host.docker.internal:9200/dav/spaces/1284d238-aa92-42ce-bdc4-0b0000009157$058bff95-6708-4fe5-91e4-9ea3d377588b"
      }
    }
  ]
}

Setup
Please describe how you started the server and provide a list of relevant environment variables or configuration files.

OCIS_XXX=somevalue
OCIS_YYY=somevalue
PROXY_XXX=somevalue

Additional context
Add any other context about the problem here.

0 represents “unlimited”

0 represents “unlimited”

Sorry, that wasn't intuitive to me. I was adding some tests where the quota of a user is set to 0 and then the user tries to upload a resource of a certain size. And I couldn't do it or rather it didn't work as expected. So I thought this must be a bug. I'll be closing it if it's not a bug.


Feat: Remove selection of account type on login page

Describe the feature
We can remove the selection of account type from the login page. Instead, users will now only be prompted to enter their credentials for authentication.
Changes Made:

Removed login  as election UI from the login page.

Reasoning:
By removing the account type selection from the login page, we streamline the login process and reduce complexity for users. They can now focus solely on logging in with their credentials without unnecessary choices.
Add ScreenShots
Just remove the login as select element from the login page
Record

[X] I agree to follow this project's Code of Conduct
[X] I'm a GSSOC'24 contributor
[X] I want to work on this issue

Assigned you this task !! Raise the PR when done

i want to work with you ,please assign me ,i make a lots of real time website ,so i expert in that

#93

#93
I created a pull request for this

I have checked it !! Will merge it soon


[feature request] Use /tmp and /run instead of /opt/piavpn-manual

/opt should be considered read-only by the scripts.
Temporary files should be stored in /tmp or better a directory in /tmp created with mktemp. For example
tmpdir=`mktemp -d -p /tmp -t piavpn-manual-XXXXX.d`
echo $tmpdir
/tmp/piavpn-manual-k5cVRf.d

The random portion of the directory name is crucial for the security and avoid race conditions.
It is not yet in the FHS, but it is already largely  used that /run is used for transient like pid_file of process or services.
For example in /run/piavpn-manual could store resolv_conf_backup, pia_pid, route_info, etc ... if some are not stored in /tmp
This is linux-centered and I don't know of BSDs ...
And the port used for port forwarding needs to be exported somewhere (in run/piavpn-manual ?) to be available to other process that needs that port.
I had to add line to port_forwarding.sh to get that done but that is expected I guess from the README.md

Temporary files should be stored in /tmp or better a directory in /tmp created with mktemp
It is not yet in the FHS, but it is already largely used that /run is used for transient like pid_file of process or services.
For example in /run/piavpn-manual could store resolv_conf_backup, pia_pid, route_info, etc ... if some are not stored in /tmp

The PIA Desktop app currently uses /opt/piavpn/ so we are currenly using /opt/piavpn-manual/ for consistency, however I do agree that /tmp/ would be better fitted for keeping the temporary configs.
If you have time to open a PR, please also test the change extensively (preferably on multiple distributions) to be sure it does not break current functionality (the scripts were designed to disconnect the previous connection before creating a new one) and doesn't have any other random effects.

The random portion of the directory name is crucial for the security and avoid race conditions.

Regarding security: The VPN technology will create an encrypted tunnel between your machine and the VPN server. This means the internet will prevent you against at attackers that have enough access on your network to be able to see your traffic. If you machine is already compromised (somebody else has access on your Laptop and is able to read all files on the system), you will first have to deal with the malicious software on your system. Using a random temporary folder might help, however if you have concerns regarding other software on your system, please try to isolate that specific software by using: QEMU, SELinux, Docker etc..
Regarding race conditions: The scripts were not designed to run in parallel. If you need to multihop, please create a fork and modify the scripts to fit your requirement.

And the port used for port forwarding needs to be exported somewhere (in run/piavpn-manual ?) to be available to other process that needs that port.

Agreed, though this a different topic/ticket. A PR for this change would be a lot easier to approve, as it can't really break anything.
If you have time to open a PR, please use different file names for OpenVPN and WireGuard. The name of the file can not be random, as it would defeat the purpose.

I had to add line to port_forwarding.sh to get that done but that is expected I guess from the README.md

Yes. The idea behind the repo is to enable you to configure your system the way you like it. This is also the reason why we can't go to deep into technology on this repo, as it has to be easy to read and to understand.

Maybe we must keep the default folder as /opt/piavpn-manual
But with a simple env var, it could be overriden ?
And at the same time remove some test as root user just for have rights to create /opt/folder. As this folder could be that one or another. My user is not root and have permissions to create in /opt/piavpn-manual (or other folders)
Without have to fork and edit your script ! As it is hard to follow evolution in time on a fork

Maybe we must keep the default folder as /opt/piavpn-manual
But with a simple env var, it could be overriden ?
And at the same time remove some test as root user just for have rights to create /opt/folder. As this folder could be that one or another. My user is not root and have permissions to create in /opt/piavpn-manual (or other folders)
Without have to fork and edit your script ! As it is hard to follow evolution in time on a fork