Recursive ownership change?

GAM has been super useful for me as I try to manage our Drive instance when many of our users don't follow appropriate protocols. Is there any way that GAM can recursively change owners on all child folders? It's quite easy to change ownership for one folder, but I can't find any GAM function that list all child elements in a folder with their owners (which I could then plug back into GAM via bulk operation). I've tried looking through the code but I don't have much python or Google Apps/Drive API experience and haven't been able to figure out how to call the appropriate Google Drive API functions to extend GAM to support such a function.
Any advice would be much appreciated!

I tried something like this a few months ago to no avail. Ended up writing a C++ program instead to generate the gam commands run them and process the output.
basically created a list of folders to process and added to that list when processing a folder to do a isparent query.
went something along the lines of
command = m_GamLocation + "gam all users show filelist query "('" + m_FolderList[count] + "' in parents";
I would love to see other approaches, as I'm sure many are much, much better than mine.

Thanks for the suggestion - how did you get the children of the folder? I'm assuming you then had to handle the auth and everything yourself in C++? That's what I was trying to avoid.

I used the gam command above to get a list of all files and folders that had the parent of the folder in question. Basically it comes down to gam all users show filelist query " folderID " in parents
Take the id's from this list, add the folders to your processing list and go again. All the C++ script handles is generating the Gam command, having gam run it, and processing gam's output while keeping a list of folders that still need to be processed.

came across this thread and have a need to alter the owner and permissions of subfolders and files recursively. is this approach discussed here still the best approach. Google permissions, owners of subfolders and files being mixed between several people is confusing for me and our users. ie when something is deleted or comes up missing especially, so I'm hoping to broadly change permissions to prevent this from happening.

I'd also like to find a solution, even if a script, that can query for the latest additions and apply appropriate permissions for example...
One thing that ticks me is that despite preserving sharing options most of the time on sub-items (folders/files), GDrive wont preserve the "writerscantshare" attribute...
If anyone has a shell or python or perl or whatever script they are willing to share, or at least something similar I can adapt I'd love to see it...  Unfortunately I am not experienced at all with c++ so the time it would take me to just get into it isnt too worth it right now... :)
If anyone has something similar though, I'd like to see it/improve on it...  The alternative would be Powertools for drive, but its so buggy and their support is so sluggish I really cant see myself being happy with it.... (I like the functionality, but its just laden with bugs...)

I'd post my old code, but I don't have access to it anymore. I'd be interested is seeing others attempts on this, but right now the best approach I have is creating 2 lists, adding the root folder to the first list, and running a search on all files querying isParent on the files for members of the 1st list that have yet to be checked, adding new folders to the 1st list, and files to the second. Once no unchecked folders exist in the first list, make changes to all entries in both lists as needed. As I'm sure you can imagine, this is a very time consuming process.

It could be sped up by taking the ID's of isParent of all files in Drive in the first pass, then processing that data to make a folder tree to get your relevant IDs to process. Again though, a timely process, but not likely to find one faster.

One option I thought of is using (not sure if its easy or available - I dont see it in the GAM documentation), a vault search / drive audit to find out changed/added items?
An initial pass through everything would be required/advisable though... And another option in addition to any strategy is to read the current "state" and not re-issue changes on files/folders that already have the required perms...

Not sure Vault is accessible via API's for anything other then licencing.
Current permissions is expensive on time, If time isn't a factor and you want to modify files as little as possible, do it. If time is a factor and modifying files each run doesn't matter, then it takes about as much time to change the permissions on every file as it does to get those permissions in the first place. From there, Time can be saved by keeping a record of the last time you made changes to the file, and if no changes need to be made, just checking the last time it was modified. If it wasn't modified since the last time the program worked with it, then leave it alone.

Gotcha... thanks for that... if anyone knows any script which does something similar I'd like to be pointed in the direction so I can modify it for this case... ;)

This GAM fork might cover your needs.

I understand from https://github.com/jay0lee/GAM/pull/217 that this will probably be merged in?
I wonder if it would be possible to add a permission change to claim... or at least a writerscantshare option... Claim isnt all that useful from a permission standpoint if you cant limit sharing imo...
But then again, its just me talking while I should be coding :p LOL
PS: I did some tests trying to list files with the filelist GAM command, and then using that CSV to issue a writerscantshare, problem is it doesnt allow me to search within a specific folder only... Still looking for a way to get that without haing to do too much recursion... I was hoping for a drive search, or a field with the entire path, which seems to not exist :(

If someone else is also interested in this, one thing that I am putting together is a command that lists all files modified after a certain date, and applies the writerscantshare attribute...
Its really quite simple as I'm just listing all files that belong to a certain user, and has a modified date > a certain time, and its running the gam command to update the permissions...  One thing that I'm looking for is a way of filtering the filelist method...  Drive API doesnt recognize a search parameters for writerscantshare (or the equivalent field name), so I cant use that in the query... But the CSV does output the column, so I'm thinking maybe there is an CLI csv filterer that would filter based on a column value.... that way I can constantly list the files and only run the GAM update to the ones that have the incorrect flag.
If anyone has come across such a thing i'd like to have a pointer... all I found were libs for other lanugages, etc... nothing CLI...
For now I think I'll use a combination of claim with this in order to claim all files/subfolders every X hours, and then apply the writerscantshare update on all files that are modified within the last X hours... Not fool proof, but it will work for now...
Another thing which could be useful is a way of getting/choosing the drive file name when exporting a CSV from GAM with the "todrive" option... That way I can use something to get that file, filter it, and then send that file as a CSV for GAM to use as input...
Currently its looking something like this:
./gam.py all users show filelist query "modifiedDate > 'date -v -1d '+%m-%d-%yT%H:%M:00'' and trashed = false and<EMAIL_ADDRESS>in owners" originalfilename id | ./gam.py csv - gam user<EMAIL_ADDRESS>update drivefile id ~id writerscantshare
This will apply a writerscantshare on all files owned by MAINDADMIN on all drives (maybe this isnt necessary? maybe I can only scan mainadmin as a user instead of all...will try it out) and applies only to files modified in the last 24hs... (you could add -xx:xx to get to the correct timezone but if its running once a day it should be ok)

PS: just realized modifiedDate is only working with <... > or >= give a 400 invalid query....
more debugging to do :/

Great CLI tool called csvkit you might see useful.https://github.com/wireservice/csvkit
Sent from Outlook Mobile
On Fri, Apr 29, 2016 at 3:59 PM -0700, "Roy"<EMAIL_ADDRESS>wrote:
PS: just realized modifiedDate is only working with <... > or >= give a 400 invalid query....
more debugging to do :/
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub

I could absolutely add functionality to the "claim" command. Just so that I'm clear on what you are asking for: you would like to be able to limit sharing of "claimed" files: "Prevent editors from changing access and adding new people"? This must be set per file (not possible on "folder level" as I understand). As of now the previous "owner" simply becomes "editor".
Just to point out - I used the "recursive" functions found in the "show filetree" command. I'm also thinking of expanding the functionality of this command to be able to output a csv of a certain folder (based on the folderID) including subfolders, plus a somewhat clearer output (regarding file/folder structure). Ross Scrogs already has made some improvements to the command in his "future" versions of GAM (using more API calls, which I would like to avoid...).
I don't know if my commands will be merged - they might need some refining before that is a possibility. But at least I made a "pull request", then I will have to see what they think of it. Our organisation is already using the functionality to transfer files on a daily basis from certain shared folders.
/Mattias

Thanks for the pointer @brianyoungblood
And thanks for the input @lerklompen ... Yes, my "idea" would be that claim has an option "writerscantshare" just like the GAM update method has...  Yes, basically it will set "Prevent editors from changing access and adding new people", making it so that only the owner/admin can share files.  The permissions initially inherit from the father, as per my tests, however this particular setting doesnt inherit... Doesnt make much sense IMO but google drive "sharing" and "corporate permissions/ACL" still butt-heads very often... so we need to always look at workarounds :(

Now added a switch "writerscantshare" to the claim command. You can try out the source code (if you are on linux/mac, or if you prepped windows to run python commands).
https://github.com/lerklompen/GAM/tree/branch-3.63
Note that this new switch requires an extra API call per file! So it will "cost" more plus take longer time.
I will draft a new release including new windows binaries soon!
/Mattias

Thia appear to be working quite well... However is it ecevuting a claim + writerscantshare on every file or only on changed/added files?  Maybe it would be possible to use modified date/create date to prevent havign to do too many calls?  I'm currently at about 500,000 files... :/

Apparently it lists all files, but only reissues permissions for the required ones...
I'm gong to explore better a combination of claim with specific permissions (recursively if possible)...  If anyone has some script already doing this it would be nice... Otherwise I'll try and whip together and post when done.
@lerklompen using your latest GAM release without issues! 👍

Hello All!
Any news about this "claim folder" functionality?
Looks like i have "GAM 4.28.01 - https://github.com/taers232c/GAMADV-X"
And when i try "claim folder" i get this:

gam.py user <claiming.user> claim folder 0B1yXh5N...
Command: gam.py user <claiming.user> claim >>>folder<<< 0B1yXh5...
ERROR: Invalid argument: Expected
Help: Documentation is at https://github.com/jay0lee/GAM/wiki
Do i have the wrong version of GAM?

This feature is not in Jay's version, it is in mine.
First, get the current version: https://github.com/taers232c/GAMADV-X/releases/tag/v4.39.20
The syntax is: gam  claim ownership  [skipids ] [skipusers ] [subdomains ] [includetrashed] [writerscantshare] [preview] [filepath] [todrive []]
Since you have a folder ID which is the default  , say: gam.py user <claiming.user> claim ownership 0B1yXh5...
Correspond with me directly on this,
Ross
<EMAIL_ADDRESS>On Jan 16, 2017, at 7:48 AM, pirx242<EMAIL_ADDRESS>wrote:

Hello All!
Any news about this "claim folder" functionality?
Looks like i have "GAM 4.28.01 - https://github.com/taers232c/GAMADV-X"
And when i try "claim folder" i get this:
gam.py user <claiming.user> claim folder 0B1yXh5N...
Command: gam.py user <claiming.user> claim >>>folder<<< 0B1yXh5...
ERROR: Invalid argument: Expected
Help: Documentation is at https://github.com/jay0lee/GAM/wiki
Do i have the wrong version of GAM?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.


OpenSpending: Frankfurt Education story

@demenech Here's the transformed dataset: https://gist.githubusercontent.com/mpolidori/59ba986bc82379191fe48faa446f5110/raw/d3863c5c958d99db43207b13abf8896d88b2e879/frankfurt-sums.csv
It sums the amount for each produktbereich_label per time_year. This is the original: https://www.openspending.org/@os-data/city-of-frankfurt-expenditure


Query parameters are missing.

[{'code': 25, 'message': 'Query parameters are missing.'}] how to fixxing that?

Yup. I had the same error. Have you been able to fix that?


pile

(See the guidelines for contributing, linked above)
steps
problem
expectation
notes

sr for the noise wrong window


When installing on small VPS, (RAM) disk runs full. Solve by supporting disko and /mnt?

See title, this would be an ideal use case (for me pretty much the most likely one, since it is likely to be more often repeated than for own hardware).

Ah thanks, might have been an misunderstanding about functionality, I wanted to use it to do a "one-click-provisioning" on a vps from a finished nixosConfigurations.myvps, but I was able to do that flawlessly with nixos-anywhere, including auto-partitioning with disko.
So tthe main use case of nixinate is to basically "include" the functionality I do now with
NIX_SSHOPTS=-t nixos-rebuild switch --target-host myfqdn --use-remote-sudo --flake .#myvps1

in the flake in order to make remote updates/reconfigs more ergonomical?

@ppenguin that's pretty much it for now yeah, and you could do provisioning in the new nixinate in idempotent pre-deploy scripts, if you wanted. I'm working hard on it, but only have so much time.


Tharmi123 patch 1

Purpose
Examples
Checklist

[ ] Linked to an issue
[ ] Updated the changelog
[ ] Added tests

Codecov Report

Merging #7 (33e29b3) into master (06e7813) will decrease coverage by 0.63%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master       #7      +/-   ##
============================================
- Coverage     80.79%   80.16%   -0.64%
  Complexity       67       67
============================================
  Files           318      335      +17
  Lines         16510    16994     +484
  Branches       3900     3783     -117
============================================
+ Hits          13340    13624     +284
- Misses         2462     2641     +179
- Partials        708      729      +21

Impacted Files
Coverage Δ

ballerina/auth_client_basic_auth_handler.bal
100.00% <ø> (ø)

...allerina/auth_client_bearer_token_auth_handler.bal
91.66% <ø> (ø)

ballerina/auth_client_oauth2_handler.bal
86.95% <ø> (ø)

...erina/auth_client_self_signed_jwt_auth_handler.bal
100.00% <ø> (ø)

ballerina/auth_desugar.bal
96.59% <ø> (ø)

...th_listener_file_user_store_basic_auth_handler.bal
95.83% <ø> (ø)

ballerina/auth_listener_jwt_auth_handler.bal
96.00% <ø> (ø)

...th_listener_ldap_user_store_basic_auth_handler.bal
95.65% <ø> (ø)

ballerina/auth_listener_oauth2_handler.bal
87.87% <ø> (ø)

ballerina/auth_types.bal
100.00% <ø> (ø)

... and 649 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 18f5df2...33e29b3. Read the comment docs.


form parameters are not added to the request

Looks like form data parameters aren't added to the API client request.

Note: Currently, form data parameters aren’t upgraded at all, I’ve tested it in #3219 and I’ve used https://petstore.swagger.io/v2/swagger.json but set the form data parameters to required: true.

Required form data parameters

Nothing appears in the API client

Would love a resolution of this issue :)


Tensor size mismatch when running CAD2 > Task2 baseline enhancement [BUG]

Describe the bug
After downloading and processing the data for the CAD2 > Task2 challenge, I attempted to run enhance.py to explore the baseline enhancement system. This resulted in tensor size mismatch errors on the load_separation_model() step (line 279). The parameters defined in the ConvTasNetStereo class do not match the pre-trained model checkpoint that is being loaded. I addressed the problem by changing three parameters in the ConvTasNetStereo definition in ConvTasNet/local/tasnet.py. I changed the following:

audio_channels=2 - This fixes the size mismatches in the encoder and decoder models.
X=10 and C=2 - These fix the size mismatch in the separator model. I am not sure if both changes are needed. I changed both because I noticed they did not match the configuration in ConvTasNet/local/conf.yml.

To Reproduce

Download the CAD2 > Task2 data.
In config.yaml, set the zenodo_download_path and root path.
Process the data by running process_dataset/process_zenodo_download.py.
Run enhance.py with default parameters.
See error.

Expected behavior
Based on the README in the baseline folder, I expect the baseline enhancement system to run with the default parameters and generate the enhanced .flac files.
Error Messages
File "/home/austin/clarity/recipes/cad2/task2/baseline/enhance.py", line 204, in load_separation_model
models[instrument] = ConvTasNetStereo.from_pretrained(
File "/home/austin/clarity/recipes/cad2/task2/baseline/enhance.py", line 279, in enhance
separation_models = load_separation_model(
File "/home/austin/clarity/recipes/cad2/task2/baseline/enhance.py", line 407, in
enhance()
RuntimeError: Error(s) in loading state_dict for ConvTasNetStereo:
size mismatch for encoder.conv1d_U.weight: copying a param with shape torch.Size([256, 2, 20]) from checkpoint, the shape in current model is torch.Size([256, 1, 20]).
size mismatch for separator.network.3.weight: copying a param with shape torch.Size([512, 256, 1]) from checkpoint, the shape in current model is torch.Size([1024, 256, 1]).
size mismatch for decoder.basis_signals.weight: copying a param with shape torch.Size([40, 256]) from checkpoint, the shape in current model is torch.Size([20, 256]).
Environment
Please include the following...
[ ] OS: Ubuntu 22.04.2 LTS (GNU/Linux <IP_ADDRESS>-microsoft-standard-WSL2 x86_64)
[ ] Python version: 3.8.19
[ ] clarity version: v0.6.0
[ ] Installed package versions:

Hi @awagenknecht ,
Thank you for submitting this issue.
I can run the baseline without getting this error.
Can you please tell us if you are still experiencing this problem?
What causality are you loading?
Thank you

Hi @groadabike, thanks for looking into it. I'm new here, so I apologize if I'm missing something obvious.
I still get the error when running the baseline as provided. I am loading noncausal models.
I can avoid the error if I change the C and audio_channels parameters in the __init__ method of the ConvTasNetStereo class definition in the recipes/cad2/task2/ConvTasNet/local/tasnet.py file, but the baseline seems like it's set up so that I shouldn't have to touch this file.
The parameters in the ConvTasNetStereo class definition do not match the config being downloaded from HuggingFace.

Hi @awagenknecht,
I am not sure why you are getting that error. As I understand, when you use the HuggingFace's from_pretrained method, the model is automatically initialised using the params saved in Huggingface. This is particulary useful if you have a model with different sizes, e.g., Whisper.
If you see the Whisper documentation in Huggingface https://huggingface.co/docs/transformers/en/model_doc/whisper, you can load the openai/whisper-tiny.en version or any of the other versions with different sizes https://huggingface.co/openai
Can you confirm that your load_separation_model in recipes/cad2/task2/baseline/enhanced.py looks like:
def load_separation_model(
    causality: str, device: torch.device, force_redownload: bool = True
) -> dict[str, ConvTasNetStereo]:
    """
    Load the separation model.
    Args:
        causality (str): Causality of the model (causal or noncausal).
        device (torch.device): Device to load the model.
        force_redownload (bool): Whether to force redownload the model.

    Returns:
        model: Separation model.
    """
    models = {}
    causal = {"causal": "Causal", "noncausal": "NonCausal"}

    for instrument in [
        "Bassoon",
        "Cello",
        "Clarinet",
        "Flute",
        "Oboe",
        "Sax",
        "Viola",
        "Violin",
    ]:
        logger.info(
            "Loading model "
            f"cadenzachallenge/ConvTasNet_{instrument}_{causal[causality]}"
        )
        models[instrument] = ConvTasNetStereo.from_pretrained(
            f"cadenzachallenge/ConvTasNet_{instrument}_{causal[causality]}",
            force_download=force_redownload,
        ).to(device)
    return models

Now, looking more in details the error messag, it doesn't look as it should:

File "/home/austin/clarity/recipes/cad2/task2/baseline/enhance.py", line 204, in enhance
separation_models = load_separation_model()

Tha call is in line 279 and it should include 3 arguments
separation_models = load_separation_model(
        config.separator.causality, device, config.separator.force_redownload
    )

The next error line says:
File "/home/austin/clarity/recipes/cad2/task2/baseline/enhance.py", line 204, in load_separation_model
models[instrument] = ConvTasNetStereo.from_pretrained()

The line number is correct, 204.
However, it also should include arguments
models[instrument] = ConvTasNetStereo.from_pretrained(
            f"cadenzachallenge/ConvTasNet_{instrument}_{causal[causality]}",
            force_download=force_redownload,
        ).to(device)

Does your baseline code include these arguments?

Yes, the load_separation_model function is identical. The causality, device, and force_redownload arguments are being passed as expected to load_separation_model, and the model string and force_download are also being passed to ConvTasNetStereo.from_pretrained.
You're right, there are some mistakes in the error message. I was using ChatGPT to help debug and must have gotten mixed up which version of the error I was copying. facepalm Sorry for the confusion. I'll edit the original post with the correct error message.
The main content of the error is the same, though. It looks like all the arguments are being passed. I can see that the config.json is being downloaded from HuggingFace and cached with the correct parameters, but for some reason they are not being applied in the model initialization. This makes me think the issue is somewhere in the HuggingFace package on my end. So I don't think it's a Clarity bug anymore, and this can be closed if no one else is getting this error.

HI @awagenknecht ,
Thank you for the clarification.
It would be very helpful to find the source of the error.
Can you please confirm that you are created a clean environment for the challenge?
Also, can you share what huggingface-hub version do you have? I am using 0.23.0

Updating huggingface-hub has fixed the problem. I previously had 0.21.4 and updated to 0.24.5.
One of these might be the relevant fix:

https://github.com/huggingface/huggingface_hub/pull/2274
https://github.com/huggingface/huggingface_hub/pull/2337

I started from a clean conda environment, but I believe I may have overlooked a permissions issue when installing clarity that caused it to use packages already installed in my base environment.
That said, does clarity have a huggingface-hub version as a requirement? I just tried again in a clean environment, ensuring I fixed the permissions issue, and I needed to install huggingface-hub separately in order to run the task2 enhance.py.
Thanks so much for helping track this down!

Hi @awagenknecht ,
Thank you for let us know.
Glad to know that the issue was just a huggingface-hub version.
Regarding the requirement, thank you for letting us know. We have a extra requirements.txt in task one but not in task2.


solve a ssl judgement bug

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


[Question]: Matching state value based on another entity

What happened?

This card contains a thermostat entity; I have another entity, input_boolean.heating_season and I would like to hide the thermostat if heating_season is off.
Card Version
1.2.7
HomeAssistant Version
current
Relevant log output
No response

Fixed by https://github.com/LesTR/homeassistant-minimalistic-area-card/pull/42


⚠ Samsung has degraded performance

In 6114798, Samsung (https://samsung.com) experienced degraded performance:

HTTP code: 200
Response time: 2890 ms

Resolved: Samsung performance has improved in 78ef09c.


fix: prettier format

What kind of change does this PR introduce?

Did you add tests for your changes?
If relevant, did you update the documentation?
Summary

Does this PR introduce a breaking change?

Other information
format code to pass checks

@meddulla  sorry, forgot to run prettier format.


Unable to build ghc-7.10.1 using ghc-7.4.2-binary

On my Nixos laptop (32bit intel) I am unable build ghc-7.10.1. It fails already when trying to start the binary ghc-7.4.2 it seems.
The source is from the nixos-unstable channel as of today.
plc-laptop% nix-env -iA nixos.pkgs.haskellPackages.ghc
replacing old ‘ghc-7.8.3-wrapper’
installing ‘ghc-7.10.1’
these derivations will be built:
  /nix/store/viqf7nwqshzackfslj2im9xg7af4s1nm-ghc-7.4.2-binary.drv
  /nix/store/5fa0cic2l33ldz5sg7yp20bgf3hrimn2-ghc-7.8.4.drv
  /nix/store/bgkkz1h6s1pc63ggf6xw605x252zi244-ghc-7.10.1.drv
building path(s) ‘/nix/store/ygp6s44nxv4z7am2g8g52iwqkbgxj04h-ghc-7.4.2-binary’
unpacking sources
unpacking source archive /nix/store/4zj30pw1rzvqshnrcsl9dw1ahdxhlz0z-ghc-7.4.2-    i386-unknown-linux.tar.bz2
source root is ghc-7.4.2
not an ELF executable
cannot find section .interp
missing ELF header
cannot find section .interp
cannot find section .interp
(... repeated some 20~ times, snipped away here for clarity)
cannot find section .interp
cannot find section .interp
not an ELF executable
not an ELF executable
patching sources
configuring
checking for path to top of build tree... /tmp/nix-build-ghc-7.4.2-binary.drv-0/ghc-7.4.2
ghc/stage2/build/tmp/ghc-stage2: error while loading shared libraries: libncurses.so.5: cannot     open shared object file: No such file or directory
Can't work out build platform
builder for ‘/nix/store/viqf7nwqshzackfslj2im9xg7af4s1nm-ghc-7.4.2-binary.drv’ failed with exit    code 1
cannot build derivation ‘/nix/store/5fa0cic2l33ldz5sg7yp20bgf3hrimn2-ghc-7.8.4.drv’: 1    dependencies couldn't be built
cannot build derivation ‘/nix/store/bgkkz1h6s1pc63ggf6xw605x252zi244-ghc-7.10.1.drv’: 1 dependencies couldn't be built
error: build of ‘/nix/store/bgkkz1h6s1pc63ggf6xw605x252zi244-ghc-7.10.1.drv’ failed

The error message seems similar to https://github.com/NixOS/nixpkgs/issues/286, but doesn't otherwise appear related to it.

This is caused by https://github.com/NixOS/nixpkgs/issues/7810. The fix has been in master for a while and will reach the nixos-unstable channel soon.


No 'detect_fuser' while training the UFF model

Hi, first of all many thanks for publishing your very interesting work.
I have a problem with the first phase of train and testing (Train and test the double lib version and save the feature for UFF training), in particular with the training of Decision Layers Fusion with DINO+Point_MAE.
The stack error I receive is the following:
Training Dicision Layer Fusion for DINO+Point_MAE on class bagel...
Traceback (most recent call last):
  File "main.py", line 124, in <module>
    run_3d_ads(args)
  File "main.py", line 20, in run_3d_ads
    model.fit(cls)
  File "/media/data/alex/3d_anomaly_detection/M3DM/m3dm_runner.py", line 72, in fit
    method.run_late_fusion()
  File "/media/data/alex/3d_anomaly_detection/M3DM/feature_extractors/features.py", line 176, in run_late_fusion
    self.detect_fuser.fit(self.s_lib)
  File "/home/alex/torch/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1207, in __getattr__
    raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'DoubleRGBPointFeatures' object has no attribute 'detect_fuser'

Do you have any idea on what could be the problem?

If you are trying to use UFF module, use --use_uff in your python command.
Besides, I noticed there is a bug in the previous version, using new feature_extractors/features.py may help you.

Many thanks for the quick reply.
Actually I wasn't trying to use the UFF module, I was trying to follow the steps that you listed on the readme.
As far as I understood the training of the UFF it's in the second step (Train the UFF).
By the way, with the new feature.py I managed to execute the first step, but now I have a problem with the second one (Train the UFF). The stack error is the following:
/home/alex/torch/lib/python3.8/site-packages/torch/distributed/launch.py:178: FutureWarning: The module torch.distributed.launch is deprecated
and will be removed in future. Use torchrun.
Note that --use_env is set by default in torchrun.
If your script expects `--local_rank` argument to be set, please
change it to read from `os.environ['LOCAL_RANK']` instead. See
https://pytorch.org/docs/stable/distributed.html#launch-utility for
further instructions

  warnings.warn(
Traceback (most recent call last):
  File "fusion_pretrain.py", line 24, in <module>
    import data.dataset
ModuleNotFoundError: No module named 'data.dataset'
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 100591) of binary: /home/alex/torch/bin/python3
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/alex/torch/lib/python3.8/site-packages/torch/distributed/launch.py", line 193, in <module>
    main()
  File "/home/alex/torch/lib/python3.8/site-packages/torch/distributed/launch.py", line 189, in main
    launch(args)
  File "/home/alex/torch/lib/python3.8/site-packages/torch/distributed/launch.py", line 174, in launch
    run(args)
  File "/home/alex/torch/lib/python3.8/site-packages/torch/distributed/run.py", line 752, in run
    elastic_launch(
  File "/home/alex/torch/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 131, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/home/alex/torch/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 245, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
============================================================
fusion_pretrain.py FAILED
------------------------------------------------------------
Failures:
  <NO_OTHER_FAILURES>
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2023-03-27_10:21:18
  host      : hades
  rank      : 0 (local_rank: 0)
  exitcode  : 1 (pid: 100591)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
============================================================

I think it could be a version conflict, since the module it's actually installed.

This is caused by our code problem, try to change data.dataset to dataset, or use the new fusion_pretrain.py.

Thanks for trying our code, we are happy to receive your feedback!
The code of Eyecandies is already released in this git, we will give the instruction later. Or you can refer to  utils/preprocess_eyecandies.py to pre-process the data and use --dataset_type eyecandies command during training.

This error is usually caused by the dataset problem. Maybe modifying the utils/preprocess_eyecandies.py file can help you, the path used in this file may not match your dataset path. The processed Eyecandies dataset has a similar structure as the MVtec3d-AD dataset.


How to use create-react-app to add React to an existing application?

I would like to use create-react-app with this boilerplate:
https://github.com/sahat/hackathon-starter
The hackathon-starter has pug templates rendered by express. I am trying to develop a view in React using the webpackDevServer configs from create-react-app; then render this view in an element from one of the templates.
To start, I created a react app called client in the public folder,
ran npm build in the client,
added script(src='/client/build/static/js/main.9fdf0e48.js')to the layout header template, and
added a#root element to the home template;
But React is not applied to the element. An error is thrown:
https://facebook.github.io/react/docs/error-decoder.html?invariant=37
The name 9fdf0e48 of the js seems consistent but I don't know how it is generated.
How I do I load the development (or prod) build of the react js into the served page?
I appreciate any help/advice.

Try including the script in the footer. If you put it in the head, the script is executed before the dom is ready.

Also, it's better to read from asset-manifest.json from the build folder instead of hard coding the filename. The hash will be updated when you update the code.
If you still want to include the script in the head, you need to listen to the DOMContentLoaded before rendering your app.

I think it can be done if hash tokens are removed from keys in asset-manifest.json.

I ejected and added new (require('webpack-assets-manifest'))({ entrypoints: true }) to webpack config, so now it generates following file which is enough to add required assets to an existing app:
{
  "entrypoints": {
    "main": {
      "js": [
        "static/js/runtime~main.4a686d48.js",
        "static/js/1.fa92c112.chunk.js",
        "static/js/main.898e5507.chunk.js"
      ],
      "js.map": [
        "static/js/runtime~main.4a686d48.js.map",
        "static/js/1.fa92c112.chunk.js.map",
        "static/js/main.898e5507.chunk.js.map"
      ],
      "css": [
        "static/css/main.83ea7ef7.chunk.css"
      ],
      "css.map": [
        "static/css/main.83ea7ef7.chunk.css.map"
      ]
    }
  },
  "main.css": "static/css/main.83ea7ef7.chunk.css",
  "main.css.map": "static/css/main.83ea7ef7.chunk.css.map",
  "main.js": "static/js/main.898e5507.chunk.js",
  "main.js.map": "static/js/main.898e5507.chunk.js.map",
  "runtime~main.js": "static/js/runtime~main.4a686d48.js",
  "runtime~main.js.map": "static/js/runtime~main.4a686d48.js.map",
  "static/media/logo.svg": "static/media/logo.5d5d9eef.svg"
}


Non-Protocol Intersection

Is there a reason why this can't be used with non-protocol classes?
This seems like the solution I have been looking for but I need it to work with non-protocol classes.
I hope that one day Python will have a native intersection type but until then this could be a useful stopgap.

Is there a reason why this can't be used with non-protocol classes?

The implementation is explicitly only targeting protocols, as it's a relatively simple problem to solve. This plugin will only ever support intersection of protocols (hence the name ;)).
General type intersection is a much more involving problem. There's a discussion on the topic in the python/typing repository https://github.com/python/typing/issues/213 and there's actually a PEP in the works to make this happen https://github.com/CarliJoy/intersection_examples I remember reading excellent examples as to why it's not so simple somewhere in that thread, but I don't have them at hand anymore unfortunately, you'll have to dig if you're interested.

Do you know if a MyPy plugin exists/could be written to allow a normal class to be promoted to a protocol so that it could be used with your intersection?
I have been experimenting with your code and setting is_protocol=True on all the classes passed to the intersection allows your plugin to work with non-protocol classes however this effects the class outside of the intersection.

Do you know if a MyPy plugin exists/could be written to allow a normal class to be promoted to a protocol so that it could be used with your intersection?

Sorry, I don't know.

I have been experimenting with your code and setting is_protocol=True on all the classes passed to the intersection allows your plugin to work with non-protocol classes however this effects the class outside of the intersection.

I would advise against modifying this plugin in such way (making it do things it's not supposed to do), unless you're familiar with internals of mypy itself. The goal of a type checker ultimately is to make you trust the code more. If you can't trust the type-checker, it misses the point.


Add tests to crates/libcgroups/src/v2/devices/controller.rs

In order to add these tests, I added mockall to the dev dependencies.
There are some things that could be cleaner, eg using mockall's double
or cfg_if but I did not want to bloat the release dependencies with extra crates.
This required some changes to the v2/devices/bpf module, introducing an
intermediate module prog to wrap the various prog_* functions inside.
Those functions were then renamed to remove the prog_ prefix.
The addition of this intermediate module allowed us to use the automock
macro to automatically generate mocked functions. This is a useful
testing pattern that can be used in the future on the bpf module itself.
Signed-off-by: Christian Burke<EMAIL_ADDRESS>

@utam0k what do you think about this strategy?
I want to continue using it in this area, in particular for the syscall wrappers inside the bpf::prog module.
I'm thinking of adding it onto this PR

Expanded in https://github.com/containers/youki/pull/711

@cr0ax Hi, thanks for your PR. Could you confirm that compiling the release flag doesn't change the size of the binary?

@utam0k I compiled once each for

b162cf58f559d948792e7736fe0ad367d90c0cd1 and
devices_tests

using ./build.sh --release
I got the same result each time:
du -s youki
5436	youki

Added 1 last thing here, made these tests serial, if they use the same mocks it seems those mock contexts are shared.

Added 1 last thing here, made these tests serial, if they use the same mocks it seems those mock contexts are shared.

This is a little hard for us. The system call mocks the following, can we consider this one?
https://github.com/containers/youki/blob/main/crates/libcontainer/src/syscall/test.rs

@utam0k after some back-and-forth with you and some testing of the mockall library, I have changed the serial annotations to be parameterized with bpf, to limit the scope of the serial tests.
This is based on some experiments where we found that static methods share a context, whether in a module or a trait.
The recommended approach from mockall is to create a mutex and share context. Adding the serial(bpf) attribute is effectively the same strategy, and simpler.


soc: it8xxx2: add support for eSPI driver

This supports eSPI slave, virtual wire, and peripheral functions.
Signed-off-by: Dino Li<EMAIL_ADDRESS>

@keith-zephyr - any comment will be appreciated.


RFM - iehv

I DON'T Want To Become The Maintainer

[x] I have followed the Package Triage Process and I do NOT want to become maintainer of the package;
[x] There is no existing open maintainer request for this package;

Checklist

[x] Issue title starts with 'RFM - '

Existing Package Details
Package URL: https://chocolatey.org/packages/iehv
Package source URL: https://github.com/dtgm/chocolatey-packages
Date the maintainer was contacted (in YYYY-MM-DD): 2021-12-13
How the maintainer was contacted: email

Hello,
Latest update is from 2011, and IE isn't available anymore, so i think this issue can be closed.
Have a nice day.


Is there an equivalent to EF6's CreateSourceQuery in EF Core?

After I query an entity like var blog = ctx.Blogs.Single(b => b.Id == id); I want to be able to call something like:
var recentPosts = blog.PostsQ
    .OrderByDescending(p => p.DateCreated)
    .Take(5)
    .ToList();

In EF6 I could do this with the following:
public partial class Blog {
    public ObjectQuery<Post> PostsQ => Posts.CreateSourceQuery();
}

How can I do this in EF Core? I'm assuming that the return type would be different, just not sure how to implement:
public partial class Blog {
    public IQueryable<Post> PostsQ => ???;
}

@skbergam The equivalent in the DbContext API (in either EF6 or EF Core) is the Query method, accessible through the EntityEntry. For example:
using (var context = new BloggingContext())
{
    Blog blog = context.Set<Blog>().Single();

    IQueryable<Post> postsQuery = context.Entry(blog).Collection(e => e.Posts).Query();
}

The more modern APIs in EF6 and EF Core are predominantly designed to work with "POCO" entity types that do not directly use EF inside the class, so there isn't an obvious equivalent for putting this in the entity type itself. However, if all you need is to be able to load posts dynamically, then you might consider using lazy-loading.


Discrepancy in ActionsQueue action space

The bounds on repeat and n is not the same throughout the code. Is this intended?
Since this is being looked at, is config.max_transfer_amount + 1 a minor bug? This defines a transfer limit that is 1 above the one set in env_cfg.
In docs: https://github.com/Lux-AI-Challenge/Lux-Design-S2/blob/main/docs/advanced_specs.md#environment-step---normal-phase-validating-actions-via-action-space
{
  [robot id]:
    ActionsQueue(
      spaces.Box(
        low=np.array([0, 0, 0, 0, 0, 1]),
        high=np.array([5, 4, 4, config.max_transfer_amount + 1, 1, 9999]),
        shape=(6,),
        dtype=np.int64,
      ),
      config.UNIT_ACTION_QUEUE_SIZE
    )
  [factory id]: spaces.Discrete(3)
}

In https://github.com/Lux-AI-Challenge/Lux-Design-S2/blob/main/luxai_s2/luxai_s2/spaces/obs_space.py#L143-L151
obs_dict["action_queue"] = ActionsQueue(
                spaces.Box(
                    low=np.array([0, 0, 0, 0, 0, 1, 0]),
                    high=np.array([5, 4, 4, config.max_transfer_amount + 1, 9999, 9999]),
                    shape=(6,),
                    dtype=np.int64,
                ),
                config.UNIT_ACTION_QUEUE_SIZE,
            )

In https://github.com/Lux-AI-Challenge/Lux-Design-S2/blob/main/luxai_s2/luxai_s2/spaces/act_space.py#L139-L147
act_space[u.unit_id] = ActionsQueue(
            spaces.Box(
                low=np.array([0, 0, 0, 0, 0, 1]),
                high=np.array([5, 4, 4, config.max_transfer_amount + 1, 9999, 9999]),
                shape=(6,),
                dtype=np.int64,
            ),
            config.UNIT_ACTION_QUEUE_SIZE,
        )

My guess is that https://github.com/Lux-AI-Challenge/Lux-Design-S2/blob/main/luxai_s2/luxai_s2/spaces/act_space.py#L139-L147 is correct since it has the most precise documentation in the lines above it.
ActionsQueue(
            spaces.Box(
                low=np.array([0, 0, 0, 0, 0, 1]),
                high=np.array([5, 4, 4, config.max_transfer_amount + 1, 9999, 9999]),
                shape=(6,),
                dtype=np.int64,
            ),
            config.UNIT_ACTION_QUEUE_SIZE,
        )

https://github.com/Lux-AI-Challenge/Lux-Design-S2/blob/main/luxai_s2/luxai_s2/spaces/act_space.py#L120
This line is also inaccurate. Should be (max_queue, 6)


CSS terminal values extracted as non-terminals

For example, annotation(<feature-value-name>) is extracted as "<annotation()>": { "value": "annotation(<feature-value-name>)" } from CSS Fonts 4 but <annotation()> does not appear in any value definition.
annotation(<feature-value-name>) is marked up with data-dfn-type="value" and according to https://github.com/w3c/csswg-drafts/issues/7380#issuecomment-1159125937, it should be treated the same as a keyword:

While our functions are often uniquely named, they should in fact be treated the same as keywords - potentially context-sensitive.

I think it should be extracted as "annotation(<feature-value-name>)": { "prose": "Enables display of alternate annotation forms (font specific, OpenType feature: nalt <feature-index>)" } or just ignored (like keywords).
Note that some non-terminals looks like function, eg. <calc()>, and some terminals are annoted between angled brackets, eg. <number>. <image-set()> is an example of a function definition (with data-dfn-type="function") referenced as a production (terminal - CSS basic data type - or non-terminal wrapped between < and >) from another definition, <image>:

<image> = <url> | <image()> | <image-set()> | <cross-fade()> | <element()> | <gradient>

Note that in its definition, <image-set()> is not wrapped between < and >, which may seem confusing...:

image-set() = image-set( <image-set-option># )

...whereas eg. <calc()> is defined with data-link-type="function" as:

<calc()>  = calc( <calc-sum> )

Yes, we could look into data-dfn-type to create entries that go into valuespaces, and e.g. only consider the ones defined with a data-dfn-type="function". That would provide more consistent results with fewer dangling values, even though we would miss a number of entries that are currently defined as value whereas it seems to me that they should rather be function. Typically, I would expect to see annotation() defined as function, and referenced through <annotation()> from the definition of font-variant-alternates so as to be able to create a link between to the definition of annotation, as done in some other CSS specs.
Are dangling values problematic though? If the goal is to get rid of dangling values from the extracts, we could also just loop through values during data curation and drop them.
On top of looking at data-dfn-type, what seems more important to me is to save the data-dfn-for attribute in the CSS extracts. That would provide the contextual flag I was alluding to in https://github.com/w3c/webref/issues/127#issuecomment-1098022559:

From a CSS extract perspective, we should rather find a way to flag the rect() definition in CSS Masking, e.g. as only applying locally.

That would also allow us to extract terminal values along with a bit of prose. Terminal values are much more likely to have contextual definitions (e.g. auto or none).

Keeping functions defined with data-dfn-for="value" in valuespaces is not really a problem to me. It would make sense to also have keywords (eg. auto or none) and to associate their contextual definition (prose and data-dfn-for).
Personally, I would ignore them when generating a productions (defined with type or function as the value for data-fn-type) record since they are useless for parsing a CSS input. Those that are currently associated to a value are errors (selector(), layout(), etc.).
Referencing them in valuspaces as productions, ie. with their name between < and >, would make this task a little more difficult but more importantly it seems inappropriate. From https://github.com/w3c/fxtf-drafts/issues/411#issuecomment-1159148605:

Naming conflicts in productions are not allowed, but they do show up. Happy to fix them when they're pointed out.

That means that productions must be defined once.
I believe there is no example of a function defined with data-dfn-for="value" that is used with different value definitions.
fit-content() is defined in CSS Grid and CSS Sizing with different semantic definitions.
annotation(), which is currently only used as annotation(<feature-value-name>) in the value definition of the font-variant-alternates property, could for example be set with annotation(<number>) in the value definition of another property or a production, associated to its own contextual definition, and possibly to specific parsing or serialization rules.

Extraction rules have now been adjusted in Reffy, and released in @webref/css v4.3.0.
Taking data-dfn-for attributes into account would allow us to drop remaining cases that are currently handled in Webref as exceptions-to-the-rules to get back to only one definition. Follow-up issue in #980.

Thank you for this update. I guess it must be super fun to parse the contents of these specs.
I can not figure out how <layout()>, which is marked up with data-dfn-type="value", is still extracted from CSS Layout API, from reading the code for the extract-cssdfn module.

I can not figure out how <layout()>, which is marked up with data-dfn-type="value", is still extracted from CSS Layout API, from reading the code for the extract-cssdfn module.

It is no longer extracted but since that's the only thing that was extracted from the spec, removing it means removing the extract, and we don't remove extracts automatically from webref (this is done on purpose to keep extracts around when there is a transient error that makes it impossible to crawl a spec on a temporary basis). We review dangling extracts once per week. I just did it in #649 and dropped the offending extract.

Got it, thanks.


[BUG] Unable to open Roblox - only with Bloxstrap

Acknowledgement of preliminary instructions

[X] I have read the preliminary instructions, and I am certain that my problem has not already been addressed.

What problem did you encounter?
I'm unable to run Roblox using Bloxstrap after installing NTVDMx64. The process is in task manager, but there's no window.

I've tried reinstalling + rebooting, it didn't work.

i uninstalled bloxstrap and reinstalled the normal roblox. it seemed to work for me.

I'm unable to run Roblox using Bloxstrap after installing NTVDMx64

I have a feeling that you've already just pointed out the cause of your issue here. As I asked in the preliminary instructions, does it happen without Bloxstrap?

Can I see the list of FastFlags you have configured in the menu?

Uninstalling NTVDMx64 fixes it though

...Yet somehow Roblox by itself works completely fine with it installed?

Yup...

Just throwing this out there, does disabling all your integrations change anything?

Nope

I'm not really sure if there's even anything I can do here, but at this point I just want to ensure that your Roblox install is identical to the stock one. Go to the folder where Bloxstrap is installed to (typically %localappdata%\Bloxstrap), send the contents of State.json here. Then, go into the version folder, delete the ClientSettings folder, and then start RobloxPlayerBeta.exe manually. See if anything different happens.

State.json
Nothing happened.

Can you send me a copy of your Bloxstrap installation folder? Throw everything into a .zip archive, upload it somewhere, send a link to it here.

https://sourceforge.net/projects/nxvdm/

This is happening to me; I have tried re-installing it myself aswell it works on ONE game then when I leave it, (sometimes it doesnt even do one game) it does the same thing, appears in the background but no window.

@TheRabbit75 As far as I'm concerned you're talking about something completely different. Open a separate issue for it.

Uninstalling NTVDMx64 fixes it though

...Yet somehow Roblox by itself works completely fine with it installed?

How was it possible? Was it something to do with the latter's ldntvdm.dll interfering with the former?

Uninstalling NTVDMx64 fixes it though

...Yet somehow Roblox by itself works completely fine with it installed?

How was it possible? Was it something to do with the latter's ldntvdm.dll interfering with the former?

Not sure. Installing ntvdm seems to affect cmd/conhost as well, changing it to the legacy console. (probably unrelated, but just throwing that out there)

Is this still occurring?

Client issues are no longer accepted.


Location based AR - Depth and distance problem

Hello everyone,
I am trying to build a location based AR project and have encountered following issues:

I loaded my gltf model on a local castle in my town. The castle is around one kilometer air distance from my house. The problem is that the model loads the moment I open my app. I want the model to be visible only when the user is around 100 meters of the castle(provided longitude/latitude of the model). I tried using the maxDistance attribute on the gps-camera but the model still loads when opening the app. Another feature that I want to implement is that model shows only in front of the castle, not when you are looking at it from the back. Something like a semicircle with center in the given location and radius of 100 meters to the front of the castle. Any ideas on how to implement this?

Another problem I found is that the scene doesn't have the feeling of depth. What I see when the model is loaded is just a model sticked on the screen in the direction of the location provided in entity.  When you do it you can still see the model normally in front of my hand like there isn't and interruptions. Also I can see the model through walls. Model should be barely visible behind my hand. Any ideas how to fix this?

Here is the basic example of code I used and screenshot of a problem(once again a reminder that the location is one kilometer from my house)
Code: https://github.com/matijafumic/AR/blob/main/index.html

Hi @matijafumic. to set the maximum distance you can adjust the far property of your camera, e.g. if you do
<a-camera gps-projected-camera near='0.1' far='100' ...>
then only content up to 100m away will be displayed.
The 'hand' problem would be a hard one to solve. The way AR.js (and probably the majority of location-based AR toolkits) works is that the augmented content is always drawn on top of the camera feed, never behind. To get this to work as you describe, we'd have to somehow calculate the real distance of real-world objects (such as your hand) in the camera feed, and then display augmented content either in front of or behind the camera feed depending on this, which would be a diffficult problem to solve.

Thank you very much for your help! I get what you are saying yeah. I'll try to do some research and if I manage to find something out, I'll post here. However do you think it would be possible to get what I want using unity?

@nickw1 @matijafumic been experiencing a similar problem like Matija. I just can't get that sense of depth on a position of a model placed in the scene. If I place it on a location that's 50 meters away from me when I look it through my phone it looks like the model is 2 meters away from me. Is there any solution for that problem?

@bere3 is this because you need to scale the model down to a more realistic size? If you move away from the model location, does it get smaller?


AccountController and group routes

This PR Fixes following

Error on 500 status code
From

// 500 error
Route::get('500', function () {
    echo $fail;
});

To
// 500 error
Route::get('500', function () {
    abort(500);
});

Group all the route middlewares

while analysing the web.php file there is a auth middleware in most of the web routes and its declared in each of the routes so its grouped together and added the auth middleware

Added AccountsController
in the routes the AccountsController is used but its never created.

Sir Do check and inform me if there is any changes to be done

check now sir its fixed

@ManojKiranA I really appreciate you finding these issues, but since this PR still has a lot of changes I don't really want, I've simply made those changes to the repo myself. I'm going to close this PR...but again, I appreciate your efforts! 👏

ok sir no problem


Remove unnecessary data prop

Hey, thanks for this awesome project! A small fix which I think doesn't hurt anything.

Nice and small fix!
Before this fix, when using the airbnb rules for eslint, we get the following problems reported:
"no-unused-vars": "error 'data' is defined but never used" (line 9)
"react/prop-types": "error 'data' is missing in props validation" (line 9)
"no-shadow": "error 'data' is already declared in the upper scope" (line 20)

No problem, glad to help :)


Update latest download link

It'll be helpful if you can update the Latest Download link on expandframework.com to point to https://github.com/expand/eXpand/releases and not https://github.com/expand/eXpand/issues.

the link you posted points to https://github.com/expand/eXpand/releases and not to issues, can you be more specific?

The redirect link in expandframework.com point to github issues page. I feel that it should be pointing to release page.
Attached screenshot highlights the issue:

fixed


varcheck: unused consts are reported as "global variable"

varcheck reports both unused variables and unused named constants.
But the MessageOverride in gometalinter is only about variables: unused global variable {message}
Test case:
package main

const x = 3

$ gometalinter -D deadcode -E varcheck
main.go:3:7:warning: unused global variable x (varcheck)

Workaround: fix message override from the command line
$ gometalinter -D deadcode -E varcheck '--message-overrides=varcheck:unused variable or constant {message}'
main.go:3:7:warning: unused variable or constant x (varcheck)

Thanks, though it probably would have been faster to send me a PR!


[Misc] Encapsulate logical gates in custom gates

It would help with visualization I think if we encapsulate logical X, Y, Z and stabilize with custom gates, so that circuit visualizations are more reasonable. Then, if a user wants to decompose the circuit further they can to see the same circuit visualizations that we currently have.

This is semi-related to this issue : #44
Also, it would be cool if we could show the internal gates that compose a logical X gate (for example) and also denote that these gates constitute an overall logical X (i.e. add in some text over the barrier constrained region with the logical X gates).


App\sage not defined when deploying using Trellis

I'm deploying using Trellis, and since Controller is a mu-plugin, I'm guessing that it gets enabled before Sage is enabled and as such is throwing a fatal error during a trellis deploy process, thereby exiting the deploy.
PHP Fatal error:  Uncaught Error: Call to undefined function App\\sage() in /srv/www/viewfind.com/releases/20170228183833/web/app/mu-plugins/controller/controller.php:46
Can we check that the namespace exists before calling the function?

Hey @aaroneight cool, I will take a look at this tomorrow. Just appears to the debugger calling the Blade directive.

@aaroneight just pushed 1.0.0beta1.2 with a check to see if the function is defined, let me know if that works.


pull request for e2e test at 01/20/2017 20:56:18

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


How to use imgaug with Detectron ?

Any example or suggestion ?

I also encounter the same question, did you find the answer?

any update on this?


_process_raw_data refactored to create market objects and call market…

….closed_market when closed
marketrecorder strategy updated to use process_closed_market and load catalogue data on closure

Coverage decreased (-0.1%) to 93.58% when pulling ece3e463488c7e3ced2f2d76afd7282844e6b914 on task/catalogue-recording into b1b07919b15206d523653fdd183a8347034cc75e on release/1.10.2.


fix: Usage pulse delivery for invalid user where manage user permission is missing

Description

As we are using updateUser method within UsagePulse (api/v1/usage-pulse) API to update the hashedEmail. This requires the user to possess required permission but as this is a internal usecase it's safe to expose a method to perform the update operation.

Fixes https://github.com/appsmithorg/cloud-services/issues/226
Type of change

Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

Manual

Checklist:
Dev activity

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[ ] PR is being merged under a feature flag

QA activity:

[ ] Test plan has been approved by relevant developers
[ ] Test plan has been peer reviewed by QA
[ ] Cypress test cases have been added and approved by either SDET or manual QA
[ ] Organized project review call with relevant stakeholders after Round 1/2 of QA
[ ] Added Test Plan Approved label after reveiwing all Cypress test

/ok-to-test sha=33a86d9

/ok-to-test sha=33a86d9b7c099b73f4c36e0d423658c166e1bcc7


addBatch method cause Java heap memory char array grows too fast

The scenario involves a large number of calls to the addBatch method, Then use jmap to view heap memory，has lots of char[],although they are all unreachable objects that can be GC dropped.But it will lead to frequent GC and high CPU usage. How to solve it

Hi @wangbing505

What is the ClickHouse jdbc version do you use?
Is it a requirement to use JDBC?

Hi @chernser

ClickHouse jdbc version is 0.2.6，Can a higher version solve this problem？
yes，or  Is there any other way to write data to clickhouse ?

@wangbing505

Yes, please, update to the latest 0.7.0 version. I do not see such code in it and everything should work fine.
Yes, you may use java client without JDBC - https://clickhouse.com/docs/en/integrations/java/client-v2

Please let me know how the update goes. Thanks!

@chernser
Thank you for your answer，I will try using them and synchronize the results

@wangbing505 as the issue solved?

@chernser hello, I have been delayed by other things and will try it next week

@chernser Thank you, I tried it, Char [] has decreased，But there are other memory uses as well，maybe it's not a problem with the client anymore


Ultimate test of parsing

Write a test to:

Iterate through Rust files in a directory
Parse using syn >> print using syn >> parse using syntex
Parse the original using syntex
Assert that the syntex AST from 2 and 3 are identical

I think this would give us more confidence in the parser than any unit test suite we could write, and it is also far easier to implement than individually testing every parser element. Once we make more progress on #4 we could run this test against a large repo like the full rustc source code.
cc @gregkatz. I plan to work on this today or tomorrow. In the meantime you don't have to worry about unit testing parsers you write (unless it is useful to you in implementing them). Let's focus on flying through #4 without tests and leave testing to this one.

Sure thing. I'll skip the unit tests in the future.

Added in 4a51dc70fc60b4030753ab6cfdf06e23eb522d63. Any *.rs file in the tests/cases directory will be tested as outlined above.


[Bug] vl3-basic example is incorrectly converted into the dashboard graphical model

Example: https://github.com/networkservicemesh/deployments-k8s/tree/main/examples/features/vl3-basic
Cause: nodes contain interfaceConnection-s without a "source"
Need to explain why such connections are added, fix it, and add validation/logging.

Seems like done


Paired T-Test visualization is broken in 1.0.0

The Paired T-Test visualization type does not work in superset 1.0.0 when running from a fresh clone of the superset repo against one of the example datasets.
Expected results
I expect the T-Test plugin to render a table with p-values.
Actual results
I see an error message stating An error occurred while rendering the visualization: TypeError: Cannot read property 'map' of undefined
Screenshots

How to reproduce the bug

Clone the superset repo: git clone https://github.com/apache/superset.git
Checkout the 1.0.0 tag: cd superset && git checkout tags/1.0.0
Start the superset server: docker-compose up
Go to http://localhost:8088 in a browser and click on "Datasets"
Click on the "Video Game Sales" Dataset
Click on the "Table" visualization type and change it to "Paired t-test"
Set up the visualization as follows: Time filter: No filter, metric: sum(global_sales), group by genre
Click "Run Query"
Get the error

Environment
(please complete the following information):

superset version: superset version

Loaded your LOCAL configuration at [/app/docker/pythonpath_dev/superset_config.py]
INFO:superset.utils.logging_configurator:logging was configured successfully
/usr/local/lib/python3.7/site-packages/flask_caching/__init__.py:192: UserWarning: Flask-Caching: CACHE_TYPE is set to null, caching is effectively disabled.
  "Flask-Caching: CACHE_TYPE is set to null, "
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Superset 1.0.0
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

python version: python --version

Python 3.7.9

node.js version: node -v

v12.20.1

Checklist
Make sure to follow these steps before submitting your issue - thank you!

[x] I have checked the superset logs for python stacktraces and included it here as text if there are any.
Server logs:

superset_app             | DEBUG:parsedatetime:eval - with context - False, False
superset_app             | INFO:superset.viz:Cache key: 37e8c9c39567b33a73e84dd8215ca361
superset_app             | DEBUG:superset.models.core:Database.get_sqla_engine(). Masked URL: postgresql://superset:XXXXXXXXXX@db:5432/superset
superset_app             | INFO:superset.connectors.sqla.models:SELECT genre AS genre, year AS __timestamp, SUM(global_sales) AS sales
superset_app             | FROM video_game_sales GROUP BY genre, year
superset_app             |  LIMIT 10000
superset_app             | DEBUG:superset.stats_logger:[stats_logger] (incr) loaded_from_source
superset_app             | DEBUG:superset.stats_logger:[stats_logger] (incr) loaded_from_source_without_force
superset_app             | DEBUG:superset.stats_logger:[stats_logger] (incr) set_cache_key
superset_app             | DEBUG:superset.stats_logger:[stats_logger] (incr) explore_json
superset_app             | <IP_ADDRESS> - - [08/Feb/2021 18:56:16] "POST /superset/explore_json/ HTTP/1.1" 200 -
superset_app             | INFO:werkzeug:<IP_ADDRESS> - - [08/Feb/2021 18:56:16] "POST /superset/explore_json/ HTTP/1.1" 200 -
superset_app             | DEBUG:superset.stats_logger:[stats_logger] (incr) log
superset_app             | <IP_ADDRESS> - - [08/Feb/2021 18:56:17] "POST /superset/log/?explode=events HTTP/1.1" 200 -
superset_app             | INFO:werkzeug:<IP_ADDRESS> - - [08/Feb/2021 18:56:17] "POST /superset/log/?explode=events HTTP/1.1" 200 -

[x] I have reproduced the issue with at least the latest released version of superset.
[x] I have checked the issue tracker for the same issue and I haven't found one similar.

Additional context
This error comes from a fresh install of superset 1.0.0

@akud Thank you so much for reporting issues at such detailed level! I am able to reproduce, this is happening to all dataset.
we can't get to it soon unfortunately 😞
If anyone would like to contribute, according to our committer, this is likely a frontend error caused by the server, and the broken line could be found in the ui plugin code. 🙏


TFS character limit reached due obscenely long folder name.

Feature Request: rename a folder in the project.
we are seeing a 95 character folder name in the connectivity plugin. can this name please be shortened as it is causing issues with our legacy project and TFS' 259 character limitations.
portable-net45+wp80+wp81+wpa81+win8+MonoAndroid10+MonoTouch10+Xamarin.iOS10+Xamarin.Mac20+UAP10 is the folder name in question.
Could it be shortened some?

I have an idea :) Give me a few days.

They have been shortened. You can now find a custom nuget feed on: https://ci.appveyor.com/nuget/connectivityplugin
New official nuget version will be soon


Fix CORS server domain list

The CORS require server host url explicitly set, also the port need
be included.
For local deployment, the port need be added to the server url,
while with DEFAULT openshift edge tls termination, the port should
not be added.
So add new TARGET and SSL_ENABLED parameters to identify the
deploy target env and tls status, and update the server list
accordingly.
By default with Helm chart deploy, the value with TARGET and
TLS_ENABLED don't need be set in the Helm chart as default in the
app/core/config.py are set with:
TARGET: openshift
TLS_ENABLED: true
Signed-off-by: Wayne Sun<EMAIL_ADDRESS>

With the server url parsed accordingly with local and openshift deployment, this fixes the url issue in:
https://github.com/waynesun09/notify-service/issues/23


It gets blocked by windows defender

Hi! The editor gets blocked by windows defender, is it something serious or just a false positive? I'm so sorry for asking this, I'm just very terrified of malware uwu

Should be a false positive. If anything you can download and compile yourself. Btw this does not work for MHST2


market_miner attempt to import from 'examples' which is not part of liualgotrader's python package

Describe the bug
I finally had some time again to come back to this project :-)
Unfortunately it seems that the current market_miner script attempts to import from 'examples' which is not part of  liualgotrader's python package. Even running from the root of the clone this fails. The only workaround seems to copy the examples folder into the examples folder into the site-packages folder of the current .venv (e.g. cp -R examples /.venv/lib/python3.8/site-packages/)
To Reproduce
Steps to reproduce the behavior:

create a fresh venv & install liualgotrader's

rm -rf .venv
python3.8 -m virtualenv -p python3.8 .venv
python setup.py install

2 call market_miner
$ market_miner
Traceback (most recent call last):
  File "/.venv/bin/market_miner", line 4, in <module>
    __import__('pkg_resources').run_script('liualgotrader==0.0.86', 'market_miner')
  File "/.venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 667, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/.venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1471, in run_script
    exec(script_code, namespace, namespace)
  File "/.venv/lib/python3.8/site-packages/liualgotrader-0.0.86-py3.8.egg/EGG-INFO/scripts/market_miner", line 16, in <module>
ModuleNotFoundError: No module named 'examples'

Expected behavior
No Exception :-)

Nice to have you back, and thanks for bringing this to my attention. The root cause of the exception is that the installation package also includes miner.toml file, this file should not be part of the installation package, it should only exist in the examples folder. I will make sure to remove it in the next build. Will also improve error messaging when a module name can't be found. It will be part of the next build. Thanks much.

Nice to have you back, and thanks for bringing this to my attention. The root cause of the exception is that the installation package also includes miner.toml file, this file should not be part of the installation package, it should only exist in the examples folder. I will make sure to remove it in the next build. Will also improve error messaging when a module name can't be found. It will be part of the next build. Thanks much.

Welcome back and thank you for bringing this to my attention. I've released v.0.0.87 which I believe resolves this issue. Kindly let me know if it's working for you too now. thanks much.

Welcome back and thank you for bringing this to my attention. I've released v.0.0.87 which I believe resolves this issue. Kindly let me know if it's working for you too now. thanks much.

Indeed with https://github.com/amor71/LiuAlgoTrader/commit/38f139faee674c16c7b85c747498dcf3400ee59e this can be considered fixed! Thanks again for the quick turnaround!

Indeed with https://github.com/amor71/LiuAlgoTrader/commit/38f139faee674c16c7b85c747498dcf3400ee59e this can be considered fixed! Thanks again for the quick turnaround!


new(git-repos): accept new host keys when fetching git repos

The git module will only accept host keys that are not present or are the same as existing ones.

Close and reopen.
/close

@FedeDP: Closed this PR.

In response to this:

Close and reopen.
/close

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/reopen

@FedeDP: Reopened this PR.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: alacuku
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [alacuku]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: alacuku, FedeDP
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [FedeDP,alacuku]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: 7da3ba73bd6b9c2490cad30911fdec9a7b1beea0


Product projection is not a separate resource

The product projection mocking is not implemented correctly. It should be an interface which searches the product resources instead.
So the same as for example the my-* services

Fixed in 0.7.0


Dynamic Stickers

Description

Wiring the plugin into my assets infrastructure.
Plugin checks to see if the current sticker being used is the most up to date.
If the sticker is not the same in the infrastructure, then it downloads and sets that as the current used sticker.
Motivation and Context

Right now all of the stickers are drawn in all different styles, I eventually want to migrate them to look roughly like this style:

Just Monika
Natsuki
Mioda Ibuki

How Has This Been Tested?

It works offline if you have already downloaded the sticker.
Types of changes

[X] Non-Functional Change (non-user facing changes)

Checklist:

[X] I updated the version.
[X] I updated the changelog with the new functionality.

[ ] Works on Windows 10

[ ] Works on Linux

[ ] Works on Remote Code

[ ] Works on MacOS


function with cosmos db output binding complains about extension not being installed

I added a second output binding to my function using the vscode extension.
When I try to run it locally, I get an error:
"The binding type(s) 'cosmosDB' are not registered. Please ensure the type is correct and the binding extension is installed."
I figured out that host.json was not updated ( related #1343 ) so I added:
{
  "version": "2.0",
  "extensions": {
    "cosmosDB": {
        "connectionMode": "Gateway",
        "protocol": "Https"
    }
}
}

But, I still get the same error as before, no change.
I cannot find anything about installing a cosmos db binding extension.
Environment

Version: 1.36.1 (user setup)
Commit: 2213894ea0415ee8c85c5eea0d0ff81ecc191529
Date: 2019-07-08T22:59:35.033Z
Electron: 4.2.5
Chrome: 69.0.3497.128
Node.js: 10.11.0
V8: 6.9.427.31-electron.0
OS: Windows_NT x64 10.0.18362

(.env) PS: github\project> python --version
Python 3.6.8

@askpatrickw the extensions property has a different purpose. You'll want to add the extensionBundle property to your host.json. It contains several extensions, including Cosmos DB - see here for more info.
{
  "version": "2.0",
  "extensionBundle": {
    "id": "Microsoft.Azure.Functions.ExtensionBundle",
    "version": "[1.*, 2.0.0)"
  }
}

NOTE: Make sure your func cli is up to date. This "extensionBundle" feature was just added last May - not sure of the specific version but I can check if you need me to
Assuming the above works, I think we can close this as a duplicate of #1343

(added a comment with the workaround to #1343)

func --version 2.7.1480
This is latest according to https://github.com/Azure/azure-functions-core-tools/releases

And now I get this error.
"Value cannot be null.
Parameter name: provider"
I deleted my function.
Created a new function bound to http. It starts fine.
I added an additional outbound binding of cosmosdb.
Debug will not start with the error I pasted above.

Okay yeah I could reproduce that error. Try running this command from the VS Code Command palette:

The functions runtime needs a storage account for internal use in order to debug and that command will help you set it. If that fixes it, we should probably file an issue on the func cli to have a better error message than "Value cannot be null. Parameter name: provider"

Yep.. they did it.  My $0.0002.

Yeah, that error message could be way more helpful.
The docs are kind of spread out all over the place. And its non-obvious all the steps which need to happen. Maybe listing all the components which may need to be installed on the plugin page more like a check list. "You will need to install: 1) The plugin, 2) the tools, 3) maybe one of the following emulators. And a section on using storage with emulators or directly online.

Just a sugestion…
Thank you for the help though!  You unblocked me and I successfully wrote data to cosmosdb!

Filed an issue on the error message here: https://github.com/Azure/azure-functions-core-tools/issues/1460
I can't help much on the docs, but we do have a few issues to improve this in VS Code, closing as a duplicate of those. https://github.com/microsoft/vscode-azurefunctions/issues/1343 https://github.com/microsoft/vscode-azurefunctions/issues/1332


NullPointerExcepton

Hi ,I run the testParser1 method in TestFastNLPProcessor .
it throw NullPointerExcepton for the below lines:
// sista: it is possible that model files are included in a jar with a different name
// (this happens in the jar distribution of processors)
// inspect all jars in the classpath, to see if any contains our model files
if(! found) {
String infoFile = getName() + File.separator + getName() + "_singlemalt.info";
InputStreamReader inputReader = new InputStreamReader(this.getClass().getClassLoader().getResourceAsStream(infoFile));
i guess it did not import the model file .  but i already import processors-2.1.jar and And I import the processors-2.1-model.jar also.  I donot know where the issue ?   (I have tested the corenlpTest , it is ok)

Haven't heard anything on this since Jun 11, 2014.  I'm going to close this, but we can always reopen if needed...


Unable to start instance in kaggle notebook

Steps

! pip3 install nebulagraph-lite
from nebulagraph_lite import nebulagraph_let as ng_let
n = ng_let()
n.start()

Collecting nebulagraph-lite
  Downloading nebulagraph_lite-0.2.2-py3-none-any.whl.metadata (7.5 kB)
Collecting udocker==1.3.13 (from nebulagraph-lite)
  Downloading udocker-1.3.13-py2.py3-none-any.whl.metadata (36 kB)
Collecting psutil>=5.9.6 (from nebulagraph-lite)
  Downloading psutil-5.9.8-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (21 kB)
Collecting nebula3-python>=3.5.0 (from nebulagraph-lite)
  Using cached nebula3_python-3.5.0-py3-none-any.whl.metadata (8.4 kB)
Requirement already satisfied: future>=0.18.0 in /opt/conda/lib/python3.10/site-packages (from nebula3-python>=3.5.0->nebulagraph-lite) (1.0.0)
Requirement already satisfied: httplib2>=0.20.0 in /opt/conda/lib/python3.10/site-packages (from nebula3-python>=3.5.0->nebulagraph-lite) (0.21.0)
Requirement already satisfied: pytz>=2021.1 in /opt/conda/lib/python3.10/site-packages (from nebula3-python>=3.5.0->nebulagraph-lite) (2023.3.post1)
Requirement already satisfied: six>=1.16.0 in /opt/conda/lib/python3.10/site-packages (from nebula3-python>=3.5.0->nebulagraph-lite) (1.16.0)
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in /opt/conda/lib/python3.10/site-packages (from httplib2>=0.20.0->nebula3-python>=3.5.0->nebulagraph-lite) (3.1.1)
Downloading nebulagraph_lite-0.2.2-py3-none-any.whl (16 kB)
Downloading udocker-1.3.13-py2.py3-none-any.whl (118 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 118.6/118.6 kB 4.8 MB/s eta 0:00:00
Using cached nebula3_python-3.5.0-py3-none-any.whl (326 kB)
Downloading psutil-5.9.8-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (288 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 288.2/288.2 kB 12.3 MB/s eta 0:00:00
Installing collected packages: udocker, psutil, nebula3-python, nebulagraph-lite
  Attempting uninstall: psutil
    Found existing installation: psutil 5.9.3
    Uninstalling psutil-5.9.3:
      Successfully uninstalled psutil-5.9.3
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
beatrix-jupyterlab 2023.128.151533 requires jupyterlab~=3.6.0, but you have jupyterlab 4.1.2 which is incompatible.
distributed 2023.7.1 requires dask==2023.7.1, but you have dask 2024.2.0 which is incompatible.
jupyterlab 4.1.2 requires jupyter-lsp>=2.0.0, but you have jupyter-lsp 1.5.1 which is incompatible.
jupyterlab-lsp 5.0.3 requires jupyter-lsp>=2.0.0, but you have jupyter-lsp 1.5.1 which is incompatible.
raft-dask 23.8.0 requires dask==2023.7.1, but you have dask 2024.2.0 which is incompatible.
Successfully installed nebula3-python-3.5.0 nebulagraph-lite-0.2.2 psutil-5.9.7 udocker-1.3.13

After taking a look at the Kaggle notebook, it appears to be quite different from the standard Jupyter Lab/Notebook.

What difference are you seeing ? Env is different in collab and kaggle notebooks. I didnt check locally

What difference are you seeing ? Env is different in collab and kaggle notebooks. I didnt check locally

It's banning all long run processes so running things in background is not feasible in kaggle notebooks 😭

seems like you need async process support then


Is there a way to retrain face recognition (dlib/compreface) without restarting Viseron?

Hello,
I'm using Viseron with dlib and love it so far! Forgive me if I am putting this in the wrong spot as I don't really see it as an "Issue". I just have a question how to do something and this seemed like the best place to do so.
I would like my own software to be able to add a new person to the face recognition path and have viseron retrain the face recognition model without restarting viseron. I'm wondering if this is possible.
I know I can change the face recognition path to anywhere I want so I can easily write any files I want in there. (Though my preference would be to do this remotely through a viseron endpoint so that Viseron can sit on it's own server without any middleware on it). But once doing so, Viseron appears to need to restart in order to retrain the model and be able to detect the new face.
I've also experimented with Compreface, (I still need to do more, but stopped because dlib was working). I was having trouble with it detecting a face. From what I understand it goes off of the faces in the same Viseron path and NOT the faces added to any application created inside Compreface itself. Compreface has the ability to retrain when a new person is added (if I am not mistaken). So is there a way to use the faces inside Compreface instead of the file structure in Viseron?
I appreciate any response even if it is that Viseron can and will not do any of these things. Worst case scenario I can create a small api server to receive an image and write it appropriately and use the websocket to restart Viseron when a new person is added. This of coarse means viseron will be effectively going down momentarily every time a new person is added to the system.
Thanks,
P.S. another option, if you point me in the right direction I can experiment with building Viseron from code and attempt a PR where an endpoint will do exactly this (receive face/name, update files, retrain model).

I would like my own software to be able to add a new person to the face recognition path and have viseron retrain the face recognition model without restarting viseron. I'm wondering if this is possible.

Not possible sadly, but there is no technical limitation or anything like that. I simply have not had the time to do it :(

I've also experimented with Compreface, (I still need to do more, but stopped because dlib was working). I was having trouble with it detecting a face. From what I understand it goes off of the faces in the same Viseron path and NOT the faces added to any application created inside Compreface itself. Compreface has the ability to retrain when a new person is added (if I am not mistaken). So is there a way to use the faces inside Compreface instead of the file structure in Viseron?

As long as the persons trained in Compreface is the same as the foldernames in the Viseron config i think it should work.
So if you train a face in Compreface called john, and create an empty dir called /config/face_recognition/faces/john/ and set train: false for compreface it should work.
Can you try that out?

P.S. another option, if you point me in the right direction I can experiment with building Viseron from code and attempt a PR where an endpoint will do exactly this (receive face/name, update files, retrain model).

I would love that! I am always looking for more contributions.

Thanks for the reply,
One last question from what you responded with.

So if you train a face in Compreface called john, and create an empty dir called /config/face_recognition/faces/john/ and set train: false for compreface it should work.

If Viseron is running and there are 3 foldernames in the Viseron config and train is false, we add a new person/image to compreface, and create a new folder in the config to match that person, will Viseron need to restart in order to match that person? (I assume yes based on how binary facial recognition events are populated).
Thanks,

I've managed to implement all the features I needed and tested them successfully.
I haven't finished fixing all the linting errors, so I haven't been able to commit yet to my fork. Once I get through them I'll submit a pr.
I've implemented a new compreface config variable called "use_subjects" which defaults to false.
When set to true it blocks the binary face recognition entities from being created based on the folder structure. Instead when compreface initializes it pulls all subjects from compreface and generates the entities with them.
I then created a new endpoint at "api/v1/compreface/update_subjects", this will fetch the subjects from compreface and add any face recognition entities that are missing.
This way a user can manage subjects through compreface's API and/or SDK, then call the endpoint when updated and be able to match new subjects without restarting viseron
This all seems to work well. However, I may need to rework how both the API and the compreface component do this. Linting is complaining about me accessing protected members (from self._vis). Which I'm definitely doing. If you are willing to email me at<EMAIL_ADDRESS>I'd like to ask some advice on how best to refactor this so I can do it appropriately.

This can now be closed. Addressed with PR: 845


Preview window in VS Code is missing images when previewing markdown

Product and Version [VS/VSCode]: VS Code 1.20.0-insiders
OS Version [macOS/Windows]: macOS and Windows
Live Share Extension Version: 0.2.58
Target Platform or Language [e.g. Node.js]: Markdown
Steps to reproduce:

Owner: Clone https://github.com/MicrosoftDocs/live-share
Owner: Open README.md
Owner: Share
Participant: Join from VS Code - README.md should appear
Participant: Click on the preview split pane icon in the upper right corner

Expected: Markdown preview for README.md appears with images.

Actual: Markdown preview appears but is missing images and a message that says "Some content has been disabled in this document" This is true both for "bang syntax" images and inline HTML w/relative path src entries.

I'm also looking for images to be supported over liveshare.
An image link to a local file working properly on my host vscode instance shows up as a broken link (thumbnail + alt text) on my guest instance. Using a web-hosted image link works fine.
Also, plantuml diagrams supported by the markdown-preview-enhanced extension shows up as Error: write EPIPE.
To reproduce:

install markdown-preview-enhanced
preview sample markdown code

These do not work:

```puml
@startuml
start
:hoge;
stop
@enduml
```

![alt text](image_in_same_folder.jpg)

This works
![alt text](https://pbs.twimg.com/profile_images/874463635950551040/IhlhyWsq.jpg)

Product and Version [VS/VSCode]: VS Code 1.36.0
OS Version [macOS/Windows]: Windows 10 1809
Live Share Extension Version: 1.0.488
Target Platform or Language [e.g. Node.js]: Markdown


config.json cannot be load occassionally in multiprocess usecase

It seems that during http request, the file config.json will be cleaned and update. In multiprocess usecase, sometimes httpie would read an empty config file and occurs an error.
The thrown exception are:
Traceback (most recent call last):
  File "c:\python36\lib\site-packages\httpie\config.py", line 47, in load
    data = json.load(f)
  File "c:\python36\lib\json\__init__.py", line 299, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "c:\python36\lib\json\__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "c:\python36\lib\json\decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "c:\python36\lib\json\decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "c:\python36\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\python36\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Python36\Scripts\http.exe\__main__.py", line 9, in <module>
  File "c:\python36\lib\site-packages\httpie\__main__.py", line 11, in main
    sys.exit(main())
  File "c:\python36\lib\site-packages\httpie\core.py", line 193, in main
    if env.config.default_options:
  File "c:\python36\lib\site-packages\httpie\context.py", line 84, in config
    self._config.load()
  File "c:\python36\lib\site-packages\httpie\config.py", line 96, in load
    super(Config, self).load()
  File "c:\python36\lib\site-packages\httpie\config.py", line 51, in load
    (type(self).__name__, str(e), self.path)
ValueError: Invalid Config JSON: Expecting value: line 1 column 1 (char 0) [D:\Users\xxxx\AppData\Roaming\\httpie\config.json]
Traceback (most recent call last):
  File "c:\python36\lib\site-packages\httpie\config.py", line 47, in load
    data = json.load(f)
  File "c:\python36\lib\json\__init__.py", line 299, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "c:\python36\lib\json\__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "c:\python36\lib\json\decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "c:\python36\lib\json\decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "c:\python36\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\python36\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Python36\Scripts\http.exe\__main__.py", line 9, in <module>
  File "c:\python36\lib\site-packages\httpie\__main__.py", line 11, in main
    sys.exit(main())
  File "c:\python36\lib\site-packages\httpie\core.py", line 193, in main
    if env.config.default_options:
  File "c:\python36\lib\site-packages\httpie\context.py", line 84, in config
    self._config.load()
  File "c:\python36\lib\site-packages\httpie\config.py", line 96, in load
    super(Config, self).load()
  File "c:\python36\lib\site-packages\httpie\config.py", line 51, in load
    (type(self).__name__, str(e), self.path)
ValueError: Invalid Config JSON: Expecting value: line 1 column 1 (char 0) [D:\Users\xxxx\AppData\Roaming\\httpie\config.json]

This is a duplicate of #788

Fixed in master, thanks!


How to disable client caching completely?

According to the In Memory Cache docs for commerce-sdk it should be possible to disable caching by setting the cacheManager setting to null. This does not work as described because the cacheManager config property type is ICacheManager | undefined.
Also, the BaseClient class sets the default cacheManager config property to use quick-lru, so I don't see of any way for disabling the cache manager functionality completely through the config values. https://github.com/SalesforceCommerceCloud/commerce-sdk-core/blob/master/src/base/client.ts#L65
Our use case does not require caching so I'd like to disable it completely.

Closing this issue. Please feel free to reopen along with requested info if you continue to experience this problem.


Memory usage reduction

Description
#6044 increased the memory usage of the compiler by an estimated 400%. This PR alleviates some of that increased footprint.
is_external in the Module struct needs to be updated to allow for compilation of dependencies (a module is not external while it is being compiled, but is external when one of its dependents is being compiled). #6044 introduced an extra level of cloning, which in this PR is changed by making Module mutable and destructively updating is_external.
Checklist

[x] I have linked to any relevant issues.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have updated the documentation where relevant (API docs, the reference, and the Sway book).

[ ] If my change requires substantial documentation changes, I have requested support from the DevRel team

[x] I have added tests that prove my fix is effective or that my feature works.
[x] I have added (or requested a maintainer to add) the necessary Breaking* or New Feature labels where relevant.
[x] I have done my best to ensure that my PR adheres to the Fuel Labs Code Review Standards.
[x] I have requested a review from the relevant team or maintainers.

This brought #6219 mem usage to ~1.7gb from ~20gb


Not compatible with M1 Mac with bumpy ver 1.21 ??

HI, I tried to run UMAP on M1 Mac on numpy version 1.21,  however one of the dependencies (numba requires 1.20 version of numpy)
any suggestions ?

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
/var/folders/tm/9klk1nzn2lz690wvdfb85s_00000gn/T/ipykernel_31208/469094592.py in <module>
----> 1 import umap
      2 reducer = umap.UMAP(n_neighbors=20,n_components=3,verbose=True)
      3 X_transformed =reducer.fit_transform(X)
      4 X_transformed.shape

~/miniforge3/envs/tensorflow/lib/python3.9/site-packages/umap/__init__.py in <module>
      1 from warnings import warn, catch_warnings, simplefilter
----> 2 from .umap_ import UMAP
      3
      4 try:
      5     with catch_warnings():

~/miniforge3/envs/tensorflow/lib/python3.9/site-packages/umap/umap_.py in <module>
     26 from scipy.sparse import tril as sparse_tril, triu as sparse_triu
     27 import scipy.sparse.csgraph
---> 28 import numba
     29
     30 import umap.distances as dist

~/miniforge3/envs/tensorflow/lib/python3.9/site-packages/numba/__init__.py in <module>
    196
    197 _ensure_llvm()
--> 198 _ensure_critical_deps()
    199
    200 # we know llvmlite is working as the above tests passed, import it now as SVML

~/miniforge3/envs/tensorflow/lib/python3.9/site-packages/numba/__init__.py in _ensure_critical_deps()
    136         raise ImportError("Numba needs NumPy 1.17 or greater")
    137     elif numpy_version > (1, 20):
--> 138         raise ImportError("Numba needs NumPy 1.20 or less")
    139
    140     try:

ImportError: Numba needs NumPy 1.20 or less

I'm running into this on an Intel Mac as well. Same exact error:
    136         raise ImportError("Numba needs NumPy 1.17 or greater")
    137     elif numpy_version > (1, 20):
--> 138         raise ImportError("Numba needs NumPy 1.20 or less")
    139
    140     try:

ImportError: Numba needs NumPy 1.20 or less

Running pip freeze produces:
cycler==0.10.0
gensim==4.1.2
joblib==1.1.0
kiwisolver==1.3.2
llvmlite==0.37.0
matplotlib==3.4.3
numba==0.54.1
numpy==1.20.0 <----- NOTE
pandas==1.3.4
Pillow==8.4.0
pynndescent==0.5.5
pyparsing==3.0.0
python-dateutil==2.8.2
pytz==2021.3
scikit-learn==1.0
scipy==1.7.1
seaborn==0.11.2
six==1.16.0
sklearn==0.0
smart-open==5.2.1
threadpoolctl==3.0.0
tqdm==4.62.3
umap-learn==0.5.1

Additional info:
I can only get to this point if I attempt to import umap like so:
from umap.umap_ import UMAP

If I import with:
import umap
red = umap.UMAP()

I get the following error:
AttributeError: module 'umap' has no attribute 'UMAP'


Add riff eslint override configuration for components/dashboard

Summary
Up to this point, I had been adding some eslint style overrides at the top of each source file.
Instead of doing this, there is now a .eslint.json file in components/dashboard that will apply to all source files in that tree.
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] Ran make check-style to check for style errors (required for all pull requests)
[ ] Ran make test to ensure unit and component tests passed
[ ] Added or updated unit tests (required for all new features)
[ ] Needs to be implemented in mobile (link to PR or User Story)
[ ] Has server changes (please link)
[ ] Has redux changes (please link)
[ ] Has UI changes
[ ] Includes text changes and localization file (.../i18n/en.json) updates
[ ] Touches critical sections of the codebase (auth, posting, etc.)

release 1.0.0-dev.29