Support namespace-like variables?

Search Terms
Namespace, not included in the documentation
Problem
The code
type Email = (value: string) => boolean;

export type Validators = {
  email: Email;
};

produces the warning
Warning: Email, defined at src/index.ts:12, is referenced by Validators.__type.email but not included in the documentation.

and the documentation

Suggested Solution
In my case, I want TypeDoc to treat the above code as
export type Validators = {
  email: (value: string) => boolean;
};

When TypeDoc sees that the unexported type Email is going to be referenced in the documentation, it could "replace" Email with (value: string) => boolean.  But what if it was an accident that the Email type wasn't exported, and now we've erased it from the documentation? 🤔
Additional Context
This was discussed on #1739, see my comment and those that follow.
I have no idea if this is a common problem. This seems like a low priority issue to me.

I didn't realize this the first time I read this since "namespace-like variables" meant something different to me, this is something you could do with a plugin but is not something I want to include in TypeDoc proper. #1823.


Use a testing library to make my life easier

Something like Jest would work I guess.

Fixed in 2271b11.


Static graph-based restore failed when building .net 8 from source

[x] This issue is blocking building from source for Solus
[x] This issue is causing unreasonable pain

I'm attempting to build from source on Solus. Here's my solus build file. It's probably easier to follow since it includes all patches.
I first tried simply ./prep.sh and ./build.sh --online, but after that didn't work I tried using some other distros as examples, for e.g. Arch.
Here are the steps I have taken so far:

simply running prep.sh and build.sh --online with no CFLAGS, CXXFLAGS, etc. set
patch in solus rids (patches here)
Include all the changes I saw in the arch source repo. You can see them in the build file
build using the build section of this yml file, using the environment variables set in the same file

I get an "Exited with code 1" error, with the error above stating the following:
/home/nelson/Development/dotnet/.dotnet/sdk/8.0.100-rc.2.23502.2/NuGet.RestoreEx.targets(19,5): error : Static graph-based restore failed with exit code '139' but did not log an error. Please file an issue at https://github.com/NuGet/Home. [/home/nelson/Development/dotnet/src/runtime/artifacts/source-build/self/src/Build.proj]
      0 Warning(s)
      1 Error(s)

Attached is the ./artifacts/log and ./artifacts/logs contents.
logs.zip
The logs in ./src/arcade/artifacts/log are here

Hi, thank you for reaching out with the issue you are having.
I'm not quite sure about why you are encountering the said issue, maybe @mthalman would have a better idea

I think @ViktorHofer may have seen this in the PoC work.

          0>MSBUILD : error MSB4166: Child node "14" exited prematurely. Shutting down. Diagnostic information may be found in files in "/tmp/MSBuildTempbuild/" and will be named MSBuild_*.failure.txt. This location can be changed by setting the MSBUILDDEBUGPATH environment variable to a different directory.

I don't think this is connected to static graph restore. This sounds more like an msbuild node issue. cc @rainersigwald
@nelsontkq do you have the files in tmp/MSBuildTempbuild still available to share? Those would provide valuable information for the msbuild team to diagnose the node creash.

I believe both "/tmp/MSBuildTempbuild*" and the folder created at MSBUILDDEBUGPATH were actually empty. I'll rerun it again and check

I run this script on a clean environment and get the same issue.
# Microsoft.DotNet.Arcade.Sdk.CalculateAssemblyAndFileVersions fails to set VersionPrefix if the version has a minor #
sudo sed -i 's/4.5/45/g' /etc/os-release
export VERBOSE=1
export DOTNET_CLI_TELEMETRY_OPTOUT=1
unset CFLAGS CXXFLAGS LDFLAGS
python3 -m venv $HOME/.pyenv
source $HOME/.pyenv/bin/activate
python -c "import sys; sys.exit(1 if sys.version_info[0] == 2 else 0)"
./prep.sh
env
./build.sh --online -- /v:d

MSBUILD : error MSB4166: Child node "14" exited prematurely. Shutting down. Diagnostic information may be found in files in "/tmp/MSBuildTempbuild/" and will be named MSBuild_*.failure.txt. This location can be changed by setting the MSBUILDDEBUGPATH environment variable to a different directory.
I couldn't find the logs anywhere.
From the build env:
build@nelson-pc ~ $ ls -la /tmp/MSBuildTempbuild
total 16
drwx------ 2 build build  4096 Jan  4 20:33 .
drwxrwxrwt 1 root  root  12288 Jan  4 20:33 ..

build@nelson-pc / $ find -name MSBuild_*.failure.txt 2>/dev/null
> not found
build@nelson-pc ~ $ ls -la ~/YPKG/root/dotnet-sdk/build/dotnet.git/artifacts/msbuild-debug
total 8
drwxr-xr-x  2 build build 4096 Jan  4 20:33 .
drwxr-xr-x 12 build build 4096 Jan  4 20:33 ..

Console logs for this build:
build2.log
Here's the full artifacts folder
artifacts.zip

I've re-ran the build a few more times and each time it's now complaining about a VersionPrefix. Perhaps the msbuild error is intermittent.
    [SourceBuiltSdkResolver] Looking for SDK Microsoft.NET.SDK.WorkloadAutoImportPropsLocator. Detected config(s) in env: 'ARCADE' for 'Microsoft.DotNet.Arcade.Sdk/8.0.0-beta.23516.4' at '/home/build/YPKG/root/dotnet-sdk/build/dotnet.git/artifacts/source-built-sdks/Microsoft.DotNet.Arcade.Sdk/'
    [SourceBuiltSdkResolver] Looking for SDK Microsoft.NET.SDK.WorkloadManifestTargetsLocator. Detected config(s) in env: 'ARCADE' for 'Microsoft.DotNet.Arcade.Sdk/8.0.0-beta.23516.4' at '/home/build/YPKG/root/dotnet-sdk/build/dotnet.git/artifacts/source-built-sdks/Microsoft.DotNet.Arcade.Sdk/'
      Determining projects to restore...
      Restored /home/build/YPKG/root/dotnet-sdk/build/dotnet.git/src/xliff-tasks/artifacts/source-build/self/src/src/XliffTasks/XliffTasks.csproj (in 50 ms).
    /home/build/YPKG/root/dotnet-sdk/build/dotnet.git/artifacts/source-built-sdks/Microsoft.DotNet.Arcade.Sdk/tools/Version.targets(16,5): error MSB4044: The "Microsoft.DotNet.Arcade.Sdk.CalculateAssemblyAndFileVersions" task was not given a value for the required parameter "VersionPrefix". [/home/build/YPKG/root/dotnet-sdk/build/dotnet.git/src/xliff-tasks/artifacts/source-build/self/src/src/XliffTasks/XliffTasks.csproj::TargetFramework=net8.0]
    ##vso[task.logissue type=error;sourcepath=/home/build/YPKG/root/dotnet-sdk/build/dotnet.git/artifacts/source-built-sdks/Microsoft.DotNet.Arcade.Sdk/tools/Version.targets;linenumber=16;columnnumber=5;code=MSB4044;](NETCORE_ENGINEERING_TELEMETRY=Build) The "Microsoft.DotNet.Arcade.Sdk.CalculateAssemblyAndFileVersions" task was not given a value for the required parameter "VersionPrefix".

    Build FAILED.

    /home/build/YPKG/root/dotnet-sdk/build/dotnet.git/artifacts/source-built-sdks/Microsoft.DotNet.Arcade.Sdk/tools/Version.targets(16,5): error MSB4044: The "Microsoft.DotNet.Arcade.Sdk.CalculateAssemblyAndFileVersions" task was not given a value for the required parameter "VersionPrefix". [/home/build/YPKG/root/dotnet-sdk/build/dotnet.git/src/xliff-tasks/artifacts/source-build/self/src/src/XliffTasks/XliffTasks.csproj::TargetFramework=net8.0]
        0 Warning(s)
        1 Error(s)

I was able to get past the VersionPrefix issue! I think there was a missing dependency which I 'solved' by adding every mentioned package in all of your build docker containers. I've got it to completely build once
I am experiencing a very flaky build process though.
I get the Static graph-based restore failed when building .net 8 from source error on the first build, then one or two packages fail with MSBUILD : error MSB4166: Child node "8" exited prematurely. Shutting down. Diagnostic information may be found in files in "/tmp/MSBuildTempnelson/" and will be named MSBuild_*.failure.txt. This location can be changed by setting the MSBUILDDEBUGPATH environment variable to a different directory.
Though there's no msbuild logs anywhere.

Unfortunately I have no idea how to resolve this flakiness without more logs but it sounds like there aren't any other logs.

I'm going to close this issue since the VersionPrefix issue is dealt with. If anyone is googling and wondering what caused the VersionPrefix issue: Our build env set the version environment variable which broke the build of one or more packages


[build] Permissions not being removed during build

Build/Submit details page URL
No response
Summary
I have the feeling that something is broken in eas build (or, as usual, I'm missing something)
Take a look at this issue: https://github.com/expo/expo/issues/14525
We are patching some changes in node_modules in the post-install, in order to remove some Permissions that are unnecessary added, but then, in the build (made with eas build), the permissions are still there. Like... eas build is ignoring the changes we are applying.
I still have some doubts if we are doing everything well there. For example:

We are applying the changes in the build folder instead of in the src folder. Not sure if this is correct. Not sure if eas build builds everything from scratch, or if it uses the build folder.
I'm not sure if the correct hook to apply the patch is post-install or if it's other.
We are not clearing cache (in case it is necessary)

Still, feels like we are missing something, or it's a possible bug in eas build
Managed or bare?
Managed
Environment
expo-env-info 1.0.2 environment info:
System:
OS: macOS 11.6.1
Shell: 5.8 - /bin/zsh
Binaries:
Node: 14.18.2 - ~/.nvm/versions/node/v14.18.2/bin/node
Yarn: 1.22.18 - ~/.yarn/bin/yarn
npm: 8.3.2 - ~/.nvm/versions/node/v14.18.2/bin/npm
Watchman: 4.9.0 - /usr/local/bin/watchman
SDKs:
iOS SDK:
Platforms: iOS 14.5, DriverKit 20.4, macOS 11.3, tvOS 14.5, watchOS 7.4
Android SDK:
API Levels: 27, 29, 30
Build Tools: 23.0.1, 23.0.3, 25.0.3, 26.0.1, 27.0.3, 28.0.3, 29.0.2
System Images: android-29 | Google APIs Intel x86 Atom, android-30 | Google Play Intel x86 Atom
IDEs:
Android Studio: 2020.3 AI-203.7717.56.2031.7784292
Xcode: 12.5.1/12E507 - /usr/bin/xcodebuild
npmPackages:
expo: ^44.0.0 => 44.0.6
react: 17.0.1 => 17.0.1
react-native: 0.64.3 => 0.64.3
npmGlobalPackages:
eas-cli: 0.51.0
expo-cli: 5.3.2
Expo Workflow: managed
Error output
No error
Reproducible demo or steps to reproduce from a blank project
Here it should be enough with adding expo-image-picker to the dependencies (I'm using "expo-image-picker": "~12.0.1").
import * as ImagePicker from 'expo-image-picker';

Then follow this instructions: https://github.com/expo/expo/issues/14525#issuecomment-986719649 in order to apply the patch to the build files, and then eas build

patch-package definitely works on eas, if you have some problems with getting that working you can debug it with eas build:inspect --stage=pre-build command and inspect the resulting build directory if everything was applied correctly
As for the instruction, I think use of eas-build-post-install is not correct, it should be done in postinstall(there is no such thing as post-install).

@wkozyra95 what do you mean with there is no post-install? I can see it in the documentation

There is eas-build-post-install there is no post-install, either way, eas-build-post-install is the wrong place to apply that patch

Thanks! Do you have any idea which is the correct place to apply the patch?

it should be done in postinstall

https://docs.npmjs.com/cli/v8/using-npm/scripts

@wkozyra95 confirmed.

I ran eas build:inspect --stage=pre-build -p android -o ./prebuild --verbose, and I could saw the permissions were present in android/app/src/main/AndroidManifest.xml
I changed eas-build-post-install with postinstall
I ran eas build:inspect --stage=pre-build -p android -o ./prebuild --verbose, and now permissions are not there anymore.

Feel free to close this issue if you consider there is no documentation to change in EAS-CLI.
I will push the new build to Google Play and see if the unnecessary permission is gone, and then update the original bug in /expo/expo


Widen logging version constraints to include 2.0 (matching WinRM core gem)

Running into gem conflicts when winrm-fs is used with other gems accepting logging 2.0.

:shipit:


Monitoring relay trigger?

Is there any way to monitor or check wether a relay was triggered through the API, I'm only able to find ways to trigger it.

There was not a way to do this in version 0.17 of their API, so this wrapper doesn't support it.
They may have added a way in the newly updated API, but I haven't had time to investigate this yet. I will likely have time in the coming weeks, and will release an update as soon as possible.

Page 10: Energize the door opener/ alarm output relay of the device.
Page 16: Get relay configuration data only
Page 24: EVENT MONITORING (UDP BROADCASTS)
Page 24 sounds promising however I am unable to understand how it could be used to monitor a relay trigger.
https://www.doorbird.com/downloads/api_lan.pdf?rev=0.21

I already have the old version of event monitoring implemented, but it doesn't look like it adds support for this :(

Was the previous version based on the UDP broadcasts?

No, just HTTP requests from the DoorBird when events fired.
Am 24.05.2018 14:09 schrieb "Nasir Khalid"<EMAIL_ADDRESS>Was the previous version based on the UDP broadcasts?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/Klikini/doorbirdpy/issues/6#issuecomment-391826414,
or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABnZCOO0jFBUzJ-3geI0A0jqZF4Eu5sxks5t1wVqgaJpZM4UKw2R
.


backend/drm: Simplify object matching code

We originally used match_obj on planes, but this was largely unnecessary. Instead, this assigns planes statically at startup.
See https://github.com/swaywm/wlroots/issues/1723#issuecomment-504247814
It'll help if some people could test this as well (especially hotplugging), so it's not just "works for me".

Holds up in testing :+1:

Thanks!


Fix/alamofire compatibility

Closes #86
There was nothing to do except return valid header fields by default (application/json)
AFNetowrking/Alamofire/etc will just work but the user will have to set a proper NSURLSessionConfiguration. So I made KakapoServer public.

👍

[Current coverage][cc-pull] is 98.46%
[][cc-pull]

No coverage report found for master at 03c548c.

Powered by Codecov. Last updated by [03c548c...6262fc0][cc-compare]
[cc-base-branch]: https://codecov.io/gh/devlucky/Kakapo/branch/master?src=pr
[cc-compare]: https://codecov.io/gh/devlucky/Kakapo/compare/03c548cabf7901faffbe3944f97719e9738219ad...6262fc0f7a8220fc59de82ecedc91854bfad6cd9?src=pr
[cc-pull]: https://codecov.io/gh/devlucky/Kakapo/pull/89?src=pr


CB-12127 - (ios) Add buildFlag support in build.json

Platforms affected
iOS
What does this PR do?
Document the "buildFlag" option in build.json
What testing has been done on this change?
Manual testing.
Checklist

[X] Reported an issue in the JIRA database
[X] Commit message follows the format: "CB-3232: (android) Fix bug with resolving file paths", where CB-xxxx is the JIRA ID & "android" is the platform affected.

Pull this in for cordova-ios 4.3.1 release.


Crash under Mac OS X

Mac OS X, using python from homebrew and nuitka version <IP_ADDRESS>.
$ python --version
Python 3.7.3
The problem in seems to be in MainProgram.c in the argument conversion. The minimal way to reproduce the crash is to compile the following program with nuitka:
import sys
print(len(sys.argv))

$ ./t.bin
Segmentation fault: 11
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x603040)
  * frame #0: 0x00007fff5bd206da libsystem_c.dylib`wcslen + 8
    frame #1: 0x000000010017075d Python`PyUnicode_FromWideChar + 72
    frame #2: 0x00000001001ed354 Python`PySys_SetArgvEx + 108
    frame #3: 0x0000000100004585 t.bin`main + 469
    frame #4: 0x00007fff5bc78ed9 libdyld.dylib`start + 1

Changing MainProgram.c from (line 156 in convertCommandLineParameters):
#ifdef __APPLE__
        argv_copy[i] = _Py_DecodeUTF8_surrogateescape(argv[i], strlen(argv[i]));
#elif PYTHON_VERSION < 350
        argv_copy[i] = _Py_char2wchar(argv[i], NULL);
#else
        argv_copy[i] = Py_DecodeLocale(argv[i], NULL);
#endif

to
#if PYTHON_VERSION >= 350
		argv_copy[i] = Py_DecodeLocale(argv[i], NULL);
#elif defined(__APPLE__) && PYTHON_VERSION >= 320
		argv_copy[i] = _Py_DecodeUTF8_surrogateescape(argv[i], strlen(argv[i]));
#else
		argv_copy[i] = _Py_char2wchar(argv[i], NULL);
#endif

fixes the issue for me and the program runs without errors. The reference from modifications was found here: https://bitbucket.org/mpi4py/mpi4py/issues/21/new-release-of-any-kind

Thanks a lot, want to make it a PR? Otherwise, just your name+email would be fine then I would attribute it to you.
Yours,
Kay

@kayhayen,
I'll open a PR later today with the relevant changes.
Thanks,
Mikko

The PR is here: https://github.com/Nuitka/Nuitka/pull/391
@kayhayen Thanks for your for your work on such a great tool.

Released as 0.6.4


[wip] make scaling configurable by gemm-argument

Summary:
My brain hurts from so many long identifiers...
Test Plan:
Reviewers:
Subscribers:
Tasks:
Tags:

Stack from ghstack (oldest at bottom):

(to be filled)


Allocate zeros

This uses calloc under the hood to give the OS a chance to optimize if it tags pages as zeroed.
Please review and let me know how you'd like this expressed/integrated and if you see any issues.
Thanks!

The CI failure looks like it has something to do with Travis/ARM/qemu/docker.

Thanks for the careful patch!
I think the additional overhead of calloc is probably low enough that we can just use calloc all the time, and leave the interface unchanged.

Ok, I've updated the PR. Now there is a single squashed commit which notes the change of default in the docstring and internal comments. Also, I've completely removed the use of caml_stat_alloc and caml_stat_free from the memory stubs. I converted ctypes_copy_bytes to use OCaml binding macros because it wasn't and put a check for ENOMEM in its body because the checked-malloc in caml_stat_alloc was removed.

Thanks: that looks good.  There's another call to Stubs.allocate that needs updating; it's currently causing the build to fail:
File "src/ctypes/ctypes_memory.ml", line 27, characters 65-72:
Error: This expression has type int -> Stubs.managed_buffer
       but an expression was expected of type
         Stubs.managed_buffer = Ctypes_memory_stubs.managed_buffer
Makefile.rules:92: recipe for target '_build/src/ctypes/ctypes_memory.cmo' failed
make: *** [_build/src/ctypes/ctypes_memory.cmo] Error 2

Oops, sorry about that. I've fixed the uses of allocate and an issue with ctypes_copy_bytes's missing CAMLparam. Let me know if you're happy with this and I can squash the commits.

Let me know if you're happy with this and I can squash the commits.

It looks good to me.  I'll merge once it's squashed.

Squished.


Menubar Icon Disappears

Thank you for the template. Why does the menubar icon disappear while the mouse button is held?

Glad to know that you liked this project. I don't know how but I think the reason behind that is the way statusItem is being initialized. If you initialize the statusItem in StatusBarController.swift using NSStatusBar.system, you shouldn't be having this problem. I hope this helps.

@pqhf5kd Did you try what I suggested? Else I am going to close this issue.

@AnaghSharma sorry, I haven't. My project stopped running for some reason and now even when I run a fresh copy of the example, I get an EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0) error.

@pqhf5kd This kind of error is kinda hard to diagnose I think. If you are able to test the above solution, let me know if it works or reopen the issue otherwise, since I am closing it for now. Thanks.


deleted unnecessary codes

Deleted some lines of code used to ascertain values expressed in the interpreting the viz.

Thanks, and sorry once again.


accessability issue

Hello
im trying to set up the system, but have hit a wall when it comes to cloning this:
git clone https://github.com/HARPLab/UnrealEngine -b DReyeVR-0.9.13 --depth 1
In another thread (#83), a response from @GustavoSilvera  said something about needing to request access before cloning.
Could this be the reason?
I have done everything else, linking to epic games, downloaded the requered material.
if this is so, may i get access?

(For clarity, you'll need to instead clone from https://github.com/CarlaUnreal/UnrealEngine and follow the git checkout instructions I posted to get a compatible UE4 version.


Tensorizer Support for Large Models( 70B+) that dont fit into a single GPU

I'm currently evaluating Tensorizer for handling large models, specifically models with parameters as larger than  70B that cannot be fit into a single GPU.
I have a few questions and concerns regarding Tensorizer's support for such large models and its handling of .tensors files.

Large Model Support:  Can Tensorizer effectively handle large models with parameters in the range of 70B and above wiht multiple distributed GPUs ? I havent seen any benchmarks for streaming large models that cannot be fit into a single A100 GPU .  Specifically, can Tensorizer scale effectively to accommodate large models across distributed environments, such as distributed training setups with multiple GPUs or nodes ?

.tensors File Format: Could you provide some insights into the .tensors file format used by Tensorizer? How does Tensorizer organize and store tensors within .tensors files, especially for large models? Does it employ any sharding or distribution mechanisms for managing large tensors?

I appreciate any insights or documentation you can provide regarding these questions. Understanding Tensorizer's capabilities and limitations will help us make informed decisions for our use case.

Absolutely! One strategy you can use is to partition the model into per-rank shards and serialize each shard into its own respective .tensors file. Here's an example of what checkpointing with tensorizer could look like for distributed training, simplified from the coreweave/megatron fork

model = unwrap_model(model)
for i in range(len(model)):
    mpu.set_virtual_pipeline_model_parallel_rank(i)
    shard_serializer = TensorSerializer(
        f"{checkpoint_name}_shard{i}.tensors"
    )
    shard_serializer.write_module(model[i])
    shard_serializer.close()

And the corresponding deserialization, again simplified from the linked code:
for i in range(len(model)):
        mpu.set_virtual_pipeline_model_parallel_rank(i)
        shard_deserializer = TensorDeserializer(
            f"{checkpoint_name}_shard{i}.tensors",
            device=torch.cuda.current_device(),
            plaid_mode=True
        )
        shard_deserializer.load_into_module(model[i])
        shard_deserializer.close()

Alternatively, if you have a checkpoint already that you want to load across multiple ranks, you can provide a filter function to indicate which tensors should or should not be streamed into any given rank.
for i in range(len(model)):
    module_names = modules_to_load_into_rank[i]
    # Should evaluate to True when the provided `key`
    # matches the name of a tensor we want on the device
    def filter_func(key: str) -> Callable[[str], bool]:
        return key in module_names

    mpu.set_virtual_pipeline_model_parallel_rank(i)
    shard_deserializer = TensorDeserializer(
        f"{checkpoint_name}_shard{i}.tensors",
        device=torch.cuda.current_device(),
        plaid_mode=True
    )
    shard_deserializer.load_into_module(model[i], filter_func)
    shard_deserializer.close()

.tensors files pack tensors into contiguous memory on disk. The chunk for a given tensor starts with a header containing metadata, and then is followed by the tensor's data. The data format doesn't do sharding or distribution on its own, but as demonstrated above is compatible with user-specified sharding. Alternatively, Coreweave storage supports both CephFS and Vast distributed file systems. We are currently undergoing benchmarking exercises whose results we will hopefully share soon. Based on our preliminary observations, reading and writing shards distributed across multiple .tensors files is likely to be faster than one big file when using one of those distributed file systems.

Did that answer your questions?

could you maybe share some code to clarify how you're deserializing? just to reiterate: I provided three chunks of code above, and one of them is a demonstration of what not to do.
also, could you elaborate a bit more on your environment, e.g. are you downloading from S3 within AWS?

Thank you @dmarx  for your prompt response.
For deserialization, I'm employing the same code structure as provided in the deserialize example. Initially, I write the .tensors file for the 6B model, and subsequently, I utilize the boto3 TensorDeserializer to stream it back onto the GPU.
Environment Details:
Our environment is internal, not AWS-based. However, during experimentation, I observed that while using the MinIO command-line utility mc to download the .tensors file,  downloading to an NVMe(not disk) I achieved speeds of at least 6GB/s, indicating that our network can indeed support higher speeds.
Despite the network's capability to achieve high speeds, the TensorDeserializer only achieves transfer speeds in the range of 500 MB/s during deserialization. Given the confirmed high download speeds, I'm curious about the factors limiting the deserialization speeds with Tensorizer.
I hope this provides the necessary clarity on the deserialization process and environment setup. Any insights for speeding up or recommendations regarding the observed deserialization speeds would be greatly appreciated.

What version of tensorizer are you using? Try updating to the latest version, 2.8.0. One of the most important performance modes in tensorizer is plaid_mode which pipelines the loading and processing of tensors. This has historically been a feature that was disabled by default, and will now be turned on by default whenever a cuda device is detected.

I am on 2.7.2, but I tried both and without plaid_mode = True in my experiments the model was streamed to GPU in almost same time

This is really unusual. I'm thankful you've brought your case to our attention and for your patience and cooperation as we try to figure out what could be going on here.
To update the library, try pip install --upgrade tensorizer or pip install tensorizer==2.8.0.
What kind of performance do you experience using tensorizer to deserialize models from local paths?

Yea sure, Thanks so much for taking a look into this.
I executed the helper utils from tensorizer
total_bytes_str = convert_bytes(deserializer.total_tensor_bytes)
duration = end - start
per_second = convert_bytes(deserializer.total_tensor_bytes / duration)
after_mem = get_mem_usage()
deserializer.close()
print(f"Deserialized {total_bytes_str} in {end - start:0.2f}s, {per_second}/s")
print(f"Memory usage before: {before_mem}")
print(f"Memory usage after: {after_mem}")

and got the following output
Deserialized 12.2 GB in 26.35s, 463.8 MB/s
Memory usage before: CPU: (maxrss: 14,341MiB F: 486,836MiB) GPU: (U: 309MiB F: 32,191MiB T: 32,501MiB) TORCH: (R: 0MiB/0MiB, A: 0MiB/0MiB)
Memory usage after: CPU: (maxrss: 26,839MiB F: 486,739MiB) GPU: (U: 23,749MiB F: 8,751MiB T: 32,501MiB) TORCH: (R: 23,348MiB/32,014MiB, A: 11,661MiB/23,315MiB)

I have 12 core CPU and for GPU I am using V100 but I didnt get much difference when using  A100 either. Both cases the speed was  ~ 500 MB/s  . I am having some issues internally installing latest version 2.8.0. I'll ask around and see whats the issue . But 2.7.2 should have given good performance as well correct ( with plaid_mode= True) ?
Local mode
yea local mode was very fast in my experiment from disk it reached speed upto 3GB/s

I took a look at the ReadStreamFile(in local mode) vs CuRLStreamFile(used in s3) . The former uses Redis backed and later uses https .  Might explain why we are seeing differences

Taking a look at the ReadStreamFile(in local mode) vs CuRLStreamFile(used in s3) . The former uses Redis backed and later uses https . Might explain why we are seeing differences

Taking a look at the RedisStreamFile(in local mode) vs CuRLStreamFile(used in s3) . The former uses Redis backed and later uses https . Might explain why we are seeing differences

Taking a look at the RedisStreamFile(in local mode) vs CuRLStreamFile(used in s3) . The former uses Redis backed and later uses https where it requires re-opening S3 connection, incurring latency . Might explain why we are seeing differences

Did you ever figure out what was going on here? The team has incorporated several performance improvements over the last month: although I wasn't able to reproduce your issues, it's possible that they were addressed in one of the updates. If you want to experiment with the pre-release package: pip install --pre 'tensorizer~=2.9.0'

Thanks @dmarx  I will test the latest one and get back


highlighting of search terms

Originally by annetteh on 2011-02-24
A user suggested to implement spires' highlighting (unbolding) of found search terms

Issue moved to inveniosoftware/invenio-search-js #75 via ZenHub


fix(tilegroup): small fix for styles to work with carbon

Summary

Needed to override the default carbon styles with our styled component styles so needed the extra layer of specificity
fix(tilecatalog): need to generate selection event for initial selection
fix(wizardinline): should take up full available width for content

Acceptance Test (how to verify the PR)

It's pretty hard to test without consuming this in another project unfortunately.

:tada: This PR is included in version 1.37.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.37.2 :tada:
The release is available on npm package (@release-1.x dist-tag)
Your semantic-release bot :package::rocket:


Deploy to cloud and implement search

Deploy to cloud and implement search
I created a CI/CD pipeline and deployed the app to Google Cloud Platform Kubernetes Engine. There are two builds available, at mbtaccess.org and latest.mbtaccess.org. The two builds are currently the same, but latest.mbtaccess.org will update with each push to GitHub, serving as a canary build.
Deployment

The app is deployed with Google Cloud Platform (GCP).
Cloud Build, available in the GCP console, enables continuous integration/continuous deployment (CI/CD) pipelines. I created cloudbuild.yml files, and set up the build triggers from within GCP to read the files. Cloud Build builds and deploys a Docker container from the GitHub repository each time a commit is pushed. There is also a Cloud Build GitHub app, but it's only in alpha, and doesn't offer enough configuration options.
Containers built with Cloud Build are stored in the GCP Container Registry (similar to Docker Cloud/Docker Store). We have an organization account on Docker Cloud, but we're not currently using it for builds.
Google Cloud Platform has three options for Python apps. We use Kubernetes Engine. Kubernetes (Greek for 'helmsman') is an open-source container management system.

The Docker container for our app is considered a pod.
The pod is deployed as a workload on Kubernetes Engine. Horizontal pod autoscaling adjusts the number of pod replicas based on traffic.
There has been a great deal of discussion on how to trigger Kubernetes deployment updates, on Stack Overflow here, and through GitHub issues here, here, and here.

The deployment needs to be configured for rolling updates and imagePullPolicy: Always.
One hack is to apply an annotation to the deployment config file with kubectl patch, which will trigger an update of the deployment. Annotations are recommended over labels. This method allows re-deployments even if the container or build SHA is the same. I included the annotation patch in our cloudbuild.yml files. The suggested annotation is a timestamp, but I couldn't get Cloud Build to calculate it properly, so I used the $REVISION_ID default variable substitution.

Networking
Cloudflare

I own the domain name mbtaccess.org through my Hover account.

Hover is configured to direct mbtaccess.org web traffic to Cloudflare nameservers for security. This is particularly important for SSL.

The entire process of implementing SSL on GCP is inordinately difficult. Normally, when adding SSL to a domain running on an Apache or Nginx server, you would install a Let's Encrypt certificate with Certbot or the provider's built-in support. For example, DigitalOcean has built-in support for Let's Encrypt and helpful documentation for implementing HTTPS in other ways. Google Cloud Platform does not.
Kubernetes adds further security challenges.

There are some security features built in, but the Kubernetes SSL certificate management system, cert-manager, is not yet stable. This has been discussed on the Google Cloud Platform podcast and the Kubernetes podcast.

Cloudflare is a simple, stable option that also includes CDN services, minification of JS/CSS, and other helpful features.
I set up a Cloudflare account for our organization. In my Hover dashboard, I pointed mbtaccess.org to Cloudflare's nameservers.

Traffic between Cloudflare and GCP is secured with a separate TLS origin certificate.

There are four steps needed to set up an origin certificate:

Create a private key
Create a certificate signing request (CSR)
Create the certificate
Activate the certificate by creating a GCP certificate resource

The certificate can be created by Cloudflare on the "crypto" tab of the Cloudflare dashboard. I opted to use Cloudflare's origin certificate, installing it in the ssl/ directory for our project.

The first three steps can also be accomplished within GCP.

Create a private key

The Cloud Shell for the project (in the browser, the SSH client is only in alpha) can be used to create a new directory and an openssl key. GCP installs openssl in projects by default for securing traffic to load balancers.
gwgnewengland@cloudshell:~ (mbtaccess)
mkdir ssl
cd ssl
openssl genrsa -out self.key 2048

Create a certificate signing request (CSR)

In Cloud Shell:
openssl req -new -key self.key -out self.csr

Create the certificate

The certificate can either be generated by Cloudflare using the CSR from GCP, or it can be self-generated. Self-generated certificates are less secure and not recommended.

Generating a self-signed certificate with the key and CSR:
openssl x509 -req -days 365 \
  -in self.csr \
  -signkey self.key \
  -out self.crt

A "certificate resource" tells GCP to use the certificate for the project. This needs to be done whether the certificate is from Cloudflare or is self-signed.
gwgnewengland@cloudshell:~/ssl (mbtaccess)$
gcloud compute ssl-certificates create cloudflare \
  --certificate cloudflare.crt \
  --private-key cloudflare.key

I also created an SSL policy to specify how SSL is used:
gcloud compute ssl-policies create mbtaccess-ssl-policy \
  --profile MODERN \
  --min-tls-version 1.0

Cloudflare is reportedly compatible with Google Cloud Kubernetes Engine, but I couldn't get the origin TLS certificate to work properly. It may be because Cloudflare's origin certificate authority is not publicly trusted. When SSL is set to "Full" in Cloudflare (everything is SSL), I get Error 525: SSL Handshake Failed. I had to switch it to "Flexible," which secures traffic between the web and Cloudflare, but not between Cloudflare and GCP. Cloudflare also has Kubernetes load balancing, but I'm not sure it would be helpful here.

Kubernetes networking

Cloudflare sends traffic to GCP through static IP addresses.

I reserved static IP addresses so the Kubernetes cluster can receive global web traffic.

Static IPs are created in the GCP console at VPC network -> External IP addresses. Ephemeral virtual IPs can't be connected with the rest of the web, so static IPs are reserved.
The way GCP has implemented this is confusing. There are regional and global static IP addresses. Each type of static IP address can only be used with a specific Kubernetes configuration. Regional static IP addresses only work with Kubernetes services. Global static IP addresses only work with Kubernetes ingresses, which are also listed under "Services" in GCP, even though Kubernetes distinguishes between ingresses and services.
For mbtaccess.org, I opted to use a global static IP address with a Kubernetes ingress and load balancer.
For the latest.mbtaccess.org canary builds, I just use a regional static IP with a load balancer service.

I set up DNS A records in Cloudflare to point to the static IP addresses.

GCP passes network requests to the Kubernetes cluster with ingresses and load balancers.

The ingress configures Kubernetes to receive external web traffic. Creation of an ingress has to be done manually with a YAML configuration file. The app name in the file is the deployed workload name.

The same YAML file can be used to create a load balancer, and link the load balancer with the ingress. A load balancer distributes the computing workload among CPU nodes in the cluster. There are three nodes with cluster autoscaling to six nodes based on load. I added the YAML file to the GCP project (through Cloud Shell, not the GitHub repo) in the ingress/ directory. I then applied the YAML file to create the ingress and service with
gwgnewengland@cloudshell:~ (mbtaccess)$ kubectl apply -f ingress/ingress.yml

Search

We had some discussion of search implementation in PR #45.
I implemented Google Places Autocomplete in the main search box.

Add places library API call back to query string
Implement Autocomplete in search box. This is an improvement over the Places search we had previously. Rather than having to submit a search, autocomplete displays results as the user types. Selection of a result triggers the showStops(location) function.
Geocode place ID to return coordinates.
Clear stop info from previous geolocation or search with the clearStops() function.
Call showStops(location) with place coordinates and name. For some reason, the place name returned by the Places service also has the address in it, so I used a regular expression (RegEx) to delete everything after the name: /,.*\n/g

I have been working with Elasticsearch and thinking about how to implement it. So far, I can pull a Docker image and get it running locally with Docker Compose, but am throwing errors when spinning up the container on the Kubernetes cluster. If I get it working, I'll submit a separate PR.

Other updates

I made the navbar wheelchair icon inline SVG instead of PNG to improve resolution and save space.
I made the alerts dismissable.
I attempted to remove the Google Maps API key from the source code query string, but couldn't get it to work.
I'm trying out Codacy as a method of continuous integration for code style and security. It runs Bandit (Python security), Pylint and ESLint. I have also checked out Hound from thoughtbot, but it has less features than Codacy, and uses flake8 for Python, which I'm not familiar with.
I updated the documentation to explain how I navigated the cloud labyrinth.
I think we can consider issue #36 resolved.

TODOs
Some potential next steps:

I would suggest we merge to master and tag this as version 0.2.0. I can perform the merge.

Elasticsearch

Search local stops database, maybe use to search stop results or filter stop list.

Task runner

Separate Docker container with a Python Celery script that updates MBTA data.
We might consider moving the database to the Celery container so it can just run updates within that container. We could move models.py and migrate.py into the /data directory, and add another Dockerfile there.

Implement team discussion feed on /about page

Our team discussion board has been helpful for sharing meeting notes and updates.
Unfortunately, only members of Grow with Google New England can see the discussions. It would be great to have a public feed of these discussions.
There is a GitHub Team Discussions API that we could use.

[x] I have reviewed the Guidelines for Contributing and the Code of Conduct.

I'm going to merge this PR after discussions with @AbdouSeck and @jethridge13 over email and Keybase.


alminer.download_data fails when forcing fitsonly or filename_must_include

Dear Aida,
I hope you are doing well.
We encountered an issue when trying to use alminer.download_data, in particular when we force fitslonly=True or we put constraints on the file names. I also tried different archive mirrors, and dryrun=True , all failed the same. I include below the output based on the example in the documentation (but we tried with other examples, with same result). I didn't dig further into the code.
Any idea of what could be happening? Thanks.
Cheers,
Carmen
`In [1]: import alminer
In [2]: observations = alminer.keysearch({'target_name':['G31.41'], 'proposal_id': ['2018']})
alminer.keysearch results

Number of projects = 2
Number of observations = 4
Number of unique subbands = 13
Total number of subbands = 16
1 target(s) with ALMA data = ['G31.41+0.31']
In [3]: alminer.download_data(observations, fitsonly=False, dryrun=True,
...:                       location='./data', print_urls=False)
This is a dryrun. To begin download, set dryrun=False.
Download location = ./data
Total number of Member OUSs to download = 4
Selected Member OUSs: ['uid://A001/X133d/X327', 'uid://A001/X133d/X325', 'uid://A001/X133d/X21b8', 'uid://A001/X133d/X21b4']
Number of files to download = 21
Needed disk space = 792.6 GB
In [4]: alminer.download_data(observations, fitsonly=True, dryrun=True, location='./data',
...:                       filename_must_include=['_sci', '.pbcor', 'cont', 'G31.41'],
...:                       print_urls=True)
This is a dryrun. To begin download, set dryrun=False.
Nothing to download.
Note: often only a subset of the observations (e.g. the representative window) is ingested into the archive. In such cases, you may need to download the raw dataset, reproduce the calibrated measurement set, and image the observations of interest. It is also possible to request calibrated measurement sets through a Helpdesk ticket to the European ARC (see https://almascience.eso.org/local-news/requesting-calibrated-measurement-sets-in-europe).
In [5]: alminer.download_data(observations, fitsonly=True, dryrun=True, location='./data',
...:                       filename_must_include=['_sci', '.pbcor', 'cont', 'G31.41'],
...:                       print_urls=True, archive_mirror = 'NAOJ')
This is a dryrun. To begin download, set dryrun=False.
Nothing to download.
Note: often only a subset of the observations (e.g. the representative window) is ingested into the archive. In such cases, you may need to download the raw dataset, reproduce the calibrated measurement set, and image the observations of interest. It is also possible to request calibrated measurement sets through a Helpdesk ticket to the European ARC (see https://almascience.eso.org/local-news/requesting-calibrated-measurement-sets-in-europe).
In [6]: alminer.download_data(observations, dryrun=True, location='./data',
...:                       filename_must_include=['_sci', '.pbcor', 'cont', 'G31.41'],
...:                       print_urls=True)
This is a dryrun. To begin download, set dryrun=False.
Nothing to download.
Note: often only a subset of the observations (e.g. the representative window) is ingested into the archive. In such cases, you may need to download the raw dataset, reproduce the calibrated measurement set, and image the observations of interest. It is also possible to request calibrated measurement sets through a Helpdesk ticket to the European ARC (see https://almascience.eso.org/local-news/requesting-calibrated-measurement-sets-in-europe).
In [7]:
In [7]: alminer.download_data(observations, fitsonly=True, dryrun=False, location='./data',
...:                       filename_must_include=['_sci', '.pbcor', 'cont', 'G31.41'],
...:                       print_urls=True, archive_mirror = 'NAOJ')
Starting download. Please wait...
Nothing to download.
Note: often only a subset of the observations (e.g. the representative window) is ingested into the archive. In such cases, you may need to download the raw dataset, reproduce the calibrated measurement set, and image the observations of interest. It is also possible to request calibrated measurement sets through a Helpdesk ticket to the European ARC (see https://almascience.eso.org/local-news/requesting-calibrated-measurement-sets-in-europe).
`

Dear Carmen,
Thanks for reaching out. I hope you're doing well too. This works fine for me:
observations = alminer.keysearch({'target_name':['G31.41'], 'proposal_id': ['2018']})
================================
alminer.keysearch results
================================
--------------------------------
Number of projects = 2
Number of observations = 4
Number of unique subbands = 13
Total number of subbands = 16
1 target(s) with ALMA data = ['G31.41+0.31']
--------------------------------

alminer.download_data(observations, fitsonly=True, dryrun=True, location='./data', filename_must_include=['_sci', '.pbcor', 'cont', 'G31.41'], print_urls=True)
================================
This is a dryrun. To begin download, set dryrun=False.
================================
Download location = ./data
Total number of Member OUSs to download = 2
Selected Member OUSs: ['2018.1.00252.S_uid___A001_X133d_X327_001_of_001.tar', '2018.1.00252.S_uid___A001_X133d_X325_001_of_001.tar']
Number of files to download = 4
Needed disk space = 48.9 MB
File URLs to download = https://almascience.eso.org/dataPortal/member.uid___A001_X133d_X327._G31.41p0.31__sci.spw25_27_29_31.cont.I.tt0.pbcor.fits
https://almascience.eso.org/dataPortal/member.uid___A001_X133d_X327._G31.41p0.31__sci.spw25_27_29_31.cont.I.tt1.pbcor.fits
https://almascience.eso.org/dataPortal/member.uid___A001_X133d_X325._G31.41p0.31__sci.spw25_27_29_31.cont.I.tt0.pbcor.fits
https://almascience.eso.org/dataPortal/member.uid___A001_X133d_X325._G31.41p0.31__sci.spw25_27_29_31.cont.I.tt1.pbcor.fits
--------------------------------

I'm using version 0.1.3. Which version are you using? You can get this info via:
alminer.__version__
Cheers,
Aida

Hi,
I am also using verison 0.1.3. The same version also been run by another person in their laptop with same error as mine.
Cheers,
Carmen

Also, I am running within a conda enviroment with python 3.9.
Yesterday I created a conda environment to install a fresh version of alminer. As it needs python 3:
conda create -n alminer python=3.9
conda activate alminer
pip install alminer

Hmm I think it works with the GitHub version but not PyPI (pip) which is odd as the PyPI version is the same as the GitHub minus some very minor differences. Can you either clone the Git repo or pip install via:
pip install git+https://github.com/emerge-erc/ALminer.git
Let me know if that works. I tested it in a virtual environment and it works for me.

Hi Aida,
Thanks so much! I uninstalled alminer and installed using your pip command:
pip install git+https://github.com/emerge-erc/ALminer.git
That fixed the issue :)


lazy-adr: Add Data Availability library

Description
This PR will contain the "DA lib" accompanied with an ADR describing the design as well as providing some context.
Rendered ADR: https://github.com/lazyledger/lazyledger-core/blob/ismail/da_lib_adr/docs/lazy-adr/adr-002-ipds-da-sampling.md
related to: https://github.com/lazyledger/lazyledger-core/issues/85, https://github.com/lazyledger/lazyledger-core/pull/163

After writing some of the implementation for the writing portion of this ADR, I think the API will have to change to accommodate some form of access to the IPFS node object if we're passing it around. If we use one of the alternatives mentioned, then the API specified still might have to be modified slightly, but that depends on which alternative we end up going with.

I posted a draft for the writing portion of this ADR at #178. The PR incorporates PutLeaves into PutBlock, as that allows for more optimization in batch adding the ipld nodes to the dag. It also adds an IPFS core object as an argument to PutBlock. Notably, the PR currently requires the erasure data be computed twice, as it is not currently cached, but that can change.

I'm not sure if defining the API perfectly is in the scope of this ADR

I mostly created this ADR such that we have a basis on which to discuss API alternatives. If no major shortcomings in the ADR, we can use it as a blueprint to start the implementation. It is definitely not complete and it won't be until we wrapped up the implementation. I do not think it is realistic to define the APIs perfectly without drafting at least "spikes" / "tracer bullet" implementations.
Actually, my suggestion is to merge this if it is sound and sane from a high-level pov. Because then, you and I can make the modifications on portions of the ADR while we work on the implementation in separate branches. The moment, we want to merge a PR that implements a part of this, we need to make sure the ADR matches the implementation API. At least that was the idea. I'm open to other suggestions.

OK, let's merge this then! I've captured a bunch of smaller todos here: https://github.com/lazyledger/lazyledger-core/issues/179


update file name for planned outages from ReEDS

ReEDS has changed the name of the output file for outages from outage_planned.csv to planned_outage.csv; this PR adapts the name in the reeds_us_mapping.json file to handle this.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 62.18%. Comparing base (0e09279) to head (2673b5f).

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #49   +/-   ##
=======================================
  Coverage   62.18%   62.18%
=======================================
  Files          29       29
  Lines        3115     3115
=======================================
  Hits         1937     1937
  Misses       1178     1178

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Looks good to me!

chore: Add new upgrader entry to move outage_planned.csv to planned_outage.csv


Confusing Statement in Step 2

In step 2, there is a note: "For guest access to work at all in Teams, you must set Members can invite to Yes."
Does this mean, that guest access in Teams would not even work if admins are adding the guest to the group?
If this is really the case, then I could never have (i) guest access in teams, and a (ii) strictly controlled management (i.e., exclusively by the admin) of the AD at the ssame time.
Is my understanding correct?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f737e278-7c89-e048-b60b-287615a829fb
Version Independent ID: 1df35258-7360-74c4-680a-e154c54291fe
Content: Microsoft Teams guest access checklist - Microsoft Teams
Content Source: Teams/guest-access-checklist.md
Service: msteams
GitHub Login: @LanaChin
Microsoft Alias: v-lanac

Hello @Christian-e-S
How are you?
Document has been updated after you added your inquiry, please see
https://github.com/MicrosoftDocs/OfficeDocs-SkypeForBusiness/commit/9c5009b241a37498e1f033913cff12a4ae57fdd0

@scanum please close this thread

@Christian-e-S Thank you for submitting feedback. We understand that this issue has been resolved.
Please feel free to re-open this issue if there is a specific area of the docs that we can improve or make better. Thank you.


Allow to reload a single file

VSCode Version: 1.17.2
OS Version: Kali 2.0 2017

Steps to Reproduce:

Reload a single file without reloading the whole windows
Using a keyboard shortcut

Reproduces without extensions: Yes/No

In which case would you want to reload a single file? If I have a text file opened in VSC, and I edit it with some other editor, VSC updates it automatically.

I generate a Java project using netbeans then I open the same project with
vsc, netbeans reload automatically a file when I modify it with vsc but it
is not always the case for vsc, so I ask a fonctionnality to do it manually
2017-11-09 14:57 GMT+01:00 Fabio Spampinato<EMAIL_ADDRESS>
In which case would you want to reload a single file? If I have a text
file opened in VSC, and I edit it with some other editor, VSC updates it
automatically.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Microsoft/vscode/issues/37901#issuecomment-343161489,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AWFtIRs2cbMrWn6m9Xzk7TVB4Md-3R4fks5s0wS8gaJpZM4QXoYL
.

This feature request will not be considered in the next 6-12 months roadmap and as such will be closed to keep the number of issues we have to maintain actionable. Thanks for understanding and happy coding!


Mockup workflow for demo

Upload data to clowder
PlantCV runs
view results
launch shell with data attached (would be nice, can also show 'mockup') later, Jupyter

Drafts:

I've placed a mockup for the demo as a presentation on google drive; anyone should be able to edit
@terraref/developers please comment / revise

Here is my own mockup of the overall workflow I created to help my own understanding after our meeting last week. This might not be perfect but perhaps it could be a useful diagram for the presentation.
The 3rd slide has a few questions that we don't necessarily need to answer before Friday.

I'd like to clarify a few things:

The Jupyter component we discussed last week is a technical piece we will need time to develop. It is different from the Jupyter demo defined in the demo workflow.
That being said, the significance of the Jupyter demo, from implementation point of view, is merely installing a Jupyter docker along with Clowder on Nebula and share a file system.
The demo workflow in my understanding is:

upload an image file by using the Clowder UI to show the user interface of data uploading;
upload another image file by using the Clowder REST API to show the batch uploading capability;
open an uploaded file in Clowder to show the mask and chart extracted by plantcv extractor; explain the architecture of the data management system and the scalable extractor framework.

Thanks,
Yan

@max-zilla I've combined your slides with Yan's earlier architecture draft here to keep the ideas in one place.
@yanliu-chn you are correct about the minimum requirement for the demo. It will also be good to demonstrate pushing via the API.


Please support to ESP32C6 (AEGHB-424)

When we is using esp32c6,this seems to be not supported .

@DoctorTag Please git apply the patch as the attachment and check again.
support_esp32c6.zip

@DoctorTag Closing this issue since there has been no update on this. Please feel free to reopen if required.


add response WriteProtobuf func

Codecov Report

Merging #1314 (78499ff) into master (fe7209e) will increase coverage by 0.12%.
The diff coverage is 53.39%.

:exclamation: Current head 78499ff differs from pull request most recent head 70a5d63. Consider uploading reports for the commit 70a5d63 to get more accurate results

@@            Coverage Diff             @@
##           master    #1314      +/-   ##
==========================================
+ Coverage   69.45%   69.58%   +0.12%
==========================================
  Files         373      372       -1
  Lines       26457    26637     +180
==========================================
+ Hits        18377    18536     +159
- Misses       6544     6547       +3
- Partials     1536     1554      +18

Impacted Files
Coverage Δ

container/gset/gset_str_set.go
91.22% <ø> (ø)

database/gdb/gdb.go
70.99% <ø> (+0.44%)
:arrow_up:

database/gdb/gdb_core_tracing.go
6.06% <0.00%> (-2.77%)
:arrow_down:

database/gdb/gdb_driver_mssql.go
0.00% <0.00%> (ø)

database/gdb/gdb_driver_mysql.go
63.51% <0.00%> (ø)

database/gdb/gdb_driver_oracle.go
19.33% <0.00%> (+3.85%)
:arrow_up:

database/gdb/gdb_driver_pgsql.go
0.00% <0.00%> (ø)

database/gdb/gdb_driver_sqlite.go
0.00% <0.00%> (ø)

database/gdb/gdb_model_join.go
88.23% <ø> (ø)

database/gdb/gdb_type_result.go
90.41% <ø> (ø)

... and 81 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 65131c6...70a5d63. Read the comment docs.


fill_in does not populate text fields appropriately for empty strings

We have a short feature spec that is essentially trying to fill in a form's text field as an empty string using fill_in, submitting it, and expecting a validation error to be returned from our server and rendered on the form. We're doing the following to fill in the empty string:
fill_in('Version Name', with: '')

Through looking at the form params received by our server controller, it seems that the text field does not actually get changed to a blank value. (1.0 was the initial value the text field was loaded with).
{"version"=>"1.0"}

It doesn't seem like there is a warning or description of an inability to pass in empty strings (just that a string must be passed as the value to the with key) for fill_in. Is this functionality supposed to be working?
Also, for context, we're using Rails and our form is built using React and Flux. Although, we've verified that using fill_in with a normal string like so does work:
fill_in('Version Name', with: 'New Name')

-- James && Marc<EMAIL_ADDRESS>Verbose feature spec log and Capybara debug logs:
In the feature spec that this log portrays, we're going to two different pages, the page with the form, and then filling in that text field. We can try to produce a reproducible test suite/app and logs if necessary.
test.txt

Hey @RochesterinNYC. A quick look at the implementation shows that blank strings should be supported. However, you'll need to ensure that you're not filling in a readonly field. Also, are you sure something else is not modifying the text? Finally, if you're loading the form field into some sort of object via JS and expecting the data to be sent via Ajax, it's quite possible that we're not firing the correct events for your JS to pick up the change. Right now, we only fire the change event. Given the field is being set to empty, we do not fire any keypress events.
This behavior has changed over time and we've struggled to find the best behavior across browsers (Chrome and Firefox via Selenium, capybara-webkit, etc). The current implementation is the best we've come up with.
If you could put together a simple test suite that replicates the issue, that'd be great. We can then pull it in to our test suite, or push it upstream to capybara for further discussion.

Closing as I've lost the original context of this issue. Cheers.


Do we still need this repository?

@FynnBe as far as I can see the functionality left here is only needed for deprecated things (the old transformations, dataset and model runner ideas).
Do we still need it for anything?
If not, we should consider either removing or at least archiving it.

There is not much left here, but

the runner should live here (still in tiktorch atm)
the transformations should be updated and live here (for numpy only as there is clear overlap with [bioimageio.torch](https://github.com/bioimage-io/pytorch-bioimage-io and potentially with a 'bioimageio.tf')

the runner should live here (still in tiktorch atm)

What exactly do you mean by runner? This: https://github.com/ilastik/tiktorch/tree/master/tiktorch/server/prediction_pipeline/_model_adapters?

the transformations should be updated and live here (for numpy only as there is clear overlap with [bioimageio.torch](https://github.com/bioimage-io/pytorch-bioimage-io and potentially with a 'bioimageio.tf')

For these, I thought that the spec repo would also be the better place, because they could serve as reference implementations.

yes, tiktorch's model adapters, or even the PredictionPipeline with pre- and postprocessing.
But each such adapter/pipeline/runner potentially goes together with it's own set of transformation implementations...
We could merge it all, but then we should have kept everything in python-bioimage-io.
I think some splitting makes sense and the spec just happens to live in python, but I see that more like a coincidence. we could scratch the whole validator and go json schema, who knows?^^
Example use cases could live there though, but not full implementations for several frameworks (and languages!?). Maybe merging pytorch-bioimage-io into python-bioimage-io (but keeping the separate modules bioimageio.core and bioimageio.torch as it is now) would make sense.

bioimageio.spec: python-bioimage-io -> spec-bioimage-io
bioimageio.core: python-bioimage-io
bioimageio.torch: pytorch-bioimage-io -> python-bioimage-io?
bioimageio.tf:   -> python-bioimage-io?

yes, tiktorch's model adapters, or even the PredictionPipeline with pre- and postprocessing.
But each such adapter/pipeline/runner potentially goes together with it's own set of transformation implementations...
We could merge it all, but then we should have kept everything in python-bioimage-io.
I think some splitting makes sense and the spec just happens to live in python, but I see that more like a coincidence. we could scratch the whole validator and go json schema, who knows?^^

I agree that some separation makes sense, but I don't think having separate repositories for python-bioimage-io, pytorch-bioimage-io and tf-bioimage-io make sense, especially since the prediction pipelines depend on having the frameworks available, so I very much like this idea:

bioimageio.spec: -> spec-bioimage-io
bioimageio.core: python-bioimage-io
bioimageio.torch: pytorch-bioimage-io -> python-bioimage-io?
bioimageio.tf:   -> python-bioimage-io?

I still think that the the pure numpy implementations of pre-and-postprocessings should go into the spec repository and serve as the reference implementation. Then, the spec repo is self-contained and python-bioimage-io depends on it anyways, so can use these transformations.

(Also, if we keep this repository and move pytorch-bioimage-io and stuff from tiktorch here, we should remove the outdated stuff in https://github.com/bioimage-io/python-bioimage-io/tree/master/bioimageio).


[21w07a] Crash when launching in dev environment

Output from IntelliJ
https://gist.github.com/kwpugh/d8dbcc67699fc51a8dbe2d2a1588e242

Just noticed the 1.17 branch, trying that now.

Tried with 0.0.1-beta22-1.17 and got the same error.

I got it working.


getR11sContainer and return IContainer

This change adds a method to getR11sContainer and changes the "getContainer" methods to return the interface instead of the container object.

What's the goal of this change? Is moving a copy of  getR11sContainer into examples a 1st step towards another goal?

What's the goal of this change? Is moving a copy of getR11sContainer into examples a 1st step towards another goal?

Good question: Updated in the description, but the goal is to help developers who are using R11s based services upgrade from our hello world demos.


add stats all test

add stats all test

Coverage increased (+0.1%) to 56.263% when pulling 66286fba89769039c34e26c413bd7802e5bb75a6 on zhaohai1299002788:add_stats_all_test into ecb061ae425f76ad28a597293b98f374ceba14ec on apache:develop.

Would you like to refer to other test cases? BTW, you also follow one issue one pr rule.


Searching terms contain hyphen vs no hyphen

I have strings in one of the fields which contain hyphen; for example anti-virus
If search query is  anti-virus, results returned contain terms anti and virus
If search query is antivirus, no results are returned.
Is there a way to update the pipeline (or any other function in lunr) to add antivirus also ?

This is caused by what characters Lunr considers to be word separators. It currently uses this regex /[\s\-]+/ so it sees "anti-virus" and converts that to the two tokens anti and virus. When you search for "antivirus" there is no hyphen and so Lunr considers it a single token, and it doesn't have a single token "antivirus", hence no results.
You can try changing what Lunr considers to be word separators to no longer include a hyphen e.g.
lunr.tokenizer.separator = /\s+/ // white space only

Then you can insert a pipeline function at the beginning of the pipeline. This will look for tokens that contain a hyphen and return the original token, plus tokens created by splitting on the hyphen, and maybe even without the hyphen for good luck. E.g. when we see "anti-virus" in the document, we will end up indexing "anti-virus", "anti", "virus", "antivirus".
var hyphenator = function (token) {
  // if there are no hyphens then skip this logic
  if (!token.toString().contains('-')) return token

  // split the token by hyphens, returning a clone of the original token with the split
  // e.g. 'anti-virus' -> 'anti', 'virus'
  var tokens = token.toString().split('-').map(function (s) {
    return token.clone(function () { return s })
  })

  // clone the token and replace any hyphens
  // e.g. 'anti-virus' -> 'antivirus'
  tokens.push(token.clone(function (s) { return s.replace('-', '') })

  // finally push the original token into the list
  // 'anti-virus' -> 'anti-virus'
  tokens.push(token)

  // send the tokens on to the next step of the pipeline
  return tokens
}

I've not tested the above, so you'll have to try it out and see how it works. Let me know how it works out or if you have any other questions.

Hi Oliver,
I am using the customPipeline in following way
var lunr =  require("lunr");
var customPipeline = function (builder) {
    var pipelineFunction = function (token) {...}; //hyphenator
    lunr.Pipeline.registerFunction(pipelineFunction, 'customPipeline');
    builder.pipeline.before(lunr.stemmer, pipelineFunction);
    builder.searchPipeline.before(lunr.stemmer, pipelineFunction);
    };
var idx = lunr(function () {
    this.use(customPipeline);
//define fields, add data
};

 var results = idx.search("anti-virus");
    console.log(JSON.stringify(results));
    var results = idx.search("antivirus");
    console.log(JSON.stringify(results));

First one prints results, but second does not.
Also I plan to use predefined/prebuilt indexes on client.
Do I need to add something like lunr.use(customPipeline); on the client ?
Please let me know

Could put together an example in jsfiddle (or similar) so I can see what is happening?
Adding your pipeline function to searchPipeline implies that is required for searching, so you will need to have your pipeline function available on the client side. The call that registers your pipeline function needs to happen both when building the index on the server side, and before loading the serialised index on the client side. As such its probably best to separate out the registering of the pipeline function from the plugin function, since you won't be calling use on the client side.
As a node module it might look something like this (untested):
var lunr = require('lunr')

var hyphenator = function (token) { ... }

// register the pipeline function, this needs to happen client & server side
lunr.Pipeline.registerFunction(hyphenator, 'hyphenator')

// define the plugin, this will only be used when building the index, not when loading
export function (builder) {
    builder.pipeline.before(lunr.stemmer, hyphenator);
    builder.searchPipeline.before(lunr.stemmer, hyphenator);
}

Oliver,
This seems to be working.
https://jsfiddle.net/msexkb0w/3/
I had missed out on the lunr.tokenizer.separator = /\s+/ earlier.
Also was unsure have to use the custom code on client side.
Thanks :)

Closing this as I think there is nothing more to do. Feel free to comment if you still have unanswered questions.

We are using your solution with the custom pipeline, but we are now facing some issues with this solution.
Especially when using in combination with multi-language support:
import * as lunr from 'lunr';
import * as lunrDE from 'lunr-languages/lunr.de';
import * as lunrMulti from 'lunr-languages/lunr.multi';
import * as lunrStemmer from 'lunr-languages/lunr.stemmer.support';

var hyphenator = function (token) { ... }

lunrStemmer(lunr);
lunrMulti(lunr);
lunrDE(lunr);
const languagePlugin = lunr.multiLanguage('en', 'de');

lunr.Pipeline.registerFunction(hyphenator, 'hyphenator');
lunr.tokenizer.separator = /\s+/; // white space only

const _that = this;

this.index = lunr(function() {
      this.b(0);
      this.ref('_id');
      this.use(languagePlugin);
      this.use(_that._plugin);
      this.pipeline.before(lunr.stemmer, hyphenator);
      this.searchPipeline.before(lunr.stemmer, hyphenator);
      this.pipeline.remove(lunr.de.stemmer);
      this.pipeline.remove(lunr.stemmer);
      this.searchPipeline.remove(lunr.de.stemmer);
      this.searchPipeline.remove(lunr.stemmer);
    });

The problem is that the hyphenator function is returning an array of tokens but lunr.stemmer.support.js expects a string as token not an array. Therefore we get an exception in lunr.stemmer.support.js at line 136. current.charCodeAt(...) -> current is not a single token but an array

Can you put together a simple reproduction in something like JSFiddle?
As far as I understand lunr-languages should work with both Lunr1 and Lunr2, from the README

Lunr Languages is compatible with Lunr version 0.6, 0.7, 1.0 and 2.X

cc @MihaiValentin

Forgive me, I manipulated your code example of hyphenator and that's why my code didn't behave like it should. I'm sorry.. 🙏🏼


swi-prolog: set meta.mainProgram and 8.3.9 -> 8.3.29

Motivation for this change
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[x] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 137061 run on x86_64-darwin 1

3 packages built:

ape
apeClex
swiProlog

Result of nixpkgs-review pr 137061 at c1fe21db run on aarch64-linux 1

5 packages built successfully:

ape
apeClex
pakcs
swiProlog
swiPrologWithGui


Updated FaceID

Added MAX78002 FaceID Files
Deleted old MAX78000 FaceID Files
Updated .yamllint for Windows compatibility
MAX78000 FaceID Files and demo-related files will be added soon.

Added MAX78000 files


How to download data????

How to download data????
When i try to download data, is there some error...

fbitem-8b16037cf12b5f958bf6d374db1c1429764c1b51
Reported by<EMAIL_ADDRESS>System: Firefox 51.0.0 / Windows 10 0.0.0
Referer: https://demo.gbif.org/occurrence/download?taxon_key=7952072
Window size: width 1366 - height 659
API log
Site log

@kbraak writes
On the current portal, after triggering a 'download' a notification box appears at the top of the download page with the following text:

Download running
The download has been started and is currently being processed.
Please expect 10 to 15 minutes for the download to complete.
A notification email with a link to download the results will be sent to the following addresses once ready:
email@address

As far as I can see this information is missing from the new portal. It is quite important to the user at it explains how and when they can retrieve the download.

@MortenHofft IMHO issues such as this that prevent the user from understanding how to download  data successfully should be a release blocker

Hi Kyle thanks for your comments - it should of course be there. And is now.


Propogate error message about argument mismatch to Python client

Codecov Report

Merging #3044 (36f53eb) into main (0388da6) will decrease coverage by 31.10%.
The diff coverage is 58.82%.

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #3044       +/-   ##
===========================================
- Coverage   73.76%   42.67%   -31.10%
===========================================
  Files          99      100        +1
  Lines       10701    10713       +12
===========================================
- Hits         7894     4572     -3322
- Misses       2807     6141     +3334

Impacted Files
Coverage Δ

python/graphscope/analytical/app/sssp.py
86.66% <0.00%> (ø)

python/graphscope/tests/unittest/test_app.py
0.00% <0.00%> (-96.94%)
:arrow_down:

python/graphscope/analytical/app/custom.py
88.88% <88.88%> (ø)

python/graphscope/analytical/app/__init__.py
100.00% <100.00%> (ø)

python/graphscope/analytical/app/wcc.py
72.41% <100.00%> (-13.80%)
:arrow_down:

... and 55 files with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0388da6...36f53eb. Read the comment docs.


Add Shared Libraries team in codeowners file

Based on Github Teams Proposal we want to update codeowners to reflect the team(s) that should be consulted before a file is changed. In this case we are adding the Shared Libraries team.

@Mrtenz Added snaps-devs as CODEOWNERS as well. cc @mcmire


sort statefulset pods from 0 to .....n

scale statefulset to 10
see pod-0, pod-1, ..., pod-10
scale statefulset to 1000
see pod-0, pod-1, pod-10, pod-100, pod-101.....

imo would make sense to "keep" the sort and not do it lexiographically
screenshot related:

https://github.com/lensapp/lens/blob/4aaf470c18900fffed25abaddf3523556b1ec11e/dashboard/client/components/%2Bworkloads-pods/pods.tsx#L75-L84

I think just properly handling names of the form ^[0-9]+.*$ and ^.*[0-9]+$ would be the best solution.


add just https://github.com/justcatthefish

just add :)
https://github.com/justcatthefish

That's cool you open-sourced full challenges. Get in touch with https://pwn.college/ctf-archive/.


Firebase remote config

Author the user story for this feature
As a developer, I want to control the features enabled on metrics so I can customize the app experience for users.
Describe the solution you'd like
Research if we can use Firebase remote config for feature toggles (such as FPS monitor or hiding the user/password login feature)

https://firebase.google.com/docs/remote-config
https://firebase.google.com/docs/remote-config/use-config-web

Considerations:

How deactivating Email/Password login option will affect security rules?
How well firebase remote config supported in flutter for web?

How deactivating Email/Password login option will affect security rules?

I think we'd keep the security rules the same. This is only about hiding the UI.

How well firebase remote config supported in flutter for web?

They have a full featured JS SDK, I think we can call into this from Dart (similar to the other FlutterFire SDKs). There's not currently an official dart wrapper.

https://stackoverflow.com/questions/62781826/accessing-remoteconfig-create-loop-flutter-web looks like this might be possible?

@bootstraponline we need to try it out, thanks for sharing!


Recursively Search for .msg files

We were trying to use mcap on a large ROS2 project and ran into an issue where .msg files were not being found in the install/<package_name>/share/<package_name>/msg/*.msg path. We got errors like below and the produced mcap files were empty.
[node-2] [WARN] [1668788082.046164202] [rosbag2_storage_mcap]: no .msg definition for AAA/BBB.msg, falling back to IDL

On closer look we saw that our package is structured to create .msg files in paths like install/<package_name>/share/<package_name>/<sub_directory_name>/*.msg. This is purely for organizational purposes and ROS2 doesn't have issue in finding these. So this PR modifies the MCAP plugin code to look for .msg files in any sub-directory within install/<package_name>/share/<package_name>/. It resolves our problem

Can you look at the contents of /opt/ros/<distro>/share/ament_index/resource_index/rosidl_interfaces/<pkg_name> ? What does it look like?
I suspect that using the resource index for rosidl_interfaces is the more "correct" way of doing this.

Looks like that file contains relative paths to msg and idl files. So would suggested solution be to read that file, form paths, match to desired .msg filename and check for existence?

I think so. @emersonknapp do you know more about the ament index and would you be able to validate this proposal?

We need to make this fix in foxglove_bridge as well. I created a ticket with a description of what I think is the correct way to use the ament_index_cpp library to resolve a datatype (package name plus type name) to an absolute path in the filesystem. https://github.com/foxglove/ros-foxglove-bridge/issues/58

This codebase is moving over to ros2/rosbag2, so i've created an issue there capturing this. Lets move discussion around what to do to that issue.


Problem with mutect when creating gemini database

Hi Brad,
I'm experiencing problems when running the pipeline, at the step where the gemini database was created (it seems to be performed by default now). The traceback is below. Could you kindly look into it pls?
[2017-07-20T08:37Z] GEMINI: create database with vcf2db
[2017-07-20T08:37Z] not in VCF: 1770_150317_N,1770_150317_N,1711_291216_N,1738_250117_N,1733_170117_N,1733_170117_N,1733_170117_N,1733_170117_N,1684_251116_N,1702_151216_B,1690_011216_N,1781_070317_B,1699_081216_B,1739_010217_B,1775_020317_N,1722_120117_B,1520_070616_B,1790_130317_B,1619_090916_N,1777_030317_B,1628_160916_B,1784_090317_B,1754_140217_N,1740_020217_N,1713_060117_N,727_180714_B,727_180714_B,1721_120117_N,1682_231116_N,1707_170117_N,939_291215_B,1776_020317_N,1390_290416_B,1771_240217_N,1704_271216_N,1697_081216_N,1795_170317_N,1793_160317_N,1552_150916_B,1522_090616_N,1081_150715_B,1486_290416_B,1787_100317_N,1752_200317_B,1627_160916_N,1764_210217_N,1742_030217_N,1694_061216_N,1785_090317_B
...
subprocess.CalledProcessError: Command '/mnt/projects/dlho/tancrc/bcbio_pipeline/anaconda/bin/vcf2db.py /mnt/projects/ngbhs/cirqseq/20170224_plasma/20170630_Set3/bcbio/set3_2/work/gemini/T1683_251116-mutect-decompose-vepeffects-annotated-gemini.vcf.gz /mnt/projects/ngbhs/cirqseq/20170224_plasma/20170630_Set3/bcbio/set3_2/work/gemini/T1683_251116-mutect-decompose-vepeffects-annotated-gemini.ped /mnt/projects/ngbhs/cirqseq/20170224_plasma/20170630_Set3/bcbio/set3_2/work/bcbiotx/tmpjVfW_B/T1683_251116-mutect.db
not in VCF: 1770_150317_N,1770_150317_N,1711_291216_N,1738_250117_N,1733_170117_N,1733_170117_N,1733_170117_N,1733_170117_N,1684_251116_N,1702_151216_B,1690_011216_N,1781_070317_B,1699_081216_B,1739_010217_B,1775_020317_N,1722_120117_B,1520_070616_B,1790_130317_B,1619_090916_N,1777_030317_B,1628_160916_B,1784_090317_B,1754_140217_N,1740_020217_N,1713_060117_N,727_180714_B,727_180714_B,1721_120117_N,1682_231116_N,1707_170117_N,939_291215_B,1776_020317_N,1390_290416_B,1771_240217_N,1704_271216_N,1697_081216_N,1795_170317_N,1793_160317_N,1552_150916_B,1522_090616_N,1081_150715_B,1486_290416_B,1787_100317_N,1752_200317_B,1627_160916_N,1764_210217_N,1742_030217_N,1694_061216_N,1785_090317_B
/mnt/projects/dlho/tancrc/bcbio_pipeline/anaconda/lib/python2.7/site-packages/sqlalchemy/sql/sqltypes.py:185: SAWarning: Unicode type received non-unicode bind param value 'T1683_251116'. (this warning may be suppressed after 10 occurrences)
(util.ellipses_string(value),))
/mnt/projects/dlho/tancrc/bcbio_pipeline/anaconda/lib/python2.7/site-packages/sqlalchemy/sql/sqltypes.py:185: SAWarning: Unicode type received non-unicode bind param value '-9'. (this warning may be suppressed after 10 occurrences)
(util.ellipses_string(value),))
/mnt/projects/dlho/tancrc/bcbio_pipeline/anaconda/lib/python2.7/site-packages/sqlalchemy/sql/sqltypes.py:185: SAWarning: Unicode type received non-unicode bind param value '2'. (this warning may be suppressed after 10 occurrences)
(util.ellipses_string(value),))
af 10
Traceback (most recent call last):
File "/mnt/projects/dlho/tancrc/bcbio_pipeline/anaconda/bin/vcf2db.py", line 850, in
impacts_extras=a.impacts_field)
File "/mnt/projects/dlho/tancrc/bcbio_pipeline/anaconda/bin/vcf2db.py", line 218, in init
self.load()
File "/mnt/projects/dlho/tancrc/bcbio_pipeline/anaconda/bin/vcf2db.py", line 282, in load
i = self._load(self.cache, create=True, start=1)
File "/mnt/projects/dlho/tancrc/bcbio_pipeline/anaconda/bin/vcf2db.py", line 275, in _load
self.insert(variants, expanded, keys, i, create=create)
File "/mnt/projects/dlho/tancrc/bcbio_pipeline/anaconda/bin/vcf2db.py", line 323, in insert
self.create(variants, variant_impacts)
File "/mnt/projects/dlho/tancrc/bcbio_pipeline/anaconda/bin/vcf2db.py", line 417, in create
self._create(dvariants, v_cols)
File "/mnt/projects/dlho/tancrc/bcbio_pipeline/anaconda/bin/vcf2db.py", line 432, in _create
col.type.length = int(1.2 * len(d[name]) + 0.5)
TypeError: object of type 'float' has no len()
' returned non-zero exit status 1

Thanks much for the report and apologies about the problem. There was a fix in vcf2db for this issue but we didn't have an up to date package for it. I updated it so if you do:
bcbio_conda install -c conda-forge -c bioconda vcf2db

or run a full bcbio --tools update. Hope this gets things working for you.

Thanks Brad. After I updated vcf2db (I ran the full bcbio --tools update), the previous error was resolved but another error occurred.  I'm not sure what is going on here and I'll have to ping @brentp to see if he has an idea. Brent, we're updated to the latest vcf2db but still running into some problems with MuTect inputs.
From the bcbio side, if you don't need gemini for this project you can set tools_off: [gemini] and it'll skip this step:
http://bcbio-nextgen.readthedocs.io/en/latest/contents/configuration.html#tweaking-defaults
Hopefully we can get the actual issue debugged but this could keep you moving forward. Thanks again for all the help figuring this out.

it looks like you are hitting a problem that is fixed in vcf2db.py:
you are getting an error at:
col.type.length = int(1.2 * len(d[name]) + 0.5)

but that is now:
col.type.length = int(1.2 * len(str(d[name])) + 0.5)

which will bypass the error you are getting.
@chapmanb I'll update the conda recipe later today.

actually, it looks like the conda recipe is updated so this should be resolved by just updating vcf2db.

Sorry Brent -- we fixed the initial error by updating vcf2db, then ran into the second error in the later message in this thread:
https://github.com/chapmanb/bcbio-nextgen/issues/2011#issuecomment-316897355
So I think we've fixed the str issue, but not sure about the second error. Thanks so much.

shoot. missed that.
@phu5ion can you send me this portion of your vcf: chr5:68288680-68288689 along with the header so I can debug?

T1486_290416-freebayes-decompose-vepeffects-annotated-gemini.vcf.gz
Hi @brentp
Here you go. It's the whole vcf as there's only 1 call inside.

Thanks, look like this got fixed and updated in bcbio. So should be all set, so closing.


zookeeper-cluster zoo.cfg   server.1 server.3 server.2 The order is wrong

hi;
#docker logs 2b9847920152bd2f648a6f8

ZOOKEEPER_ID=1
tickTime=2000
initLimit=20
syncLimit=10
maxClientCnxns=60
server.1=<IP_ADDRESS>:2888:3888
server.3=<IP_ADDRESS>:2888:3888
server.2=<IP_ADDRESS>:2888:3888
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper/bin/../conf/zoo.cfg

server.1 server.3 server.2 The order is wrong

The order doesn't matter but if it is important to you, you can change env into env | sort in start.sh.
If you create a pull request for that, I'll gonna merge it but please test it before ;)


Failed to resolve: android.arch.lifecycle:runtime:2.1.0

Hi
I have a problem in integrating PayPal Checkout. I am getting this error:
Failed to resolve: android.arch.lifecycle:runtime:2.1.0 along with Failed to resolve: android.arch.lifecycle:extensions:2.1.0.
When I run the app, the build crashes

Hey @dscoder421! Thank you for reporting this.
Can you provide more information about the issue and the integration you have? I.e. Android version, checkout sdk version, moment of the crash, etc. Thanks!

Thanks @andrespelaezp. The Android API level is 26 , PayPal Checkout version is 0.2.0 and when I try to run my app it crashes at the task :app:mergeDebugNativeLibs

Hey @dscoder421, looks like there could be some dependencies and Jetifier issues with the 0.2.0 version, we highly recommend upgrading to the last version of the checkout SDK which is 0.5.2 where these issues where fixed.
Let me know if this change fixes your crash.

Thanks @andrespelaezp, It worked. I appreciate your quick support.


error

git submodule add https://github.com/budparr/gohugo-theme-ananke.git themes/ananke
fatal: not a git repository (or any parent up to mount point /Volumes)
WHY??

 test

Hi @arguskao This looks like a local GIt issue to me. Might try to the community forums as there might be some folks there who have encountered this sort of thing before .https://discourse.gohugo.io/


Fail to compile with skalibs <IP_ADDRESS>

Sockexec not compatible with skalibs <IP_ADDRESS> and above because of api changes
From the release page:
skalibs/ushort.h, skalibs/uint.h and skalibs/ulong.h removed. skalibs/types.h is now a generic header for fmtscan primitives for various types.

Done and done! Thanks for the report, I've updated the code to work with skalibs <IP_ADDRESS>+, and updated the travis builds to use the newer skalibs.


Protocol error while handling subscribe error

I'm publish message from the latest autobahn-js to a autobahn-python subscriber. The router is not crossbar, but a wamprpc 2.0 implementation wamp-server. The python code uses serializers=[JsonSerializer()].
Now the python subscriber receives a message like this
    [36,<PHONE_NUMBER>665722,<PHONE_NUMBER>665722, 'testpy.echo', [], {}]

But I met the error
  File "C:\Users\xusta\Miniconda3\lib\site-packages\autobahn\wamp\message.py", line 274, in check_or_raise_extra
    raise ProtocolError(u"{0}: invalid type {1}".format(message, type(value)))
autobahn.wamp.exception.ProtocolError: 'details' in EVENT: invalid type <class 'str'>

This is how I subscribe
    client.subscribe(shutdown, topic="testpy.shutdown", options=SubscribeOptions(details=False))

The 4th parameter is topic. I see the WAM RPC spec support to be a string.
    [16, 239714735, {}, "com.myapp.mytopic1", [], {"color": "orange",
        "sizes": [23, 42, 7]}]

In autobbahn\wamp\message.py line 2439, the 4th parameter is not topic but details
        # my proposed workaround
        # if type(wmsg[3]) == str:
        #     wmsg[3] = dict(topic=wmsg[3])

        details = check_or_raise_extra(wmsg[3], u"'details' in EVENT")

It requires this param to be a dict, like this {topic: "'testpy.echo"}. Is it possible to accept a string arg? I currently use a workaround. But I think autobahnjs and autobahnpython should be able to communciate very well.

not a bug here, conforms to spec: https://github.com/crossbario/autobahn-python/blob/master/autobahn/wamp/message.py#L2306

Thank you very much, it looks like a wamp-server (nodejs) implementation problem. I think I found the root cause.

@stanleyxu2005 np, glad you found it!


Ability to auto-generate Subapplication #loaded, #failedRemove, and #removed methods with pharo compatible mode

Whenever we have specified "pharo compatible mode/strategy", we should allow to autogenerate those 3 methods as the old MonticelloImporter did.
Each generated subapplication should include class-side #loaded, #failedRemove, and #removed methods. These methods are identical for all generated Subapplications.
loaded
 "Send #initializeOnLoad to every class defined in this subapplication that implements it."

 defined do: [ :aClass |
  (aClass class includesSelector: #initializeOnLoad) ifTrue: [ aClass initializeOnLoad ] ]

failedRemove

 self loaded

removing
 "Send #unload to every class defined in this subapplication that implements it."

 defined do: [ :aClass |
  (aClass class includesSelector: #unload) ifTrue: [ aClass unload ] ]

The #loaded method depends on the special handling of class-side #initialize methods in the original (Pharo) Seaside classes. The Monticello Importer replaces #initialize methods with class-side #initializeOnLoad methods having identical content.
Generally the only sender of #initializeOnLoad in Grease and Seaside is the #loaded method.

Not sure why MonticelloImporter replaces class side initialize with initializeOnLoad

This was solved in d90ac1d


How to use OpenCL datatypes like image2d_t in Compute Library code?

Output of 'strings libarm_compute.so | grep arm_compute_version':
Platform:
arm64-v8a
Operating System:
Android O

Problem description:
How can I use OpenCL datatypes like image2d_t in Compute Library code?

Hello,
At the moment we have no API for using other cl types apart from cl_buffer. Is there a specific reason for wanting to used image2d_t?


TypeError: Cannot read properties of null (reading 'db')

my code :
import localforage from "localforage"; export const db_upload = localforage.createInstance({ name: "test", storeName: "upload" });
error：
TypeError: Cannot read properties of null (reading 'db')     at _tryReconnect (localforage.js:770:1)     at localforage.js:819:1
The localforage did not initialize successfully, causing instance creation to fail. How can we ensure that the localforage is initialized successfully before creating an instance.

Up

@Chinesename, are you using React or what?
Can you provide more context?


Generate form without using scaffold

Hi I noticed that the form generator was deleted and the formtastic:install copy the template but to get the form i need to run scaffold, is there a way to just get the form without running scaffold?

Hmmm, it was removed for 2.0. Would accept a pull request patch that brings it back!


Automate testing for backlevel liboqs variants

oqsprovider can & must be built independently of liboqs. Together with separate packaging and distribution mechanisms for liboqs and oqsprovider this necessitates the capability to generate suitable oqsprovider code for different, possibly pre-installed, versions of liboqs. This capability is enabled by retaining and using suitable "generator.yml" files for different liboqs versions.
This issue it to suggest automating correct code generation and testing for (various) such oqsprovider+specific (downlevel) liboqs versions.
Tagging particularly @dstebila for input/suggestions how, whether & how far back to enable this as well as whether it is worth while integrating such test facility into the CI chain of liboqs to avoid the possibility to create mismatching versions of liboqs and oqsprovider (as happened with 0.9.1).

Closing as no feedback received as to the value of supporting downlevel versions of liboqs.


Virtualisation.Framework seems broken on 12.2.1 (arm64)

Hey,
While trying to boot a linux VM using Virtualisation.Framework, it seems broken(see error attached). I have also tried vmcli to start a VM and getting a similar error.
So not a bug against UTM, but a PSA, and perhaps a call for workarounds?
This is a MBA, freshly formatted, so kinda stumped!

Sorry, seems I had a busted kernel image!

Sorry, seems I had a busted kernel image!

Hey! Can you tell me which kernel image did you use eventually? I used the fedora one here but I saw the same error as you did. Thanks!

Were you able to get a serial console with that kernel image?

Were you able to get a serial console with that kernel image?

Yes I was! But it was very buggy - there were lots of kernel errors when I do updates.


Configure the entire timeline with one index and style

I was thinking about this lot and I probably don't want to go this path. I'm posting here just for FYI. My reasoning:
We want to utilize composition in a way that the item/step is independent from its parent container. That allows you to delegate a lot of functionality and resolution to the logic that is handling the particular step rendering.
If the step would not be "independent", i.e. (part of) its visual would be decided/affected by a parent container, then there is a question, why not make the component a single call completely, e.g. something like this in Kotlin (simplified):
Timeline(
   steps = listOf("Step 1", "Step 2"),
   activeStep = 1,
)

Though, I'm still not convinced about the composition approach.

I was thinking about this lot and I probably don't want to go this path. I'm posting here just for FYI. My reasoning:
We want to utilize composition in a way that the item/step is independent from its parent container. That allows you to delegate a lot of functionality and resolution to the logic that is handling the particular step rendering.
If the step would not be "independent", i.e. (part of) its visual would be decided/affected by a parent container, then there is a question, why not make the component a single call completely, e.g. something like this in Kotlin (simplified):
Timeline(
   steps = listOf("Step 1", "Step 2"),
   activeStep = 1,
)

Though, I'm still not convinced about the composition approach.

The TimeLine is a single call. Whether the data for steps are provided using a list parameter or using the @ViewBuilder parameter depends on further usage. The @ViewBuilder may be more composable and flexible, especially when we want to allow caller to provide custom @ViewBuilder closure for each step.
We can also keep the current approach where it is callers responsibility to model state correctly (after all, it is possible that there will be multiple active states or more combinations of past/future+status states).
This draft goal is now mostly to:

Provide similar interface to Tabs (which has similar semantics - only one item can be active at a time)
Proof of concept for any future similar composited components

Closing as obsolete for now.


Scanner does not reach end of file token under some circumstances

I haven't looked into this in full detail yet -- I have only the one example -- but if I have code of the form:
var scanner = ts.createScanner(ts.ScriptTarget.ES5, false, sourceFile.text);
while (scanner.scan() !== ts.SyntaxKind.EndOfFileToken) {
    // do stuff here
}

but sourceFile.text is something that doesn't compile like just var a=; then the loop never exits, as the end of file token is never hit -- it just hits the semicolon token over and over again. Is this on purpose? Is there a more standard way to loop until you hit the last token? (I would assume checking if the previous start is the same as the current start, which is easy enough to do, but I'm not sure how intentional this is, regardless).

Woah, that doesn't seem like it could be right - I tried this out
/// <reference path="bin/typescript.d.ts" />

import ts = require("typescript");
var sourceText = "var a=;";

var scanner = ts.createScanner(ts.ScriptTarget.ES5, false, sourceText);

while (true) {
	var token = scanner.scan();
	console.log("" + token + ": " + ts.SyntaxKind[token]);

	if (token === ts.SyntaxKind.EndOfFileToken) {
		break;
	}
}

> node .\bin\tsc.js .\scanSample.ts --module commonjs
scanSample.ts(10,48): error TS4085: Index expression arguments in 'const' enums must be of type 'string'.

(ignore the error)
node .\scanSample.js
96: VarKeyword
5: LastTriviaToken
63: Identifier
51: FirstAssignment
21: FirstOperator
1: EndOfFileToken

...that is decidedly odd. That simple bit seems to work fine for me as well -- but my code is very clearly breaking here, and I'm not sure why. Let me investigate a bit.

...nevermind. I figured out where I went wrong; it was something else. Sorry, and thanks!


Include yarn.lock in the repository

Summary
Commit yarn.lock to the repository. A lockfile is automatically generated whenever yarn install is run, yet yarn.lock is not an untracked file (it is absent from .gitignore) and so it should be checked in to allow the same dependency tree to be installed across all contributors' machines.
Things to Consider
There can be potential for variation in the committed yarn.lock files - chiefly this is caused by differing Node versions between users (nvm being the culprit for that). We may need to introduce a set Node version (let's say 10, seeing as 12 is reported to cause problems according to the README).
Also, as Heroku will be used to host the site soon (at the time of writing), this is even more important to avoid breaking anything on the FE if updated libraries get introduced by accident.
Criteria for Completion

[ ] An up-to-date, compatible yarn.lock file generated with a suitable Node version.

Thank you for opening this PR @stanleypliu! For context -- the deployed version is set to node-v12.16.2-linux-x64. So we should fix it to node 12.
@stanleypliu would you like to take this on?

Sure, that Node version sounds sensible.


 was created without expected prop 'form'

When I try to use the login form component, I get the following error:
 was created without expected prop 'form'

I'm not able to reproduce the error.
I close the issue for now.
Please give me more details in the future issue.