Gene mapping between user-provided scRNA and genes from SCimilarity during vae training

When using the pre-trained weight from SCimilarity, how does VAE_train.py account for different genes between user provided adata and SCimilarity? What if a gene is present in user's data but not in SCimilarity? Or what if a gene is present in SCimilarity but not in user's data?
There is indeed a num_genes parameter from VAE_train.py to control the dimension of VAE, such that it fits user provided scRNA, but I am not seeing it having any control over the gene order. When trying to reproduce vae training step from this command:
CUDA_VISIBLE_DEVICES=0 python VAE_train.py --data_dir '/workspace/projects/001_scDiffusion/data/data_in/tabula_muris/all.h5ad' --num_genes 18996 --state_dict "/workspace/projects/001_scDiffusion/scripts/scDiffusion/annotation_model_v1" --save_dir '../checkpoint/AE/my_VAE' --max_steps 200000 --max_minutes 600
I got these loss reports, which are always around 0.04:
step  0 loss  0.21746787428855896
step  1000 loss  0.04769279062747955
step  2000 loss  0.048065099865198135
step  3000 loss  0.04667588323354721
step  4000 loss  0.045960813760757446
Could you please provide some clarification or possible solution to this? Thank you so much!

Hi! The num_genes parameter is used to control the shape of the first layer of encoder and the last layer of decoder, which you can see from the VAE_model.py line 52. This means that the first layer of the encoder would not use the weight from SCimilarity (so does the last layer of the decoder) and the size would be the same as the user's input gene set.
The loss seems to drop normally, have you tried finishing the training process and seeing the reconstruction result?

Thanks. Yes, I have tried both pre-trained model from SCimilarity and VAE without pre-trained model, my VAE loss is always around 0.04, even near the end of the training
step  194000 loss  0.042236313223838806
step  195000 loss  0.03814302757382393
step  196000 loss  0.043102916330099106
step  197000 loss  0.04345700144767761
step  198000 loss  0.042990997433662415
step  199000 loss  0.039488837122917175
Is this actually expected for the WOT data?
Additionally, what classifier is considered to be "good enough"? For the WOT data, the training accuracy at the end of the training process is 0.164, which is far from being perfect. Is this expected as well? What train_acc should we expect in general?
| grad_norm      | 0.544    |
| param_norm     | 101      |
| samples        | 1.28e+07 |
| step           | 9.99e+04 |
| train_acc@1    | 0.164    |
| train_acc@1_q0 | 0        |
| train_acc@1_q1 | 0        |
| train_loss     | 2.42     |
| train_loss_q0  | 3.6      |
| train_loss_q1  | 2.59     |
Thank you so much!

The training loss looks normal. This might indicate that the model has reached its convergence. If you still have concerns about that, you can try to use the model you trained to reconstruct the data and see if the reconstructed data matches the training data.
As for the classifier, since the training samples are noised, the training accuracy might be very low, which is in line with expected. The accuracy of 0.164 should be normal, but I'm sorry that I can't give you specific expectations. I think it's good enough so long as it is obviously higher than random choice.

The training loss looks normal. This might indicate that the model has reached its convergence. If you still have concerns about that, you can try to use the model you trained to reconstruct the data and see if the reconstructed data matches the training data.
As for the classifier, since the training samples are noised, the training accuracy might be very low, which is in line with expected. The accuracy of 0.164 should be normal, but I'm sorry that I can't give you specific expectations. I think it's good enough so long as it is obviously higher than random choice.

Thank you! I have another general question about the VAE model. Technically, it is just an autoencoder, not VAE, since we are only optimizing reconstruction loss but not kl divergence. Is there any advantage of using autoencoder over VAE? Given that we don't heavily rely on the pre-trained weight from SCimilarity.
Additionally, given that we are using the entire dataset for training, how can we avoid overfitting?
Thank you!

For the first question, since the diffusion model has no strict requirements for the distribution of the input training data, there is no need to constrain the feature in the hidden space using KL loss or other variational loss, and the original distribution of the encoder output is just fine. Though we didn't put an experiment to prove that, I do think the extra loss in the latent space would make reconstruction less effective.
As for the overfitting problem, if the target is to augment the existing data, overfitting might not be a serious problem. But for the out of distribution data generation, there indeed needs a validation set to justify that (which we didn't do, cause we didn't observe overfitting). You can split a part of the dataset (totally apart from the rest) to validate the model.

For the first question, since the diffusion model has no strict requirements for the distribution of the input training data, there is no need to constrain the feature in the hidden space using KL loss or other variational loss, and the original distribution of the encoder output is just fine. Though we didn't put an experiment to prove that, I do think the extra loss in the latent space would make reconstruction less effective.
As for the overfitting problem, if the target is to augment the existing data, overfitting might not be a serious problem. But for the out of distribution data generation, there indeed needs a validation set to justify that (which we didn't do, cause we didn't observe overfitting). You can split a part of the dataset (totally apart from the rest) to validate the model.

Thank you so much!


Update github.com/cgroschupp/go-client-confluent-cloud to suppress debug logging

Hey :wave:
It would be awesome to update the github.com/cgroschupp/go-client-confluent-cloud dependency when / if this PR lands https://github.com/cgroschupp/go-client-confluent-cloud/pull/22. That PR allows configuring debug logging for the Confluent client, which at the moment is hard coded to true.
Cheers

Thanks @raytung  I'll pull it in when it gets merged.

Hey @Mongey, sorry to be a pain, but is there any ETA on a release for this change? 😇

Released in v0.0.12

Thanks my dude! ❤


usersplitter by date returns less items in tests than required

LAMA team reported that UserSplitter returns less items than required by the user_test_size with shuffle=False.
Users have sufficient interactions, but timestamps could be equal for all/ a part of interactions.

We was not able to reproduce


GDB in VScode can't get file pointer (get 0x0) and result in segmentation fault.

VSCode Version: 1.45.1
OS Version: Linux arch 5.6.14-arch1-1 #1 SMP PREEMPT Wed, 20 May 2020 20:43:19 +0000 x86_64 GNU/Linux

test code:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>

#define BUFFERSIZE 64
int main()
{
    char buff[BUFFERSIZE];
    FILE *file_p = fopen("testfile.txt","r");
    if (!file_p)
        printf("error, reason: %d",errno);
    printf("ptr addr: %p\n",file_p); // you can see the addr in terminal's gdb or stdout.
    while(fgets(buff,BUFFERSIZE,file_p)==NULL); // In Vscode gdb, segmentation error will happen.
    printf("%s",buff);
    fclose(file_p);
    return 0;
}

launch.json:
{
    "version": "0.2.0",
    "configurations": [

        {
            "name": "gcc - 建置及偵錯使用中的檔案",
            "type": "cppdbg",
            "request": "launch",
            "program": "${fileDirname}/${fileBasenameNoExtension}",
            "args": [],
            "stopAtEntry": false,
            "cwd": "${workspaceFolder}",
            "environment": [],
            "externalConsole": false,

            "MIMode": "gdb",
            "setupCommands": [
                {
                    "description": "啟用 gdb 的美化顯示",
                    "text": "-enable-pretty-printing",
                    "ignoreFailures": true
                }
            ],
            "preLaunchTask": "gcc build active file",
            "miDebuggerPath": "/usr/bin/gdb"
        }
    ]
}

tasks.json
{
    "tasks": [
        {
            "type": "shell",
            "label": "gcc build active file",
            "command": "/usr/bin/gcc",
            "args": [
                "-g",
                "${file}",
                "-o",
                "${fileDirname}/${fileBasenameNoExtension}"
            ],
            "options": {
                "cwd": "/usr/bin"
            }
        }
    ],
    "version": "2.0.0"
}

and the content of  testfile.txt:
hello

As title. The code is simplified from origin but still can run in normal execution or gdb called by command line.
But, the problem is: I can't get the file_p pointer address in VScode gdb.
If I have any deficiencies, please let me know.
Thanks for your help.

\extC

Oops, I should have double checked the position of this post.
Sorry for that.
I will re-post again on the true repo.


Tests for parsing strings, long strings and long comments

Finally had some time to add tests for strings and comments

Merged! :)


Instruction Types

Moin,
thanks for your work!
I wonder if other instruction types or step parameters exist? It would be nice to also create the instructions for the machine like

set  time to 5 seconds and blender speed to 8.5
switch to scale

Danke und Gruß,
Markus

Moin Markus,
yes that is possible.
If you add steps to the recipe they support annotations like this:
{
  "type": "STEP",
  "text": "New Step using blender for 5 Sek./Stufe 8.5",
  "annotations": [
    {
      "type": "TTS",
      "data": { "speed": "8.5", "time": 5 },
      "position": { "offset": 27, "length": 16 }
    }
  ]
}

The position member seems to encode the labels position in the text, which will be highlighted. I'll try to further explore the mechanism and update the docs in this repository.
VG
Christian

Moin Christian,
very cool, thank you.
VG
Markus


Ability to open files in a single tab one after another

Hello there!
My project consists of many small files and it's becoming quite difficult to manage this codebase with the default approach of "one file per editor tab". About 85% of my vertical screen space is left empty and unused. While there is already a feature to search for files within a tab, which allows me to view multiple files at once using 'search \n hack', it's currently not possible to edit these files from this view.

I believe it would be very useful to allow people to open files one after another in a single tab for editing, similar to how the search view displays them. This would allow users to easily navigate through many files they are working on, as they would scroll through each file one after the other.
Thank you for your attention to this issue. I look forward to hearing back from you soon.

/duplicate https://github.com/microsoft/vscode/issues/143024

@bpasero Please reopen this issue as it is not a duplicate of https://github.com/microsoft/vscode/issues/143024. My proposal differs from that one. I do not want to see split views with independent scrolling. Instead, I would like to have the ability to view files one after another in the editor. When you open three files in the editor, the contents of the first file will appear at the top of the screen. As you scroll to the end of that file, the contents of the next file will start to appear. After the contents of the second file, you will see the contents of the third file, and so on. This is similar to how notebooks are displayed, as a long scrolling, but with file contents instead. Please look at the attached screenshot for a better understanding of what I mean.


Deploy failure with ESM

Impact: This completely breaks now.sh for me. I won't be able to deploy new builds until we get this fixed.
Update: I am no longer depending on the module mentioned below, but am still getting errors about esm. Apparently, something else is also depending on esm.
I just created a new npm module that uses esm, and then tried to deploy a project that uses the module. However, the deployment is failing with the following error output:

./node_modules/gql-validate/node_modules/esm/esm.js Critical dependency: require function is used in a way in which dependencies cannot be statically extracted  ./node_modules/gql-validate/node_modules/esm/esm.js Module not found: Can't resolve 'internal/bootstrap/loaders' in '/tmp/634390ac/node_modules/gql-validate/node_modules/esm'
--
Jul 17 2019 11:43:47:701 | ...package.json | > Build error occurred
Jul 17 2019 11:43:47:703 | ...package.json | { TypeError: Cannot destructure property dir of 'undefined' or 'null'.     at Object. (/tmp/634390ac/.next/serverless/pages/register.js:9022:1704)     at Object.eqrp (/tmp/634390ac/.next/serverless/pages/register.js:9023:30)     at webpack_require (/tmp/634390ac/.next/serverless/pages/register.js:23:31)     at Object. (/tmp/634390ac/.next/serverless/pages/register.js:9011:11)     at Object.epOS (/tmp/634390ac/.next/serverless/pages/register.js:9015:30)     at webpack_require (/tmp/634390ac/.next/serverless/pages/register.js:23:31)     at Module.edYE (/tmp/634390ac/.next/serverless/pages/register.js:8825:20)     at webpack_require (/tmp/634390ac/.next/serverless/pages/register.js:23:31)     at /tmp/634390ac/.next/serverless/pages/register.js:91:18     at Object. (/tmp/634390ac/.next/serverless/pages/register.js:94:10) type: 'TypeError', '$error': '$error' }

See this issue in standard-things/esm.

@jdalton (esm author) took a look at the error. He seems convinced that a bundle script is ignoring the package.json "module" field:

If you can find out why a bundle script is ignoring module then I think you'll resolve the issue but that's a bit beyond esm. Keep digging and report back.

All of my now deploys are failing now with an error like this:
2019-07-23T23:52:55.635Z  Creating an optimized production build ...
2019-07-23T23:52:55.637Z
2019-07-23T23:52:56.026Z  > Using external babel configuration
2019-07-23T23:52:56.026Z  > Location: "/tmp/36a95be8/.babelrc"
2019-07-23T23:53:14.113Z  Compiled with warnings.
2019-07-23T23:53:14.113Z
2019-07-23T23:53:14.114Z  ./node_modules/esm/esm.js
2019-07-23T23:53:14.114Z  Critical dependency: require function is used in a way in which dependencies cannot be statically extracted
2019-07-23T23:53:14.114Z
2019-07-23T23:53:14.114Z  ./node_modules/esm/esm.js
2019-07-23T23:53:14.114Z  Module not found: Can't resolve 'internal/bootstrap/loaders' in '/tmp/36a95be8/node_modules/esm'
2019-07-23T23:53:14.114Z
2019-07-23T23:53:14.202Z  > Build error occurred
2019-07-23T23:53:14.204Z  TypeError: Cannot destructure property `dir` of 'undefined' or 'null'.
2019-07-23T23:53:14.204Z      at Object.<anonymous> (/tmp/36a95be8/.next/serverless/pages/index.js:4819:1781)
2019-07-23T23:53:14.204Z      at Object.JC+t (/tmp/36a95be8/.next/serverless/pages/index.js:4820:30)
2019-07-23T23:53:14.204Z      at __webpack_require__ (/tmp/36a95be8/.next/serverless/pages/index.js:23:31)
2019-07-23T23:53:14.204Z      at Object.<anonymous> (/tmp/36a95be8/.next/serverless/pages/index.js:4796:11)
2019-07-23T23:53:14.204Z      at Object.Ici4 (/tmp/36a95be8/.next/serverless/pages/index.js:4800:30)
2019-07-23T23:53:14.204Z      at __webpack_require__ (/tmp/36a95be8/.next/serverless/pages/index.js:23:31)
2019-07-23T23:53:14.204Z      at Object.q27P (/tmp/36a95be8/.next/serverless/pages/index.js:14590:65)
2019-07-23T23:53:14.204Z      at __webpack_require__ (/tmp/36a95be8/.next/serverless/pages/index.js:23:31)
2019-07-23T23:53:14.204Z      at Object.1TCz (/tmp/36a95be8/.next/serverless/pages/index.js:1015:24)
2019-07-23T23:53:14.204Z      at __webpack_require__ (/tmp/36a95be8/.next/serverless/pages/index.js:23:31)
2019-07-23T23:53:14.216Z  npm
2019-07-23T23:53:14.216Z   ERR! code ELIFECYCLE
2019-07-23T23:53:14.216Z  npm
2019-07-23T23:53:14.217Z   ERR! errno
2019-07-23T23:53:14.217Z   1
2019-07-23T23:53:14.218Z  npm
2019-07-23T23:53:14.218Z   ERR! @ now-build: `next build`
2019-07-23T23:53:14.218Z  npm
2019-07-23T23:53:14.218Z  ERR! Exit status 1
2019-07-23T23:53:14.218Z  npm ERR!
2019-07-23T23:53:14.218Z  npm ERR! Failed at the @ now-build script.
2019-07-23T23:53:14.218Z  npm
2019-07-23T23:53:14.218Z  ERR! This is probably not a problem with npm. There is likely additional logging output above.
2019-07-23T23:53:14.230Z
2019-07-23T23:53:14.230Z  npm ERR! A complete log of this run can be found in:
2019-07-23T23:53:14.230Z  npm ERR!     /tmp/.npm/_logs/2019-07-23T23_53_14_219Z-debug.log
2019-07-23T23:53:14.234Z  Error: Exited with 1
2019-07-23T23:53:14.234Z      at ChildProcess.child.on (/tmp/58eefe6453c9eb32/.build-utils/node_modules/@now/build-utils/dist/fs/run-user-scripts.js:30:24)
2019-07-23T23:53:14.234Z      at emitTwo (events.js:126:13)
2019-07-23T23:53:14.234Z      at ChildProcess.emit (events.js:214:7)
2019-07-23T23:53:14.234Z      at maybeClose (internal/child_process.js:925:16)
2019-07-23T23:53:14.234Z      at Process.ChildProcess._handle.onexit (internal/child_process.js:209:5)

That's the output from a recent deploy: now logs tdd-day-d2gepinax.now.sh

Hey @ericelliott, thanks for the report. I've been told that we are working on a fix for this now. Will have an update for you soon!

Cool @TooTallNate -- I tracked down the other dependency breaking my build. It's another module I authored that also uses esm. As you can see, if the bundler knows how to handle ES6 modules without esm, it should be able to use the module field from package.json.

Hey @ericelliott!
We've tracked down what's causing this and have opened two PRs with potential fixes:

https://github.com/zeit/next.js/pull/8081

transparently remove the usage of esm via source transforms (it's not needed in webpack)

https://github.com/zeit/next.js/pull/8082

maintain a list of broken packages and their fixes

esm doesn't play friendly with webpack, thus the error you're experiencing.
Unfortunately, it's not as simple as preferring the module key over the main key, as this breaks an unknown number of npm packages.
A notable example is node-fetch (~10 million weekly downloads), along with other packages listed in this webpack issue: https://github.com/webpack/webpack/issues/4742

I'd like to hear what @jdalton thinks of these proposals.
I like the idea of using the option that is least breaking. 4 years on from the ES2015 I just want things to finally work.

We'll hopefully have one of these two solutions merged sometime soon. In the meantime, you can use one of them via a custom next.config.js or .babelrc!

How do I fix this in next.config.js or .babelrc?

Implementing the proposal in https://github.com/zeit/next.js/pull/8082 can be done with a next.config.js like so:
module.exports = {
  webpack(config, { isServer }) {
    if (isServer) {
      config.resolve.mainFields = ['module', 'main']

      // Fix all packages that this change breaks:
      config.resolve.alias['node-fetch'] = 'node-fetch/lib/index.js'
    }
    return config
  },
}

I've omitted the Babel variant because it's trickier to pull off.
However, the Babel option is the less-breaking than https://github.com/zeit/next.js/pull/8082.

Worked for me.

It seems like the issue was resolved! Please let me know if any other problems occur, then we can re-open it immediately.

It seems like the issue was resolved! Please let me know if any other problems occur, then we can re-open it immediately.

Did one of those permanent fixes get merged?

Not yet, @ericelliott -- however, let's move this issue to the Next.js repository.
I've filed a follow-up issue here: https://github.com/zeit/next.js/issues/8113


Unable to get YouTube download link

Hello guys, have anyone saved the cropped ytb datasets in BaiduYun or somewhere else?
The download link can not be accessed now.
Could anybody give me a copy link? I would highly appreciate it :).

链接：https://pan.baidu.com/s/12q9k6oqDpi030iiwxnQhCw
提取码：sw32
百度网盘链接

Could anybody give me a copy link? I would highly appreciate it :).

Many thanks for your share.

链接：https://pan.baidu.com/s/12q9k6oqDpi030iiwxnQhCw
提取码：sw32

链接：https://pan.baidu.com/s/12q9k6oqDpi030iiwxnQhCw 提取码：sw32

Thanks for you share, but only ‘{:06d}.{:02d}.x.jpg' is in the extracteddirector, no '{:06d}.{:02d}.z.jpg'. Shouldn't it contain both?

大佬请问有没有crop的Youtube-bb数据集吗

链接：https://pan.baidu.com/s/12q9k6oqDpi030iiwxnQhCw 提取码：sw32 百度网盘链接

大佬，这个资源是官方提供的失效链接指向的资源吗

@MagnetoYuWang @HOLYlmx
Could you find the Baidu cloud disk link for the dataset? If so, could you provide a copy? Thank you very much!


Implement the SD enumeration processes

SD enumeration process working except highspeed handshaking - but I close this issue anyway


files in volumes not updated

Describe the bug
In development, we use nodemon under docker to hot-reload our code. Since the latest version 0.14.0, the files don't reload, so I have to go into the container and perform a read operation on the file: "cat index.js" so that nodemon knows there's been a change and reloads my code.
The problem doesn't exist in 0.13.0
Expected behavior
When I modify a file mounted in a volume, I wait for docker to automatically detect that there has been a change.

I'm having the same thing. I php projects, swapping branches or running composer update can result in new files not being found in the container filesystem.

@tobz-nz If you have the time, it'd be really helpful if you could put together a sample PHP project to reproduce this. Thanks!

I just want and ran cat vendor/optimistdigital/nova-sortable/src/ToolServiceProvider.php (the file that was not being read) and now its all working.
.. then swapped branch and ran composer install which installed some updated packages, and now /var/www/html/vendor/laravel/nova/src/NovaCoreServiceProvider.php cannot be found. It is there though, obv.
I'll see if I can replicate in a new project.

Spent a few minutes trying to replicate in a new project but no luck :(

Fixed for the next version, at least with the test cases I've been using (which may or may not be caused by the same underlying issue). Let me know if it still doesn't work and I'll reopen this.

Released in v0.15.1.

I was running into this issue as well. The v0.15.1 release seems to have fixed it upon initial check. Thanks Danny!


Theme.of(context).accentColor not being migrated by dart fix

After upgrading to Flutter 2.5.0, I ran dart fix to rename all the deprecated accentColor properties.
All of them were migrated correctly expect the ones that are accessed from Theme.of(context).accentColor.
I've tried adding a test case + expectation for this in the master branch and it seems to also fail.
Steps to reproduce

Inside a Flutter 2.5.0 project place Theme.of(context).accentColor somewhere to get a deprecation warning
Run dart fix lib --apply
The line is not migrated to colorScheme.secondary

It's not a big deal, since it's easy to migrate them with a "find and replace", but maybe this is an easy fix.

Hi @davidmartos96
Thanks for filing the issue
Investigation
Indeed accentColor is not picked up dart fix, while another deprecated API setEnabledSystemUIOverlays does get picked. Run the code sample below on stable or master, bundle dart, and only 1 fix suggested that is setEnabledSystemUIOverlays.
If setEnabledSystemUIOverlays is commented, there is no fix.
Reproducible code sample

complete minimal runnable code sample
import 'package:flutter/material.dart';
import 'package:flutter/services.dart';

void main() {
  SystemChrome.setEnabledSystemUIOverlays([SystemUiOverlay.top]);
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Material App',
      theme: ThemeData.dark(),
      home: Home(),
    );
  }
}

class Home extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text('Material App Bar'),
      ),
      body: Center(
        child: Container(
          color: Theme.of(context).accentColor,
          child: Text('Hello World'),
        ),
      ),
      floatingActionButton: FloatingActionButton(
        child: Icon(Icons.add),
        onPressed: () {},
      ),
    );
  }
}

Logs

logs
tahatesser@Tahas-MacBook-Pro triage % dart fix --dry-run

Computing fixes in triage (dry run)... 18.9s
Nothing to fix!
tahatesser@Tahas-MacBook-Pro triage %

stable
master

❗
❗

Check flutter doctor -v outputs for each channel below

flutter doctor -v
[✓] Flutter (Channel stable, 2.5.0, on macOS 12.0 21A5506j darwin-x64, locale en-GB)
    • Flutter version 2.5.0 at /Users/tahatesser/Code/flutter_stable
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 4cc385b4b8 (30 hours ago), 2021-09-07 23:01:49 -0700
    • Engine revision f0826da7ef
    • Dart version 2.14.0

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at /Users/tahatesser/Code/SDK
    • Platform android-31, build-tools 30.0.3
    • ANDROID_HOME = /Users/tahatesser/Code/SDK
    • Java binary at: /Users/tahatesser/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/203.7621141/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7281165)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode-beta.app/Contents/Developer
    • Xcode 13.0, Build version 13A5212g
    • CocoaPods version 1.10.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2020.3)
    • Android Studio at /Users/tahatesser/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/203.7621141/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7281165)

[✓] Android Studio (version 2020.3)
    • Android Studio at /Users/tahatesser/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/203.7621141/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7281165)

[✓] Android Studio (version 2020.3)
    • Android Studio at /Users/tahatesser/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/203.7583922/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7281165)

[✓] VS Code (version 1.60.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.26.0

[✓] Connected device (2 available)
    • macOS (desktop) • macos  • darwin-x64     • macOS 12.0 21A5506j darwin-x64
    • Chrome (web)    • chrome • web-javascript • Google Chrome 93.0.4577.63

• No issues found!

[✓] Flutter (Channel master, 2.6.0-1.0.pre.219, on macOS 12.0 21A5506j darwin-x64, locale en-GB)
    • Flutter version 2.6.0-1.0.pre.219 at /Users/tahatesser/Code/flutter_master
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 58944e6b88 (9 hours ago), 2021-09-08 19:37:34 -0700
    • Engine revision 6a3b04632b
    • Dart version 2.15.0 (build 2.15.0-82.0.dev)

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at /Users/tahatesser/Code/SDK
    • Platform android-31, build-tools 30.0.3
    • ANDROID_HOME = /Users/tahatesser/Code/SDK
    • Java binary at: /Users/tahatesser/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/203.7621141/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7281165)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 13.0)
    • Xcode at /Applications/Xcode-beta.app/Contents/Developer
    • CocoaPods version 1.10.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2020.3)
    • Android Studio at /Users/tahatesser/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/203.7621141/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7281165)

[✓] Android Studio (version 2020.3)
    • Android Studio at /Users/tahatesser/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/203.7583922/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7281165)

[✓] VS Code (version 1.60.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.26.0

[✓] Connected device (2 available)
    • macOS (desktop) • macos  • darwin-x64     • macOS 12.0 21A5506j darwin-x64
    • Chrome (web)    • chrome • web-javascript • Google Chrome 93.0.4577.63

• No issues found!

✅: No Issue ❗: Issue reproduced
Thanks for filing the issue

CC @Piinks

@bwilkerson is this a limitation of dart fix? I don't know if it can identify this case.

I don't know whether dart fix is currently able to support this. At a superficial glance it looks like it ought to be able to; the fix data for it looks right to me. I'll try to take a look at debugging it fairly soon.

Thanks for taking a look @bwilkerson I have filed https://github.com/dart-lang/sdk/issues/47173 in the dart repo with my findings as this appears to be an issue with the tool. I am going to close this  to track the issue there. Thanks for reporting @davidmartos96!

So when can we expect this fix?


netty-transport-native-epoll-VERSION-linux-x86_64.jar isn't an OSGi bundle

Hello, I've noticed that the linux-x86_64 versions of the netty-transport-native-epoll jar isn't an OSGi bundle while the regular jars are.  This seems to be the case for all versions of this jar.

@almibe - Thanks for reaching out.  Do you have any interest in submitting a PR to resolve this issue?  We love PRs :)

I like making pull requests but I haven't been able to make netty build at all.  If I check out netty and do "mvn install" shouldn't that work?

If I check out netty and do "mvn install" shouldn't that work?

Yes. If you provide more details perhaps we can help.  Make sure you are using a "recent" version of maven.  See http://netty.io/wiki/setting-up-development-environment.html for details.

@almibe did you ever manage to find time ?

@almibe ping

Sorry for some reason I didn't see the notifications on this issue.  I only ran into this issue with ratpack and since filing this issue I've started using vertx-web which doesn't use this feature of netty.  I've also started building "fat bundles" in OSGi which avoids this issue entirely so I'm no longer having this issue.  I'll close this since I can't really test it anymore and if someone else wants netty-transport-native-epoll-VERSION-linux-x86_64.jar to be an OSGi bundle they can open an issue.  Sorry for my late response.


Applying editorial changes to the distributed how-to.

Also fixes a couple of incorrect names in the code examples.
Change: 119605636

Can one of the admins verify this patch?


ValueError: Exception encountered when calling layer "max_pooling2d_1" (type MaxPooling2D).

I am trying to use Deepface in a jupyter notebook on AWS SageMaker. I use a conda_python3 env and the python version is 3.10. I install deepface from source code.
The error occurs when I use find or analyze functions. According to the error message, the problem comes from the build_model function. I highly appreciate any hints for this.
Below is the error message when use find function:
File ~/SageMaker/deepface/deepface/DeepFace.py:650, in represent(img_path, model_name, enforce_detection, detector_backend, align, normalization)
    620 """
    621 This function represents facial images as vectors. The function uses convolutional neural
    622 networks models to generate vector embeddings.
   (...)
    646         E.g. FaceNet returns 128 dimensional vector; VGG-Face returns 2622 dimensional vector.
    647 """
    648 resp_objs = []
--> 650 model = build_model(model_name)
    652 # ---------------------------------
    653 # we have run pre-process in verification. so, this can be skipped if it is coming from verify.
    654 target_size = functions.find_target_size(model_name=model_name)

File ~/SageMaker/deepface/deepface/DeepFace.py:81, in build_model(model_name)
     79 model = models.get(model_name)
     80 if model:
---> 81     model = model()
     82     model_obj[model_name] = model
     83 else:

File ~/SageMaker/deepface/deepface/basemodels/VGGFace.py:90, in loadModel(url)
     86 def loadModel(
     87     url="https://github.com/serengil/deepface_models/releases/download/v1.0/vgg_face_weights.h5",
     88 ):
---> 90     model = baseModel()
     92     # -----------------------------------
     94     home = functions.get_deepface_home()

File ~/SageMaker/deepface/deepface/basemodels/VGGFace.py:46, in baseModel()
     44 model.add(ZeroPadding2D((1, 1)))
     45 model.add(Convolution2D(128, (3, 3), activation="relu"))
---> 46 model.add(MaxPooling2D((2, 2), strides=(2, 2)))
     48 model.add(ZeroPadding2D((1, 1)))
     49 model.add(Convolution2D(256, (3, 3), activation="relu"))

File ~/anaconda3/envs/python3/lib/python3.10/site-packages/tensorflow/python/trackable/base.py:205, in no_automatic_dependency_tracking.<locals>._method_wrapper(self, *args, **kwargs)
    203 self._self_setattr_tracking = False  # pylint: disable=protected-access
    204 try:
--> 205   result = method(self, *args, **kwargs)
    206 finally:
    207   self._self_setattr_tracking = previous_value  # pylint: disable=protected-access

File ~/anaconda3/envs/python3/lib/python3.10/site-packages/keras/utils/traceback_utils.py:70, in filter_traceback.<locals>.error_handler(*args, **kwargs)
     67     filtered_tb = _process_traceback_frames(e.__traceback__)
     68     # To get the full stack trace, call:
     69     # `tf.debugging.disable_traceback_filtering()`
---> 70     raise e.with_traceback(filtered_tb) from None
     71 finally:
     72     del filtered_tb

File ~/anaconda3/envs/python3/lib/python3.10/site-packages/tensorflow/python/framework/ops.py:1973, in _create_c_op(graph, node_def, inputs, control_inputs, op_def, extract_traceback)
   1970   c_op = pywrap_tf_session.TF_FinishOperation(op_desc)
   1971 except errors.InvalidArgumentError as e:
   1972   # Convert to ValueError for backwards compatibility.
-> 1973   raise ValueError(e.message)
   1975 # Record the current Python stack trace as the creating stacktrace of this
   1976 # TF_Operation.
   1977 if extract_traceback:

ValueError: Exception encountered when calling layer "max_pooling2d_1" (type MaxPooling2D).

Negative dimension size caused by subtracting 2 from 1 for '{{node max_pooling2d_1/MaxPool}} = MaxPool[T=DT_FLOAT, data_format="NCHW", explicit_paddings=[], ksize=[1, 1, 2, 2], padding="VALID", strides=[1, 1, 2, 2]](Placeholder)' with input shapes: [?,128,112,1].

Call arguments received by layer "max_pooling2d_1" (type MaxPooling2D):
  • inputs=tf.Tensor(shape=(None, 128, 112, 1), dtype=float32)

It seems there is something wrong with VGGFace.
I notice the emotion model doesn't use VGGFace architecture. But similar error happens when I try DeepFace.build_model(model_name="Emotion").
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[7], line 1
----> 1 DeepFace.build_model(model_name="Emotion")

File ~/SageMaker/deepface/deepface/DeepFace.py:81, in build_model(model_name)
     79 model = models.get(model_name)
     80 if model:
---> 81     model = model()
     82     model_obj[model_name] = model
     83 else:

File ~/SageMaker/deepface/deepface/extendedmodels/Emotion.py:40, in loadModel(url)
     37 model = Sequential()
     39 # 1st convolution layer
---> 40 model.add(Conv2D(64, (5, 5), activation="relu", input_shape=(48, 48, 1)))
     41 model.add(MaxPooling2D(pool_size=(5, 5), strides=(2, 2)))
     43 # 2nd convolution layer

File ~/anaconda3/envs/python3/lib/python3.10/site-packages/tensorflow/python/trackable/base.py:205, in no_automatic_dependency_tracking.<locals>._method_wrapper(self, *args, **kwargs)
    203 self._self_setattr_tracking = False  # pylint: disable=protected-access
    204 try:
--> 205   result = method(self, *args, **kwargs)
    206 finally:
    207   self._self_setattr_tracking = previous_value  # pylint: disable=protected-access

File ~/anaconda3/envs/python3/lib/python3.10/site-packages/keras/utils/traceback_utils.py:70, in filter_traceback.<locals>.error_handler(*args, **kwargs)
     67     filtered_tb = _process_traceback_frames(e.__traceback__)
     68     # To get the full stack trace, call:
     69     # `tf.debugging.disable_traceback_filtering()`
---> 70     raise e.with_traceback(filtered_tb) from None
     71 finally:
     72     del filtered_tb

File ~/anaconda3/envs/python3/lib/python3.10/site-packages/keras/layers/convolutional/base_conv.py:354, in Conv.compute_output_shape(self, input_shape)
    347         return tf.TensorShape(
    348             input_shape[:batch_rank]
    349             + [self.filters]
    350             + self._spatial_output_shape(input_shape[batch_rank + 1 :])
    351         )
    353 except ValueError:
--> 354     raise ValueError(
    355         "One of the dimensions in the output is <= 0 "
    356         f"due to downsampling in {self.name}. Consider "
    357         "increasing the input size. "
    358         f"Received input shape {input_shape} which would produce "
    359         "output shape with a zero or negative value in a "
    360         "dimension."
    361     )

ValueError: One of the dimensions in the output is <= 0 due to downsampling in conv2d_4. Consider increasing the input size. Received input shape [None, 48, 48, 1] which would produce output shape with a zero or negative value in a dimension.

Here are the package versions:
deepface                      0.0.79      /home/ec2-user/SageMaker/deepface
keras                         2.12.0
tensorflow                    2.12.0
tensorflow-estimator          2.12.0
tensorflow-io-gcs-filesystem  0.32.0

Hi, I solved the problem.
It is due to the data format setting for tf.keras.
Anyone facing similar error could try to add the following line to  VGGFace.py  and other related scripts.
tf.keras.backend.set_image_data_format('channels_last')


Feature Request: Configuration parameter followSslRedirects is misleading its real purpose and should be renamed

The configuration parameter followSslRedirects is misleading from its name to its real purpose and it would be better to rename it. We have already stumbled upon over it.
A better name for it would be followProtocolRedirects like the method parameter is already called, because its used inside the RetryFollowUpInterceptor to check if a scheme has changed between a call and a redirect.
boolean sameScheme = url.scheme().equals(userResponse.request().url().scheme());
if(!sameScheme && !this.client.followSslRedirects()) {
    return null;
}

A rename should be done because of the following reasons:

followSslRedirects indicates that if you set it to true and followRedirects to false, that okHttp will only follow SSL redirects but redirects are disabled completely
Setting followSslRedirects to false, indicates that redirects from https to http should be valid, but are dropped internally
Even the parameter of followSslRedirects is already followProtocolRedirects which indicates the real purpose

I think we fix this with documentation rather than with API changes.
And we should consider getting close to turning plaintext off by default!


Disentangling Dask between Grids and Fields

When looking on recent discussions, it would be good to access NetCDF files that contain the mesh grid separately from the data NetCDF files, to make it possible to chunk one and the same file differently.
In the current form, all fields' chunk sizes of the same shared grid need to be the same, as well as the chunk sizes for the mesh of a grid (which is loaded with the first field of a fieldset?). Disentangling that would make it possibly easier in the post-summer times to make this lazy-loading more adaptable.
Thanks to @CKehl for bringing this up

This is now fixed in #932


feat: Enable feature to call an action after a property update

Description

Add a feature to call a redux action after a property update.
Along with modify and remove lists, batchUpdateProproperty action can also include postUpdateActions
postUpdateActions is an array of ReduxActionType
The default payload included in postUpdateActions is widgetId.

Fixes #17862
Type of change

New feature (non-breaking change which adds functionality in codebase)
Internal API Change

How Has This Been Tested?

There are no functionalities added here, hence, all existing cypress tests must pass.

Test Plan

NA

Issues raised during DP testing

NA

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[NA] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[NA] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes

/ok-to-test sha=fb847fc

@marks0351 This may work if we're only updating the redux store, however, once redux-sagas get involved, we might find this to not work as expected. So, I'll for now consider the original issue reported by @rahulramesha and make this into a single action for every batch update property action.

/ok-to-test sha=6ce9a25

/ok-to-test sha=e8647ea

/ok-to-test sha=0dee106

/ok-to-test sha=45d3434

/ok-to-test sha=0969206

/ok-to-test sha=9ced65d

/ok-to-test sha=ad9d8d2

/ok-to-test sha=c72b7be

/ok-to-test sha=71cb340


change CocoaPods version to 1.7.0

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


[MacOS] Exclude release candidates from XCODE_DEVELOPER_DIR variables

Description
RC versions of Xcode should be excluded from XCODE_DEVELOPER_DIR variables
Related issue:
Check list

[ ] Related issue / work item is attached
[ ] Tests are written (if applicable)
[ ] Documentation is updated (if applicable)
[ ] Changes are tested and related VM images are successfully generated

/azp run macos1013, macos1014, macos1015, macos110

/azp run macos1013, macos1014, macos1015, macos110


Documenter tour

it would be useful to provide a tour of the documenter website for new users
e.g. using http://bootstraptour.com/

closing with https://github.com/FAIRsharing/JSONschema-documenter/commit/e359f32e0e1ee1831b045abb475c6ee780f14fea


Is zero-stride VertexDecl legal?  (for constant vertex attributes)

Hello —
I'm porting from an OpenGL engine where I make some use of constant vertex attributes as a convention.  For example, a flat object might have the same normal for all vertices while other attributes vary.
Is it legal within BGFX to create a VertexBuffer declaration with zero stride in order to hold these vertex constants?  At a glance, it looks like this would create a logic error with VertexDecl::getSize, which assumes stride and element size are equal — but none of BGFX's code seems to reference this function (at least under Windows).

No, your step/stride for attributes has to be the same for all attributes, and you can't specify different step/stride for attributes individually, even if you use multiple vertex streams.
You should use uniforms for cases where whole object gets the same value for all vertices.

you can't specify different step/stride for attributes individually, even if you use multiple vertex streams.

Wait, so I can't use a stride-12 steam of vertex positions with a stride-20 stream of colors and texture coordinates?  Does the API documentation mention this limitation?  Which rendering APIs impose this constraint?

You can have different strides with multiple streams. Number of vertices inside stream should match. Stride is used to calculate number of vertices from buffer size.


preserve bond index tags in replaceBond!

this PR fixes a bug where the tags of the bond index of an MPS were not preserved by replaceBond!.

Codecov Report

Merging #81 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master      #81      +/-   ##
==========================================
+ Coverage   93.21%   93.22%   +<.01%
==========================================
  Files          26       26
  Lines        2417     2420       +3
==========================================
+ Hits         2253     2256       +3
  Misses        164      164

Impacted Files
Coverage Δ

src/mps/mps.jl
100% <100%> (ø)
:arrow_up:

src/itensor.jl
96.2% <100%> (+0.02%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ed2a236...23e9639. Read the comment docs.

Looks good, thanks!


[DistributionBundle] Fix real vendor dir , reading from composer.json file

Q
A

Bug fix?
[yes]

New feature?
[no]

BC breaks?
[no]

Deprecations?
[no]

Tests pass?
[-]

Fixed tickets
#184

License
MIT

Doc PR
[]

Couldnt find the test, I tested manually replacing the SymfonyRequirements in a 2.6 distribution.
It's my first PR, so, i hope this can help. Thanks.

This one has just been reverted until there is a fix.


[Xamarin.Android.Build.Tasks] Reduce BuildDependsOn duplication

I was looking to extend the BuildDependsOn target ordering used to
structure application and library builds and noticed a fair amount of
duplication that may benefit from some consolidation.
A new AndroidBuildDependsOn property has been added to include the
core targets that all builds depend on.
The  _CheckNonIdealBindingConfigurations target has been merged with
_CheckNonIdealConfigurations.
The _CheckProjectItems target will now run for both app and library
projects, as the corresponding CheckProjectItems task is already
conditional based on the value of  $(AndroidApplication).
The _AddAndroidDefines target has been removed in favor of the more
widely used AndroidPrepareForBuild target, both of which are empty
targets that depend on $(_OnResolveMonoAndroidSdks).
The _RemoveLegacyDesigner target has been removed from the library
specific target list, it should already run for each project type as part
of the UpdateGeneratedFiles target.

Ok, I think the changes look right after another review. It doesn't look like any targets have changed (run for different projects), just refactoring.
Should we add the new public properties to this doc?

https://github.com/dotnet/android/blob/main/Documentation/docs-mobile/building-apps/build-properties.md

Sorry I'm not sure what we'd want to document here, the new AndroidBuildDependsOn target list is internal to our build process, and I don't think anything public is changing that needs documentation

Oh, should we prefix $(_AndroidBuildDependsOn) with an underscore, if customers aren't meant to use it?

Oh, should we prefix $(_AndroidBuildDependsOn) with an underscore, if customers aren't meant to use it?

I'm not entirely sure what the right conventions are, I was trying to follow suit with $(BuildDependsOn). I guess customers could try to use $(AndroidBuildDependsOn) in a similar way if they wanted to extend the build? Maybe I should add a comment about it to https://github.com/dotnet/android/blob/main/Documentation/guides/MSBuildBestPractices.md#should-i-use-beforetargets-or-aftertargets?

$(BuildDependsOn) came from MSBuild, and they documented it as a public thing:

https://learn.microsoft.com/en-us/visualstudio/msbuild/how-to-extend-the-visual-studio-build-process?view=vs-2022#extend-the-dependson-properties

We could do that for ours, or just put _ as a signal that it's internal.

$(BuildDependsOn) came from MSBuild, and they documented it as a public thing:

https://learn.microsoft.com/en-us/visualstudio/msbuild/how-to-extend-the-visual-studio-build-process?view=vs-2022#extend-the-dependson-properties

We could do that for ours, or just put _ as a signal that it's internal.

After some more testing it looks like both $(BuildDependsOn) and $(_AndroidBuildDependsOn) can't easily be overridden in a customers project file due to our import ordering, so I've added the _ prefix to our new property.


CUDA_ERROR_ILLEGAL_ADDRESS: an illegal memory access was encountered.

Please make sure that this is a bug. As per our
GitHub Policy,
we only address code/doc bugs, performance issues, feature requests and
build/installation issues on GitHub. tag:bug_template
This is an urgent issue! It has caused a production problem!
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux (Amazon Sagemaker)
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
TensorFlow installed from (source or binary): binary
TensorFlow version (use command below): latest: 2.5
Python version: 3.8
Bazel version (if compiling from source):
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version: 11.020/8.202
GPU model and memory: Tesla T4/13.8 GB

You can collect some of this information using our environment capture
script
You can also obtain the TensorFlow version with:

TF 1.0: python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"
TF 2.0: python -c "import tensorflow as tf; print(tf.version.GIT_VERSION, tf.version.VERSION)"

Describe the current behavior
I updated my model and when I train, I get the following error:
`
2021-07-11T23:08:22.514-04:00	2021-07-12 03:08:22.053193: W tensorflow/core/grappler/costs/op_level_cost_estimator.cc:689] Error in PredictCost() for the op: op: "Softmax" attr { key: "T" value { type: DT_FLOAT } } inputs { dtype: DT_FLOAT shape { unknown_rank: true } } device { type: "GPU" vendor: "NVIDIA" model: "Tesla T4" frequency: 1590 num_cores: 40 environment { key: "architecture" value: "7.5" } environment { key: "cuda" value: "11020" } environment { key: "cudnn" value: "8100" } num_registers: 65536 l1_cache_size: 24576 l2_cache_size: 4194304 shared_memory_size_per_multiprocessor: 65536 memory_size:<PHONE_NUMBER>0 bandwidth: 320064000 } outputs { dtype: DT_FLOAT shape { unknown_rank: true } }
2021-07-11T23:08:22.514-04:00	2021-07-12 03:08:22.053330: W tensorflow/core/grappler/costs/op_level_cost_estimator.cc:689] Error in PredictCost() for the op: op: "Softmax" attr { key: "T" value { type: DT_FLOAT } } inputs { dtype: DT_FLOAT shape { unknown_rank: true } } device { type: "GPU" vendor: "NVIDIA" model: "Tesla T4" frequency: 1590 num_cores: 40 environment { key: "architecture" value: "7.5" } environment { key: "cuda" value: "11020" } environment { key: "cudnn" value: "8100" } num_registers: 65536 l1_cache_size: 24576 l2_cache_size: 4194304 shared_memory_size_per_multiprocessor: 65536 memory_size:<PHONE_NUMBER>0 bandwidth: 320064000 } outputs { dtype: DT_FLOAT shape { unknown_rank: true } }
2021-07-11T23:08:32.516-04:00	2021-07-12 03:08:32.060022: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudnn.so.8
2021-07-11T23:08:34.517-04:00	2021-07-12 03:08:34.183639: I tensorflow/stream_executor/cuda/cuda_dnn.cc:359] Loaded cuDNN version 8202
2021-07-11T23:08:36.517-04:00	2021-07-12 03:08:36.256607: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcublas.so.11
2021-07-11T23:08:38.518-04:00	2021-07-12 03:08:38.161603: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcublasLt.so.11
2021-07-11T23:08:50.608-04:00
2021-07-12 03:08:50.058288: E tensorflow/stream_executor/cuda/cuda_event.cc:29] Error polling for event status: failed to query event: CUDA_ERROR_ILLEGAL_ADDRESS: an illegal memory access was encountered
2021-07-12 03:08:50.058288: E tensorflow/stream_executor/cuda/cuda_event.cc:29] Error polling for event status: failed to query event: CUDA_ERROR_ILLEGAL_ADDRESS: an illegal memory access was encountered`
Describe the expected behavior
Contributing

Do you want to contribute a PR? (yes/no):
Briefly describe your candidate solution(if contributing):

Standalone code to reproduce the issue
Provide a reproducible test case that is the bare minimum necessary to generate
the problem. If possible, please share a link to Colab/Jupyter/any notebook.
Other info / logs Include any logs or source code that would be helpful to
diagnose the problem. If including tracebacks, please include the full
traceback. Large logs and files should be attached.
LOGS:
logs.txt

@nectario ,
Can you please try installing TensorFlow v2.5 with CUDA 11.2 and cuDNN 8.1 and check if you are facing the same error.For more information please take a look at the tested build configurations. Thanks!

@nectario ,
Can you please try installing TensorFlow v2.5 with CUDA 11.2 and cuDNN 8.1 and check if you are facing the same error.For more information please take a look at the tested build configurations. Thanks!

Thank you. Running on my my local machine, where I have these versions, it works fine. But the real issue is that I am using Amazon Sagemaker and that build is fixed. The Docker image I am pulling is tensorflow/tensorflow:latest-gpu-jupyter
I also tried pulling the Nvidia image of tensorflow from nvcr.io/nvidia/tensorflow:21.06-tf2-py3
but I still have the same problem. This happens with the specific deep learning architecture I am using. My older architecture does not have this problem.

This issue has completely derailed our production release. I hope it's given some urgency.

Is there a tensorflow docker image where I can specify the CUDA/CUDNN version specifically?
From my Dockerfile I am calling:
FROM tensorflow/tensorflow:latest-gpu-jupyter

From the logs, I see it's loading CUDNN version 8.2:
2021-07-13 21:21:35.642481: I tensorflow/stream_executor/cuda/cuda_dnn.cc:359] Loaded cuDNN version 8202

This error also happens with the tf-nightly image.

I got it to work when I use the tensorflow Docker image for TF 2.4.2:
tensorflow/tensorflow:2.4.2-gpu-jupyter
So it is clearly a problem with later versions of CUDA/CUDNN!

This is the Google Colab with the code that triggers this error. If you run this here, it will not happen in the colab notebook as it happens mainly in the Docker image tensorflow/tensorflow:latest-gpu-jupyter
Code:
https://colab.research.google.com/drive/1E_IZ3zmPrQHbuiYl47gvL591hHfS-VkN#scrollTo=ts7l0877A0_d

Is there any update on this? I really would like to use version 2.6, but cannot.

Hi @nectario,
Can you try running with CUDA_LAUNCH_BLOCKING=1 in the environment & share the logs?  That should isolate the error to the actual GPU operation that caused it.
Also: do you have a reproducer that does not depend on separate data files, like ./y_val_two_week_return.pkl?

Hi @sanjoy
Sorry for my late response. I just came back from vacation. I did as you said, but surprisingly, the issue is not reproduceable now. I tried running it in the different environments in AWS and also my local machine, where it was happening, and it does not happen now. The version of tensorflow I tested with  was 2.6. Fyi, when this first happened, it was under 2.5. Anyway, I do remember testing also with 2.6 back then, and it was happening when 2.6 first came out.
If you want me to run it with 2.5 with the above setting, let me know.

@sanjoy I take it back. This error did manage to happen again. Surprisingly it happened when I removed the above flag CUDA_LAUNCH_BLOCKING=1. From my 250 model training 13 succeeded but the rest all failed with that error. Unfortunately it was without the flag. Will run it  with it and will attach the logs.

@sanjoy Now that I think about it, the day before yesterday I trained my 250 models with the flag on and the error did not happen. Although training seemed a little bit slower than usual. And today I trained with the flag removed, and 200+ model trainings failed.

@sanjoy this flag helped me as well, although I only found it after 1 week of head-scratching.
Config:
Ubuntu 20.04.3 LTS
Docker 20.10.8
Image: tensorflow/tensorflow:2.6.0-gpu-jupyter
GPU: Telsa P40 (Azure VM)
Drivers: 470.57.02
CUDA: 11.4

This problem continues to happen and it only stops happening if I include the env flag: CUDA_LAUNCH_BLOCKING=1

This problem continues to happen and it only stops happening if I include the env flag: CUDA_LAUNCH_BLOCKING=1

Unfortunately this means this is a heisenbug. :(
Running with CUDA_LAUNCH_BLOCKING=1 "for real" is not a good idea since it affects performance, as you indicated in https://github.com/tensorflow/tensorflow/issues/50735#issuecomment-917314918.
Will you be able to extract a minimal reproducer that shows the issue?  Ideally something that only has a few ops?

I just emailed you a file with the code that reproduces this issue on a Linux box.

This problem continues to happen and it only stops happening if I include the env flag: CUDA_LAUNCH_BLOCKING=1

Unfortunately this means this is a heisenbug. :(
Running with CUDA_LAUNCH_BLOCKING=1 "for real" is not a good idea since it affects performance, as you indicated in #50735 (comment).
Will you be able to extract a minimal reproducer that shows the issue? Ideally something that only has a few ops?

I get the same error on AWS instances running in my EKS cluster.
My environment:

instance type: p2.xlarge (having K80 GPUs)
architecture: linux (amd64)
AMI: 0dec13e4e8336258b (Amazon EKS optimized accelerated Amazon Linux AMI, it is based on Amazon Linux 2)
Kernel version: 5.4.188-104.359.amzn2.x86_64
Tensorflow 2.8.0
nvidia-smi output: NVIDIA-SMI 470.57.02    Driver Version: 470.57.02    CUDA Version: 11.4
cuDNN version 8400
container runtime: docker 20.10.13
kubelet version: v1.22.6-eks

Addendum: I just tested the same container image on a p3.2xlarge instance and the error did not occur. To confirm this I switched back to p2 and the error reappeared...

Addendum: I just tested the same container image on a p3.2xlarge instance and the error did not occur. To confirm this I switched back to p2 and the error reappeared...

Can you provide the information on the environment in which it did work (the p3.2xlarge)? Information like what image is the container based on and/or what CUDA/tensorflow versions you have.

Addendum: I just tested the same container image on a p3.2xlarge instance and the error did not occur. To confirm this I switched back to p2 and the error reappeared...

@rsnitsch
Can you provide the information on the environment in which it did work (p3.2xlarge)? Information like what image is the container based on and/or what CUDA/tensorflow versions you have.

I described the environment in detail in this comment above: https://github.com/tensorflow/tensorflow/issues/50735#issuecomment-1118669114

This might be #60192, which shows that since TF 2.5, masked cuDNN GRU/LSTM operations randomly crash and also give incorrect results (when they do not crash). The culprit seems to be (at least to me) the cuDNN 8.1 codepath; when cuDNN 8.0 or older is used, or when no masking is used (so not ragged tensors nor masked elements), everything works fine.

Hi @nectario ,
Can you confirm whether this issue is duplicate of 60192 as mentioned in above comment ? If so we can close it here and track the issue there as it is active there.
If not can you please check and confirm with latest versions ? If not we can escalate this issue to concerned. Thanks!

@sanjoy Now that I think about it, the day before yesterday I trained my 250 models with the flag on and the error did not happen. Although training seemed a little bit slower than usual. And today I trained with the flag removed, and 200+ model trainings failed.

Hi @nectario ,
I also faced the same issue. But I fix this by giving proper CUDA version for Tensorflow.
I am using tensorflow==2.7.0 and CUDA 11.2
I think this will help.
Thank You.

Hi all, the original problem causing cuDNN implementation of masked LSTM/GRU to fail with the CUDA_ERROR_ILLEGAL_ADDRESS on TF 2.5+ has been solved in cuDNN 8.9.2. So if you upgrade cuDNN to this version (you can use official TF build, even if it is build for cuDNN 8.2, the CUDA/cuDNN are backward compatible within the same major version), it should be fine -- see #60192.

I did a very small mistake , I have opened the python kernel instead of pytorch/tensorflow kernel in JupyterLab and got the same error.


Is it possible to use this task on a self-hosted agent running Windows 10?

The description of this task is clear, "run a Bash script on macOS, Linux, or Windows."  What setup is required on the build agent to run a Bash script on a self-hosted agent running Windows 10 Pro?  I've tried enabling WSL, and installing Ubuntu from the Microsoft Store.  When I attempt to run my bash script, I see this error message:  "Windows Subsystem for Linux has no installed distributions."

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 07e70665-a794-82fd-287b-65a747488efa
Version Independent ID: 0c34f5c3-bc80-061d-cf7a-f30c224ce3b1
Content: Bash task - Azure Pipelines
Content Source: docs/pipelines/tasks/utility/bash.md
Product: devops
Technology: devops-cicd
GitHub Login: @vtbassmatt
Microsoft Alias: macoope

Then the documentation needs to be changed to indicate that this Task will only run on Windows if you are using a Microsoft-hosted Windows agent.

@vtbassmatt Can you guide how to configure the agent to run pls? I got the same issue but cant resolve it

@vtbassmatt what's the reason for closing this ? I'm searching for resolution to this and all github issues are closed without any resolution.

2022 and still looking for a solution. Anyone can help, please?

@jbizcloud have a look here https://github.com/JayDoubleu/ado-virtual-environments
It's a set of pipelines which allow you to essentially "1-1 MS Hosted agents" as your own private images which you can use with private agents.


[Feature] Allow setting of batch_size as a top-level property of a model

Housekeeping

[X] I am a maintainer of dbt-core

Short description
In #10624 we're building a new type of incremental model strategy, microbatch. To support that we need a concept an batch_size. The batch_size is essentially how "big" the incrementing of the model is. It should support the values day, month, and year. The effect in practice is that if your batch_size is day then when the event_start_time is generated in #10636, we can properly calculate what the "start" of a batch should be. Additionally, the lookback which is introduced in #10662 will essentially offset the event_start_time by an integer multiple of the batch_size
If defining it in the model sql it'd look like
-- my_model.sql
{{ config(
     materialization='incremental',
     incremental_strategy='microbatch',
     event_time='my_time_field',
     batch_size='day', -- supported values: day, month, year
   )
}}
...

If defining it in the model yaml it'd look like:
models:
  - name: my_model
     config:
       event_time: my_time_field
       incremental_strategy: microbatch
       batch_size: day # supported values: day, month, year

Acceptance criteria

batch_size can be set as day, month, or year for an incremental microbatch model

Suggested Tests

the batch size makes it into the python representation of the model

Impact to Other Teams
Cloud artifacts, new property in the config of models
Will backports be required?
No
Context
No response

Resolved by #10594


Question about Multi-Objective Optimization

Hey Guys,
Thanks so much for making this library accessible, it is a great help.
I had a question about your Multi-Objective Optimization solver.
I am trying to set up a scenario like this.
Say that you want to make an batch of Pudding.
You want the batch of Pudding to have a yield of 100g.
In your pudding you have the following three ingredients:
-chocolate { fat: 5g/100g, carbs: 0g, protein: 0g, chocolate: 1}

milk { fat: 0, carbs: 10g/100g, protein: 0, milk: 1}
-eggs  { fat: 0, carbs: 0g, protein: 13g/100g, eggs: 1}

Here are our constraints:

We want to make 100g of pudding.
We want the pudding to be 30% fat, 10%protein and 25% carb

How would we set up our model in such a way that allows you to enter a contraint that chocolate+ milk + eggs equals 100g?
const model = {
    "optimize": {
      // Do we need to add optimize here? If we don't what is the outcome?
    },
    "constraints": {
        'carb': { "equal": 25 },
        'fat': { "equal": 30 },
        protein: { "equal": 10 },
        We want ... egg + chocolate + milk = 100
    },
    "variables": {
        "chocolate":  { fat: 5g/100g, carbs: 0g, protein: 0g, chocolate: 1},
        "milk": { fat: 0, carbs: 10g/100g, protein: 0, milk: 1},
        "egg": { fat: 0, carbs: 0g, protein: 13g/100g, eggs: 1}
    }
};

Note this example is probably not feasible value wise (as i just made it up) but i'm just curious about how to add the total value constraints of the 3 variables. I'm also curious if there is a way to constrain a variable using a min AND a mix?
THanks a TONNNNN.
Andrew

I had had very similar problem, here it is my solution:
let model = {
        "optimize": "price",
        "opType": "min",
        "constraints": {
            "mais": {"min": 30, "max": 80},
            "fieno": {"min": 2, "max": 50},
            "soia": {"min": 1, "max": 5},
            "girasole": {"min": 2, "max": 3},
            "colza": {"min": 0.1, "max": 3},
            "avena": {"min": 0.1, "max": 3},
            "calciocarb": {"min": 0.01, "max": 3},
            "tot": {"equal": 100},
        },
        "variables": {
            "mais": {
                "price": 1,
                "mais": 1,
                "tot": 1,
            },
            "fieno": {
                "price": 1,
                "fieno": 1,
                "tot": 1,
            },
            "soia": {
                "price": 1,
                "soia": 1,
                "tot": 1,
            },
            "girasole": {
                "price": 1,
                "girasole": 1,
                "tot": 1,
            },
            "colza": {
                "price": 1,
                "colza": 1,
                "tot": 1,
            },
            "avena": {
                "price": 1,
                "avena": 1,
                "tot": 1,
            },
            "calciocarb": {
                "price": 1,
                "calciocarb": 1,
                "tot": 1,
            },
        }
    };

Sorry for the late response.
@elia-c nailed it.
Just add a "fake" attribute to each thing with a value of 1; and in the constraints, set that attribute equal to 100 (or whatever you want the total amount to be).
const model = {
    "optimize": "eggs",
    "opType": "max",
    "constraints": {
       'carb': { "max": 13.075413 },
       'fat': { "equal": 12.486736 },
       'protein': { "equal": 9.846738 },
       'total_weight': {"equal": 100}
    },
    "variables": {
        "chocolate":  { fat: 0.3057, carbs: 0.5801, protein: 0.0851, chocolate: 1, total_weight: 1},
        "milk": { fat: 0.0325, carbs: 0.048, protein: 0.0315, milk: 1, total_weight: 1},
        "egg": { fat: 0.0951, carbs: 0.0072, protein: 0.1256, eggs: 1, total_weight: 1}
    }
};


Code cleanup

Codecov Report

Merging #9 (3c176bb) into master (8bb10bf) will decrease coverage by 0.22%.
The diff coverage is 28.57%.

@@             Coverage Diff              @@
##             master       #9      +/-   ##
============================================
- Coverage     82.75%   82.53%   -0.23%
+ Complexity      136      134       -2
============================================
  Files             8        8
  Lines           661      664       +3
  Branches         87       87
============================================
+ Hits            547      548       +1
- Misses          100      102       +2
  Partials         14       14

Impacted Files
Coverage Δ

...ablesaw/io/parquet/TablesawParquetReadOptions.java
75.34% <ø> (ø)

...blesaw/io/parquet/TablesawParquetWriteOptions.java
100.00% <ø> (ø)

...h/tablesaw/io/parquet/TablesawRecordConverter.java
73.43% <16.66%> (-0.38%)
:arrow_down:

.../tech/tablesaw/io/parquet/TablesawReadSupport.java
87.14% <100.00%> (ø)

...tech/tablesaw/io/parquet/TablesawWriteSupport.java
96.77% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8bb10bf...3c176bb. Read the comment docs.


fix: redirect to correct prefix according to drop

Thank you for your contribution to the KodaDot - One Stop Shop for Polkadot NFTs.
👇 __ Let's make a quick check before the contribution.
PR Type

[x] Bugfix

Needs QA check

@kodadot/qa-guild please review

Context

[x] Closes #8716

auto-redirect
/ahk/drops/christmascity -> /ahp/drops/christmascity
Did your issue had any of the "$" label on it?

[x] My DOT address: Payout

Screenshot 📸

[ ] My fix has changed UI

Copilot Summary
copilot:summary
copilot:poem

wait not sure if? i think there was two test cases and one was on ahk iirc?

wait not sure if? i think there was two test cases and one was on ahk iirc?

NO : ) It's a bug.
The drop only belong to a single prefix

Any visible drop is on ahp now


Update to tempo2 v2024.12.1

Checklist

[x] Used a personal fork of the feedstock to propose changes
[x] Reset the build number to 0 (if the version changed)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

Does what the title says!

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

@conda-forge-admin, please rerender

Just trying another re-render in case it fixes the osx build

Hi! This is the friendly automated conda-forge-webservice.
I tried to rerender for you, but it looks like there was nothing to do.
This message was generated by GitHub Actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/12265873137. Examine the logs at this URL for more detail.

Was just looking at this.  The OSX build is failing on #include <string>.  I think the CXXFLAGS are getting broken somehow.  conda claims this:
+CXXFLAGS=-march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fstack-protector-strong -O2 -pipe -stdlib=libc++ -fvisibility-inlines-hidden -fmessage-length=0 -fPIE -isystem $PREFIX/include -fdebug-prefix-map=$SRC_DIR=/usr/local/src/conda/tempo2-2024.12.1 -fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix

But tempo2 configure reports this:
configure: C++ compiler is x86_64-apple-darwin13.4.0-clang++ -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fstack-protector-strong -O2 -pipe -fPIE -isystem $PREFIX/include -fdebug-prefix-map=$SRC_DIR=/usr/local/src/conda/tempo2-2024.12.1 -fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix -Wno-error -fopenmp -DHAVE_OPENMP

In particular the -stdlib=libc++ is gone which is probably the issue.  Not sure why and/or what changed, but posting this info in case anyone else wants to pick it up.  If not I'll get back to it when possible..

If this release is required for anything urgent, I'm happy for this to be merged with the osx build turned off, i.e., change:
skip: True  # [win]

to
skip: True  # [win or osx]

in the recipe. A new build could then be done later if/when we can fix the OSX issue.

OK thanks. I have one more fix to try.  If this doesn't work maybe we'll skip OSX for now.


[Improvement]: OOTB grafana dashbord json

Search before asking

[X] I have searched in the issues and found no similar issues.

What would you like to be improved?
We got an excellent feature of prometheus metrics exporter in version 0.7, and further more, we can give user more convenient tools for out-of-the-box monitor dashboard such as template of grafana.
How should we improve?
As mentioned below, maybe a grafana json file or template id is a good ways for user to build a dashboard.
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Subtasks
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

@jiamin13579 Would you plz work on it?

@jiamin13579 Would you plz work on it?

Glad to have it done


Correct mis-wording

@carehart : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Looks good; thank you.


[ramda] add types for chain function

https://ramdajs.com/docs/#chain

Could you add a test to cover this addition?

Tests added.


option_channel_type was negotiated but channel_type is missing

Background
Cannot open a channel to Eclair
Your environment

lnd v0.14.1-beta
eclair https://github.com/ACINQ/eclair/pull/2063

Steps to reproduce

Connect Eclair to LND
LND initiates channel open via lncli openchannel

Expected behaviour
Channel opened.
Actual behaviour
Error: option_channel_type was negotiated but channel_type is missing
Investigation
It seems that lnd is silently not sending the channel type tlv when OpenChannelRequest.CommitmentType isn't set. The lncli default is to not set the channel type.
https://github.com/lightningnetwork/lnd/blob/c75c3bd847afa86030697960990b4aaa88256efc/rpcserver.go#L1995

Yes

Seems they've changed behavior (in master?) since their last release.

It seems that lnd is silently not sending the channel type tlv when OpenChannelRequest.CommitmentType isn't set.

Thing is, we don't necessarily know if the other side actually supports the feature when a user goes to open a channel type. So we opt to not use explicit funding if the user doesn't select a type manually.
As the spec is written right now, it's technically ok for them to always require it (the "MAY" wording), but there're still some other ambiguous cases, like: we don't set it, but they do (in accept_channel) now we both go to implicit. So it's up to an implementation today if they always want it or not.

Ok, cleared things up here with some of the other BOLT-ers, we can both ensure that users don't get downgraded, but also maintain compat across all levels. We'll soon have some CI up to run cdecker's old cross-impl tests to make sure this gets caught during the rc process (ideally we only target the last major release, since the master branch can at times be a moving target).


Document class is not present in the package

Is your feature request related to a problem? Please describe.
I am writing an Azure Function with CosmosDBTrigger. The function reads the documents that are of different types, with different fields, so I can't just do a binding to a certain class type, like this
[CosmosDBTrigger(
            databaseName: "ToDoList",
            collectionName: "Items",
            ConnectionStringSetting = "string",
            LeaseCollectionName = "leases",
            CreateLeaseCollectionIfNotExists = true)] IReadOnlyList<ClassType> input

I could just bind it to a IReadOnlyList<object> input
BUT
I really liked the support for such case in version 2 of the package, where there was a Document class, which would give me access to id property and also GetPropertyValue method
I can install this package:
https://www.nuget.org/packages/Microsoft.Azure.DocumentDB.Core/#readme-body-tab
And use the Document class, the only thing that is bothering me is that the package is marked as deprecated now
Describe the solution you'd like
Would it be possible to include the Document class into the v3 library and if not, what was the reason of not transitioning it into the v3 library? Is there another alternative in v3 library that I missed?
Describe alternatives you've considered
I could just use the object for the input, but in my case it would cause me to Deserialize the same object twice. I could provide an explanation as to why is it that way, but not sure if you guys want to know it

For V2 to V3 SDK migration see: https://docs.microsoft.com/en-us/azure/cosmos-db/sql/migrate-dotnet-v3?tabs=dotnet-v3#classes-replaced-on-net-v3-sdk (particularly the area denoted in this link, which would answer your question).
There is no plan to include Document in the V3 SDK.
Azure Functions is outside of the scope of this repo. Please reach out to https://github.com/Azure/azure-webjobs-sdk-extensions/ for issues on that area.
As a side note, the CosmosDBTrigger lets you only bind to Document on the 3.X version (your code snippet looks like 3.X). The 4.X package that uses this repo's SDK, let's you use any <T> but your code snippet does not point to that version.

Do you know if there's another way to get the document in the input and be able to read it's properties dynamically?
Instead of using object?

The linked documentation explains the alternatives:  https://docs.microsoft.com/en-us/azure/cosmos-db/sql/migrate-dotnet-v3?tabs=dotnet-v3#classes-replaced-on-net-v3-sdk

Because the .NET v3 SDK allows users to configure a custom serialization engine, there's no direct replacement for the Document type. When using Newtonsoft.Json (default serialization engine), JObject can be used to achieve the same functionality. When using a different serialization engine, you can use its base json document type (for example, JsonDocument for System.Text.Json). The recommendation is to use a C# type that reflects the schema of your items instead of relying on generic types.

You can use JObject and it allows exactly the same operations.

Wow, can't believe I missed that part lol Thank you so much for your time!


Removed forward slashes from the tests

https://github.com/rancher/qa-tasks/issues/1578
Currently, slack reporter for recurring run fails with:
05:15:58.491  [31mERRO[0m[0114] error reporting: error with slack message post: 400 Bad Request 400: ****
The potential problem is: that it uses string manipulation to check the forward slashes. It sends the wrong parsed string to the slack API.
Example names (there are more, make sure to check names) that need updates: https://github.com/rancher/rancher/blob/release/v2.9/tests/v2/validation/auth/userretention/userretention_settings_test.go#L191-L193
Extra information about forward slashes in the test names: https://github.com/rancher/rancher/tree/main/tests#how-to-write-tests

@caliskanugur I will create backport PRs once the issue fixed with this merge.


How do you show the full size image?

@wmburke says: Previously clicking on the image would open it full size in a new browser window. Now, it doesn't seem possible to see the full size image at all.
I think this is important, especially if you're trying to see if it's the same person in two images - or get a better look at a tattoo, etc...
I would suggest another icon under the images (maybe the picture icon?) that when clicked opens it full size. This could be a browser window for now, but we should probably think about adding light box style functionality so that you could actually move through the images like you do through any web gallery.

Done. You click on the image in the details pane to see the full size image in a new tab.


Tech docker add bash

Description
Add bash in docker build.

Codecov Report

Merging #149 into master will not change coverage.
The diff coverage is n/a.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a37ab86...1badcac. Read the comment docs.


Question about quantitive studies

Thanks for you awesome work; however i am curious:

Number of video samples to compute FVD.
The train-test split of RAVDESS dataset (Since StyleFaceV needs an initial frame).
Will you guys release retrained Di-GAN, StyleGAN-V, MC-HD?

Initial frame can be sampled from the noise directly.
No. We retrain them by their default setting and it is easy to reproduce.


Add '|' and '-' in regex fixes #479

As reported in #479 flow router wasn't working with certain regex in the path. This PR allows for path definitions such as:
/blog/:action(edit|create)
Added the hyphen to also allow for:
/blog/:action(edit-page|create-page)
There might be some other URL-friendly characters that need to be catered for but this should help in most cases!

There is a new feature of path-to-regexp which does this. At the time I'm writing this it was not there.
Could you simply use that in this PR: https://github.com/pillarjs/path-to-regexp#compile-reverse-path-to-regexp

At first glance looks like pathToRegex.compile handles things differently to the custom code before?
E.g.
var pathDef = '/blog/:blogId/some/:name';
  var fields = {
   blogId: '1001',
};
var expectedPath = '/blog/1001/some';`

But with pathToRegex.compile an exception is thrown rather than a path returned due to :name being missing.
Not sure how you want to handle it as if people have relied on that functionality I guess it could constitute a breaking change?

That's because name is not an optional parameter. If path def looks like this, I think it's fine.
var pathDef = '/blog/:blogId/some/:name';

We are planning for version 4.0 and it's a pretty good time for a breaking change.
Is there any other issue other than this?

Looks like pathToRegex encodes the URL params too which was causing some of the other tests to fail.
I've updated all the tests to pass although I'm still a little new when it comes to testing so you might want to give it a once over!

Great. I know what's happening there.
See here: https://github.com/kadirahq/flow-router/blob/ssr/client/router.js#L160
I encoded in the client side. So, here are the changes we need.

Remove the client side override for path()
In the common path() function,

just don't encode params
but encode queryParams

This will work properly then.

Awesome. This is great.

Hey man love your work, big fan.
Just letting you know I tracked down a bug that basically disallowed you from using [] in route definitions. Usage caused FlowRouter#go to put some URL encoded characters in the url.
The following regExp was altered in Router.prototype.path
var regExp = /(:[\w\(\)\\\+\*\.\?\{\}\[\]-]+)+/g;


[DC-487] Add catalog export case to import-data

<!--
Hello, friend!
Remember to mention what sort of testing/verification you performed in the course of building this PR, i.e. checking functionality in the browser locally.
Also, so if a screen recording [1] and/or screenshots would be a helpful way to communicate context, please consider including some in your PR description.
Thanks!

[1] https://support.apple.com/guide/quicktime-player/record-your-screen-qtp97b08e666/mac
--!>

@pshapiro4broad yeah, exporting snapshots will be much more tricky! This is not currently called with the catalog, although I could test this by hand-rolling a URL with the right query params. I may try to do that.
@fboulnois its unclear to me what we do for unit tests rn in the UI. I would say we may want to add a test similar to what we have for other tests for this, but we need to wait for that until the button works.

I have manually tested now with both existing and new workspace, and the new code works


Unable to compile Demo - Nimble linker command exit code

Anyone able to compile this project demo on Xcode 12 ?
I followed the steps to retrieve the submodules, and compiled the frameworks targets successfully before trying to run the demo.

so I disabled TESTs and I was able to run in Device.


Links remain when text removed in editor

VSCode Version: 1.36.1 (system setup)
OS Version: Windows_NT x64 10.0.18362

Steps to Reproduce:

Add a link somewhere in your document.
Remove a line anywhere above your link, the bottom most link's 'underline' will remain where it was before the text was removed.
Sometimes the underline will blink continuously. It appears that if you save the document twice the underline is removed (only sometimes as well), otherwise the underline will be removed after a while.

Here's a GIF of the issue.

I have tested the issue in the latest insiders (1.37.0-insider (system setup)) and the error appears to still be there.

Does this issue occur when all extensions are disabled?: Yes

Possible duplicate of #48321


MySQL enum values with special characters

Hello
I'm trying to store some special values in a MySQL enum field like ' (single quote) and \ (backslash). Here is an example:
$this->table("test")->changeColumn('code', 'enum', array('values' => array('1','22','33','\'','"','\\','/',',',',,')))->update();`

Error message
[PDOException]   SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that    corresponds to your MariaDB server version for the right syntax to use near '"', '\', '/', ',', ',,') NULL COMMENT    '4' AFTERdeleted' at line 1
Query
ALTER TABLE `test` CHANGE `code` `code` ENUM('1', '22', '33', ''', '"', '\', '/', ',', ',,') NULL AFTER `deleted`

Expected query:
ALTER TABLE `test` CHANGE `code` `code` ENUM('1', '22', '33', '\'', '"', '\\', '/', ',', ',,') NULL AFTER `deleted`

To make the query working I've added a backslash before ' and \.
' => '
\ => \\
I think it's an small issue with a missing value escaping. How could I fix it?
Version: 0.6.2

Hi.
If you want to make a Pull Request, then take a look at \Phinx\Db\Adapter\MysqlAdapter::getColumnSqlDefinition() and \Phinx\Db\Adapter\SQLiteAdapter::getColumnSqlDefinition().
The iteration of the values returned by $column->getValues() need to be quoted.
$def .= "('" . implode("', '", $values) . "')";
becomes
$def .= "('" . implode("', '", $this->getConnection()->quote($values)) . "')";
and
$def .= " CHECK({$column->getName()} IN ('" . implode("', '", $values) . "'))";
becomes
$def .= " CHECK({$column->getName()} IN ('" . implode("', '", $this->getConnection()->quote($values)) . "'))";
Any unit tests that cover this code will need to be updated.

Which is obviously wrong as each value in $values needs to be quoted. This is what happens when I've worked too many hours!!!!

I try to create a PR.

Here is my PR: https://github.com/robmorgan/phinx/pull/899


ROX-12503 implement sensor's pipeline component

Description
Introducing a new sensor component that will wrap the k8s events management. This PR tackles all the refactoring on the handlers to use a queue to publish events and detector requests.
Checklist

[x] Investigated and inspected CI test results
[x] Unit test and regression tests added
~[ ] Evaluated and added CHANGELOG entry if required~
~[ ] Determined and documented upgrade steps~
~[ ] Documented user facing changes (create PR based on openshift/openshift-docs and merge into rhacs-docs)~

If any of these don't apply, please comment below.
Testing Performed

CI
Manually executing sensor-integration-test and go-unit-tests
comparing benchmarks (see)
long-running cluster with the following workload:

deploymentWorkload:
- deploymentType: Deployment
  lifecycleDuration: 30m0s
  numDeployments: 1000
  numLifecycles: 0
  numLabels: 10
  randomLabels: true
  podWorkload:
    containerWorkload:
      numImages: 0
    lifecycleDuration: 10m0s
    numContainers: 3
    numPods: 5
    processWorkload:
      alertRate: 0.001
      processInterval: 1m0s
  updateInterval: 10m0s
networkPolicyWorkload:
- lifecycleDuration: 30m0s
  numNetworkPolicies: 500
  numLifecycles: 0
  updateInterval: 10m0s
networkWorkload:
  batchSize: 100
  flowInterval: 1s
nodeWorkload:
  numNodes: 1000
rbacWorkload:
  numBindings: 1000
  numRoles: 1000
  numServiceAccounts: 1000
serviceWorkload:
  numClusterIPs: 334
  numNodePorts: 333
  numLoadBalancers: 333
matchLabels: true
numNamespaces: 10

For the record if anyone finds this PR in the future. This was first implemented in pair in this PR: #3495. All initial review comments are there.

I ran the code in this branch against the benchmark in #3656 and compared the results with the benchmark shown in the PR:
% benchstat ./old.result.txt new-pipeline.result.txt
name          old time/op  new time/op  delta
_Pipeline-12   297ms ± 7%   267ms ± 8%  -9.88%  (p=0.032 n=5+5)

The pipeline here seems to be 9.8% faster. I think it's negligible since we're only considering deployments. But at least we know it's not significantly worse.

/retest

I ran the code in this branch against the benchmark in #3656 and compared the results shown in the description:
% benchstat ./old.result.txt new-pipeline.result.txt
name          old time/op  new time/op  delta
_Pipeline-12   297ms ± 7%   267ms ± 8%  -9.88%  (p=0.032 n=5+5)

The pipeline here seems to be 9.8% faster. I think it's negligible since we're only considering deployments. But at least we know it's not significantly worse.

That's great! Deployments are the resources that should take more time/resources to process so I think still good seeing that it's slightly better 🎉

/help

/test openshift-oldest-operator-e2e-tests
/test openshift-newest-operator-e2e-tests

/test all

/test all


[ROS 2] Error messages if action server exists

If there is an action server on the ROS graph, when rqt_service_caller starts we see errors message about invalid service types.
To reproduce, start turtlesim
ros2 run turtlesim turtlesim_node

And start rqt_service_caller:
ros2 run rqt_service_caller rqt_service_caller

Example error message:
[ERROR] [1589841268.191775247] [get_service_class]: Malformed srv message_type: turtlesim/action/RotateAbsolute_GetResult
ServiceCaller.on_refresh_services_button_clicked(): could not get class turtlesim/action/RotateAbsolute_GetResult of service /turtle1/rotate_absolute/_action/get_result. Skipping it.
[ERROR] [1589841268.192278390] [get_service_class]: Malformed srv message_type: turtlesim/action/RotateAbsolute_SendGoal
ServiceCaller.on_refresh_services_button_clicked(): could not get class turtlesim/action/RotateAbsolute_SendGoal of service /turtle1/rotate_absolute/_action/send_goal. Skipping it.

I don't necessarily expect the service caller to offer support for calling these "hidden" services, but it should know enough to ignore them without producing errors.

I proposed https://github.com/ros-visualization/rqt_service_caller/pull/20 to fix this.

Closed with #20


Using "quotes" Style Breaks Styled Component

Environment
npx envinfo --system --binaries --npmPackages styled-components,babel-plugin-styled-components --markdown --clipboard

System:

OS: Linux 5.4 Linux Mint 20.1 (Ulyssa)
CPU: (8) x64 Intel(R) Core(TM) i7-4810MQ CPU @ 2.80GHz
Memory: 1.22 GB / 23.40 GB
Container: Yes
Shell: 5.0.17 - /bin/bash

Binaries:

Node: 15.10.0 - /usr/bin/node
Yarn: 1.22.5 - /usr/bin/yarn
npm: 7.3.0 - /usr/local/bin/npm

npmPackages:

babel-plugin-styled-components: latest => 1.12.0
styled-components: latest => 5.2.1

Reproduction
Add a quotes CSS property to a styled component.  For instance (not sure if every style matters, but for completeness):
const Quote = styled.div`
  background: #f9f9f9;
  border-left: 10px solid #ccc;
  margin: 1.5em 10px;
  padding: 0.5em 10px;

  &:before {
    color: #ccc;
    content: open-quote;
    font-size: 4em;
    line-height: 0.1em;
    margin-right: 0.25em;
    vertical-align: -0.4em;
  }
  &:after {
    color: #ccc;
    content: close-quote;
    font-size: 4em;
    line-height: 0.1em;
    margin-right: 0.25em;
    vertical-align: -0.4em;
  }
  quotes: "\201c" "\201d";
`;

Steps to reproduce
Try the above code.  In the Chrome debugger you will see the following for the element's code:
.cdwbly {
    null: ;
}

Now, remove that final style from the component, ie.:   quotes: "\201c" "\201d";
Magically everything now works.  Now try one more thing: using the Chrome debugger, add that exact same line directly, as a style on the element.   It works perfectly fine, without issue.
Expected Behavior
quotes is a valid CSS style.  It should not only work, but also it should not break any other styles, simply by it's existence.
Actual Behavior
Adding quotes breaks everything, making the rendered styles "null".

I'm not at a laptop right now and since I haven't got this in a CodeSandbox I can't try this out immediately, but what you have to keep in mind is that \201c is an escape code. However, it's an escape code inside a tagged template literal which also supports string escape codes, hence it'll be evaluated to something different entirely which will end up in stylis.
Now, stylis v3 isn't a huge fan of string processing and known to get some of it wrong, but there's a good chance that fixing this by writing \\201c will fix this rule. In general, specific properties shouldn't cause this but it's likely the Unicode point at fault.

\\201c fixed it, thanks so much!
If someone could stick a footnote about this in the documentation somewhere, that'd be truly awesome, but if not this issue will serve as a (lesser) form of documentation ... so I'll go ahead and close the ticket.  Thanks again!


Allowing finer grained control over random variate generation

In quite a few places in the main (non-test) code, random variates are sampled using functions in the top-level numpy.random namespace which map to the methods of a (globally shared) RandomState instance. From a reproducibility / ease of debugging perspective it would be better to instead allow explicitly passing an instance of the numpy.random.Generator class (introduced in Numpy v1.17) or the legacy numpy.random.RandomState class to functions which use (pseudo) random variates - see for example this blog post for a nice write up of why this is recommended. The current interface could be maintained by making this an optional keyword argument that defaults to a global Generator / RandomState instance if not specified, while allowing the user to explicitly control the random number generator if desired. This would also facilitate ensuring the tests are deterministic as discussed in #135.
Just to be clear as this would be quite a substantial change, I'm suggesting this more as a feature request / nice to have in future rather than as a blocker to acceptance for the JOSS review!

Thanks again for the suggestion, @matt-graham, I agree it would be good to modernize how we handle random states and seeds!