NixOS Build Broken

While following the NixOS build instructions I came across a few issues:
Building initially gave an error

error: experimental Nix feature 'nix-command' is disabled; use '--extra-experimental-features nix-command' to override

I was able to fix this by using  nix --extra-experimental-features nix-command build -f yubikey-installer.nix -o installer nixos-yubikey
The next issue was I got an error about yubioath-desktop being deprecated

error: yubioath-desktop has been deprecated by upstream in favor of yubioath-flutter

I was able to fix that by replacing yubioath-desktop with yubioath-flutter
The final issue I have is

error: builder for '/nix/store/g7pw6yjayp9qxmbh72b8pwx82j1jxq2v-hopenpgp-tools-0.23.7.drv' failed with exit code 1;
last 10 log lines:
In an equation for ‘spGetSCT’:
Patterns of type ‘Pkt’ not matched:
PKESKPkt _ _ _ _
SKESKPkt _ _ _ _
OnePassSignaturePkt _ _ _ _ _ _
SecretKeyPkt _ _
...
|
211 | spGetSCT (SignaturePkt s) = fmap fromIntegral (sigCT s)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
For full logs, run 'nix log /nix/store/g7pw6yjayp9qxmbh72b8pwx82j1jxq2v-hopenpgp-tools-0.23.7.drv'.

It looks like hopengpg-tools is broken for nix? https://github.com/NixOS/nixpkgs/blob/6f3728df507fd1683cf16e49a0078dc800330259/pkgs/development/haskell-modules/configuration-hackage2nix/broken.yaml
https://github.com/NixOS/nixpkgs/search?q=hopenpgp&type=issues
I just wanted to open this issue to see if anyone has solved this. In the meantime, I'll keep researching and if I can get it working, I'll post a PR.
Thanks for this great guide drduh! And thanks to everyone else who has contributed

It looks like the current airgapped NixOS config was based on https://github.com/dhess/nixos-yubikey (according to https://github.com/drduh/YubiKey-Guide/pull/342)
Some files of interest
https://github.com/dhess/nixos-yubikey/blob/master/nix/sources.json
https://github.com/dhess/nixos-yubikey/blob/master/lib/default.nix
https://github.com/dhess/nixos-yubikey/blob/master/default.nix
In that repo, it looks like specific versions are being targeted to prevent hopengpg from breaking?
I have no experience with NixOS, so I just wanted to document what I've found so far, in case it helps save time for someone who knows NixOS better.

It looks like the current airgapped NixOS config was based on https://github.com/dhess/nixos-yubikey (according to https://github.com/drduh/YubiKey-Guide/pull/342)
Some files of interest
https://github.com/dhess/nixos-yubikey/blob/master/nix/sources.json
https://github.com/dhess/nixos-yubikey/blob/master/lib/default.nix
https://github.com/dhess/nixos-yubikey/blob/master/default.nix
In that repo, it looks like specific versions are being targeted to prevent hopengpg from breaking?
I have no experience with NixOS, so I just wanted to document what I've found so far, in case it helps save time for someone who knows NixOS better.
Looping in @zeorin @rvl @reissmann as you three have all made nice contributions to the Nix build. Sorry to bother you, just wanted to loop you in in case you were interested

Hi,
I've managed to make it work using a dirty workaround. I did three main things:
1 - Changed the desktop package to the flutter one,
yubioath-flutter
2 - Pinned the version of the Haskel program to one that compiles properly:
`    pkgs = import (builtins.fetchTarball {
url = "https://github.com/NixOS/nixpkgs/archive/3b05df1d13c1b315cecc610a2f3180f6669442f0.tar.gz";
}) {};
  myPkg = pkgs.haskellPackages.hopenpgp-tools;`

and removed:
        # Testing         ent         (haskell.lib.justStaticExecutables haskellPackages.hopenpgp-tools)
3 - Forced the kernel of the image to be a ZFS compatible to avoid some weird license issues.
boot.kernelPackages = config.boot.zfs.package.latestCompatibleLinuxPackages;
It booted and I've managed to prepare my yubikey.
Note that this is my first week in nixos, I am not sure I am doing all as it is supposed to be done.
The modified file is below for inspection. Hope it helps.
yubikey-installer.nix.txt

A simple workaround is to use the stable nixos-22.05 channel:
nix build -f yubikey-installer.nix -o installer nixos-yubikey -I nixpkgs=channel:nixos-22.05

As noted above, you might have to temporarily enable nix-command, but that's a separate issue.
nix build --extra-experimental-features nix-command -f yubikey-installer.nix -o installer nixos-yubikey -I nixpkgs=channel:nixos-22.05

Nixos 22.05 is now deprecated (and so is 22.11 at the time of writing, long live 23.05!)
If it is desirable, I will make a PR to pin nixpkgs to the current stable version of Nixos.

Apparently neither the updated version from @nodupe of yubikey-installer.nix, nor the additional command-line options for nix build from @pkel work anymore:
[vagrant@nixbox:~]$ time nix build -f yubikey-installer.nix -o installer nixos-yubikey -I nixpkgs=channel:nixos-22.05
warning: unable to download 'https://nixos.org/channels/nixos-22.05/nixexprs.tar.xz': Peer certificate cannot be authenticated with given CA certificates (60); retrying in 332 ms
warning: unable to download 'https://nixos.org/channels/nixos-22.05/nixexprs.tar.xz': Peer certificate cannot be authenticated with given CA certificates (60); retrying in 692 ms
warning: unable to download 'https://nixos.org/channels/nixos-22.05/nixexprs.tar.xz': Peer certificate cannot be authenticated with given CA certificates (60); retrying in 1040 ms
warning: unable to download 'https://nixos.org/channels/nixos-22.05/nixexprs.tar.xz': Peer certificate cannot be authenticated with given CA certificates (60); retrying in 2352 ms
warning: Nix search path entry 'channel:nixos-22.05' cannot be downloaded, ignoring
error: anonymous function at /nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/nixos/release.nix:1:1 called with unexpected argument 'configuration', at /home/vagrant/yubikey-installer.nix:165:11

real    0m4.917s
user    0m0.075s
sys     0m0.031s

[vagrant@nixbox:~]$

Any ideas on how to get this working are welcome.

This looks like there's something wrong with your environment preventing it from verifying the certificates (or there were issues with nixos.org when you tried it that have since cleared up). The URL it complains about (https://nixos.org/channels/nixos-22.05/nixexprs.tar.xz) resolves and has a valid certificate, and the command works fine on my machine.

The certificate for nixos.org is expired:
[nix-shell:~]$ wget https://nixos.org/channels/nixos-22.05/nixexprs.tar.xz
--2023-06-28 13:56:23--  https://nixos.org/channels/nixos-22.05/nixexprs.tar.xz
Resolving nixos.org (nixos.org)... <IP_ADDRESS>, <IP_ADDRESS>, 2600:1f18:2489:8201::c8, ...
Connecting to nixos.org (nixos.org)|<IP_ADDRESS>|:443... connected.
ERROR: cannot verify nixos.org's certificate, issued by ‘CN=R3,O=Let's Encrypt,C=US’:
  Issued certificate has expired.
To connect to nixos.org insecurely, use `--no-check-certificate'.

The same command works when run directly on the machine that is hosting the Vagrant VM.
For some reason, the certificate for nixos.org cannot be updated.
Output of nix-info:
[vagrant@nixbox:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 4.14.32, NixOS, 18.03.131807.489a14add9a (Impala)`
 - multi-user?: `yes`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.0`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs`

Nixos 22.05 is now deprecated (and so is 22.11 at the time of writing, long live 23.05!)
If it is desirable, I will make a PR to pin nixpkgs to the current stable version of Nixos.

That would be very helpful, but not sure how easy that is to do.

First off thanks to all the contributers to this repo. It is a really well documented guide.
In order for nix build to complete without issues, a certain version of nixpkgs should be used. The easiest way is to use the -I flag (As mentioned by @pkel). It is also possible to update this code and use a nix flake, which has a pinned version of nixpkgs in its inputs section. If I understood correctly, this shouldn't be a problem as it is intended for a complete offline installation.
@drduh is it acceptible if someone created a pull request with the simple solution (using -I to set nixpgks version)?

In order for nix build to complete without issues, a certain version of nixpkgs should be used. The easiest way is to use the -I flag (As mentioned by @pkel). It is also possible to update this code and use a nix flake

I spent some time trying to make yubikey-installer.nix work with flakes, but to no avail: I could not find a signature similar to this one:
nix-repl> builtins.functionArgs (import <nixpkgs/nixos/release.nix>)
{ configuration = true; nixpkgs = true; stableBranch = true; supportedSystems = true; }

... when nixpkgs is imported via a flake.
I think it we need to at least specify -I  to make sure this keeps building when nixpkgs evolves, especially across releases/breaking changes. I would personally also favor flakes, since that adds full determinism, and the trade-offs, in my view, seem to be OK with the nature of this image.
@kaikikokuke do you have an example you could share that makes this work with flakes?

Hey all, :wave:
Sorry for the radio silence on this issue. I kept meaning to get round to fixing this, but my bandwidth for OSS has been really low recently. Anyway, my GPG key expired :joy: and I need to extend/renew it so this suddenly became a pressing issue.
PR is up at https://github.com/drduh/YubiKey-Guide/pull/396. Please try it out and let me know how it goes.

Apparently neither the updated version from @nodupe of yubikey-installer.nix, nor the additional command-line options for nix build from @pkel work anymore:
warning: Nix search path entry 'channel:nixos-22.05' cannot be downloaded, ignoring

Any ideas on how to get this working are welcome.

Experiencing the same with 23.05, but it's more about        at «none»:0: (source not available)

@bphd Would you be able to copy/paste your command line input together with its output here? I just ran nix-build yubikey-installer.nix --out-link installer --attr nixos-yubikey with the current yubikey-installer.nix and it built just fine.

@bphd Would you be able to copy/paste your command line input together with its output here? I just ran nix-build yubikey-installer.nix --out-link installer --attr nixos-yubikey with the current yubikey-installer.nix and it built just fine.

I've figured that I was asking 25.03 instead of 23.05, hence the problem

My attempt to build is failing with
error: cannot download optparse-applicative-<IP_ADDRESS>-r1.cabal from any mirror
error: builder for '/nix/store/ym132z6q999lplzw22y9zyji6w5as1w2-optparse-applicative-<IP_ADDRESS>-r1.cabal.drv' failed with exit code 1;
       last 7 log lines:
       >
       > trying https://hackage.haskell.org/package/optparse-applicative-<IP_ADDRESS>/revision/1.cabal
       >   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
       >                                  Dload  Upload   Total   Spent    Left  Speed
       >   0    77    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
       > curl: (22) The requested URL returned error: 404
       > error: cannot download optparse-applicative-<IP_ADDRESS>-r1.cabal from any mirror
       For full logs, run 'nix-store -l /nix/store/ym132z6q999lplzw22y9zyji6w5as1w2-optparse-applicative-<IP_ADDRESS>-r1.cabal.drv'.

@EricWardner the version number it's trying to download there doesn't seem correct… Not sure where it would get that from, the version in the README.md is <IP_ADDRESS>, not <IP_ADDRESS>-r1.cabal.

I attempted the conversion to a flake. See https://github.com/drduh/YubiKey-Guide/issues/406. Feedback is much appreciated :)

Regenerating doesn't work for me :(
$ maintainers/scripts/haskell/regenerate-hackage-packages.sh
warning: Nix search path entry '/nix/var/nix/profiles/per-user/root/channels' does not exist, ignoring
these 2 paths will be fetched (2.43 MiB download, 15.92 MiB unpacked):
  /nix/store/7h0wrvl3ckgx2hc5460mjja3w7jijq6i-cabal2nix-unstable-2023-11-02
  /nix/store/2572jl8vhhimwcdy0rj5qq230l4ynw4z-distribution-nixpkgs-<IP_ADDRESS>-data
copying path '/nix/store/2572jl8vhhimwcdy0rj5qq230l4ynw4z-distribution-nixpkgs-<IP_ADDRESS>-data' from 'https://cache.nixos.org'...
copying path '/nix/store/7h0wrvl3ckgx2hc5460mjja3w7jijq6i-cabal2nix-unstable-2023-11-02' from 'https://cache.nixos.org'...
maintainers/scripts/haskell/regenerate-hackage-packages.sh: line 71: warning: setlocale: LC_ALL: cannot change locale (C.UTF-8): No such file or directory
Obtaining Hackage data …
warning: Nix search path entry '/nix/var/nix/profiles/per-user/root/channels' does not exist, ignoring
this derivation will be built:
  /nix/store/cbz1ki2rngmdgyz8fsg395inxsvsldy0-unpacked-cabal-hashes.drv
this path will be fetched (65.43 MiB download, 66.95 MiB unpacked):
  /nix/store/87bg2gniirfrz4qr1lfiw9iyb6dq07v6-all-cabal-hashes-def4ad9.tar.gz
copying path '/nix/store/87bg2gniirfrz4qr1lfiw9iyb6dq07v6-all-cabal-hashes-def4ad9.tar.gz' from 'https://cache.nixos.org'...
building '/nix/store/cbz1ki2rngmdgyz8fsg395inxsvsldy0-unpacked-cabal-hashes.drv'...
Generating compiler configuration …
warning: Nix search path entry '/nix/var/nix/profiles/per-user/root/channels' does not exist, ignoring
Running hackage2nix to regenerate pkgs/development/haskell-modules/hackage-packages.nix …
warning: Nix search path entry '/nix/var/nix/profiles/per-user/root/channels' does not exist, ignoring
hackage2nix: /home/motiejus/code/nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix: withFile: invalid argument (cannot encode character '\8211')

[motiejus@fwminex:~/code/nixpkgs]$ git diff --stat
 pkgs/development/haskell-modules/hackage-packages.nix | 321918 +---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 1 file changed, 1 insertion(+), 321917 deletions(-)


[storage-blob] add jsdocs

Exports the APIs that were missing from the index.ts file, and adds TSDoc comments where needed.
One caveat...I did not export the abstract StorageClient class from index.ts, since I don't think we actually want users to use it directly. If we do want to export it we'll have to export the StorageClientContext (and possibly other interfaces) as well.

There are types that are not directly used in the public api, but cause warnings in api etractor review reports. I chose not to export them because I don't think those types are likely to be used by our users. The structure of the response results are more important.
Sent from my Windows 10 device
From: Harsha<EMAIL_ADDRESS>Sent: Sunday, October 27, 2019 4:07
To<EMAIL_ADDRESS>Cc: Jeremy<EMAIL_ADDRESS>Review<EMAIL_ADDRESS>Subject: Re: [Azure/azure-sdk-for-js] [storage-blob] add jsdocs (#5857)
@HarshaNalluru commented on this pull request.

In sdk/storage/storage-blob/src/generatedModels.tshttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fazure-sdk-for-js%2Fpull%2F5857%23discussion_r339336871&data=02|01|Jeremy.Meng%40microsoft.com|e8c4964a80a24c1684c408d75acddd4c|72f988bf86f141af91ab2d7cd011db47|1|0|637077712539272143&sdata=XoT7fhj8AJzbqCzzAroHeH34ltp1pSGJpyK8SPkd2WM%3D&reserved=0:

AppendPositionAccessConditions,
ServiceGetUserDelegationKeyHeaders,
ServiceSubmitBatchHeaders,

ServiceGetAccountInfoHeaders,
ServiceGetPropertiesHeaders,

?
—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fazure-sdk-for-js%2Fpull%2F5857%3Femail_source%3Dnotifications%26email_token%3DABZ3QX77L3BCG6UVZ6D4OMLQQVY7HA5CNFSM4JFPFYGKYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCJKRQFA%23pullrequestreview-307566612&data=02|01|Jeremy.Meng%40microsoft.com|e8c4964a80a24c1684c408d75acddd4c|72f988bf86f141af91ab2d7cd011db47|1|0|637077712539282138&sdata=ONmf%2FEs6wZzXqiT1rcHNntmd2O1mAdejU%2BEmdwqqfuI%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABZ3QX4ISUW24CIWKCY74JDQQVY7HANCNFSM4JFPFYGA&data=02|01|Jeremy.Meng%40microsoft.com|e8c4964a80a24c1684c408d75acddd4c|72f988bf86f141af91ab2d7cd011db47|1|0|637077712539282138&sdata=Q%2FyE48wSLkd%2B%2FYjJnV85BxaeYtQhJ4dvYK5yOYX9I8A%3D&reserved=0.

There are types that are not directly used in the public api, but cause warnings in api etractor review reports. I chose not to export them because I don't think those types are likely to be used by our users. The structure of the response results are more important.

Why would we not want to export these though? Are we concerned these interface names may change in the future?
I noticed the typings that are used today are not the rolled-up ones. So, type inference does work, but if a user wanted to reference a type that isn't exported from index.ts they end up having to reach into our package:
import {BlobServiceClient, SharedKeyCredential} from '@azure/storage-blob';
import { ServiceGetAccountInfoHeaders } from '@azure/storage-blob/typings/src/generated/src/models';

async function getAccountParsedHeaders(): Promise<ServiceGetAccountInfoHeaders> {
  const client = new BlobServiceClient(`url`, new SharedKeyCredential('account', 'key'));
  const result = await client.getAccountInfo();

  // return the parsed headers...
  return result._response.parsedHeaders;
}

My fear with the above is that now the typings will break if we ever change the internal structure of our package.
If we did use the rollup-typings (Should we be? We do in the other SDK packages), these interfaces actually get pulled into the flattened typings so are available from the root of the package. In this case, exporting them from the index.ts file gets rid of the ae-forgotten-export warnings.

I tend to think those types should be exported from the root.
@chradek re: question about rollup types, I think we should do that eventually, but I wasn't going to push for it since api-extractor was broken with storage until @jeremymeng fixed it very recently.

Reverted policy renames in this PR. Will post in separate PR.

Are we concerned these interface names may change in the future?

Yeah I was thinking that we might need to rename these codegen types so was trying to reduced the exposed surface.


Intellisense not picking up PYTHONPATH modules

1.27.1-insider / Linux / Python 3.6

I think this was broken in the ms-python release this week.  Everything related to PYTHONPATH defined in my environment variables on Windows is broken for me.
I can see that PYTHONPATH gets dropped in the command line to ms-python.python-2018.8.0\pythonFiles\experimental\ptvsd:
cd 'c:\Users\janne\foo'; ${env:PYTHONPATH}='c:\Users\janne\.vscode\extensions\ms-python.python-2018.8.0\pythonFiles\experimental\ptvsd'; ${env:PYTHONIOENCODING}='UTF-8'; ${env:PYTHONUNBUFFERED}='1'; & 'python' '-m' 'ptvsd' '--nodebug' '--host' 'localhost' '--port' '54083' 'c:\Users\janne\foo\foo.py'

I was able to work around this by creating a workspace and addign PYTHONPATH into a .env file placed in the workspace root folder.  But this has been a source of confusion for my team, as they're used to VSCode behaving the same as running python on the command line (e.g., python foo.py works because PYTHONPATH is in environment, but doesn't work anymore from VSCode).
VSCode matching python behavior on the command line without adding any workspace or folder configuration in VSCode was one of the major selling points that convinced my team to switch to VSCode.

Can you please verify, @d3r3kk ?

I too ran into an issue with the PYTHONPATH environment variable not being set properly on runs using the default launch.json configurations to run a single file.
It looks like the extension rather than appending the path to ptvsd to the PYTHONPATH environment variable it instead overwrites it with the path to ptvsd
I was able to resolve the issue by placing this in my launch.json configuration:
`
{
        "name": "Python: Current File (Integrated Terminal)",

        "type": "python",

        "request": "launch",

        "program": "${file}",

        "env": {

            "PYTHONPATH":"temp"
        },
        "console": "integratedTerminal"

},
`
and the vscode properly appended the ptvsd path to my existing PYTHONPATH environment variable. Hope this provides some more information on the issue

@The-O-King Your issue has been reported here #2509

I can reproduce this using the prescribed method of placing the PYTHONPATH into a .env file in the root of a workspace.
@aniljava, if you switch to Jedi does it rectify the trouble for you?

Oops, closed accidentally


[icon] Bugfix for +art ~coupling.

Icon depends_on('libfyaml', when='+coupling') but consumed libfyaml also if +art.

launch jenkins icon

tsa
:green_circle: unit testTest:green_circle:summary :green_circle: integration testTest:green_circle:icon-spack_info :green_circle:icon-spack_spec :green_circle:icon_serialization=create_claw=std-spack_spec

This PR should the PR https://github.com/C2SM/spack-c2sm/pull/727 : we think the libfyaml is only need for +coupling (i.e. yac) but not for +art . With the current spack we are not able to compile with +art without +coupling - this PR should fix that issue

balfrin
:green_circle: unit testTest:green_circle:summary :green_circle: integration testTest:green_circle:icon-spack_info :green_circle:icon-spack_spec :green_circle:icon_serialization=create_claw=std-spack_spec :green_circle: system testTest:green_circle:Icon-install_2_6_6_gcc :green_circle:Icon-install_2_6_6_nvhpc :green_circle:Icon-install_nwp_cpu :green_circle:Icon-install_nwp_gpu

daint
:green_circle: unit testTest:green_circle:summary :green_circle: integration testTest:green_circle:icon-spack_info :green_circle:icon-spack_spec :green_circle:icon_serialization=create_claw=std-spack_spec :green_circle: system testTest:green_circle:Icon-install_2_6_6_gcc :green_circle:daint_cpu_cce :green_circle:daint_cpu_gcc :green_circle:daint_cpu_nvhpc :green_circle:daint_cpu_nvhpc_out_of_source :green_circle:daint_gpu_nvhpc


Determine a system for cataloging dbt plugins

I love that there are so many new dbt plugins in the world! At the time of this writing, I know about:

https://github.com/mikaelene/dbt-sqlserver
https://github.com/jacobm001/dbt-mssql
an in-progress Vertica plugin
an in-progress Azure Data Warehouse plugin
an in-progress MySQL plugin

We should figure out a good way to catalog these plugins such that users can determine the list of community-contributed plugins out there in the world. This is useful for two groups of people:

dbt users who want to use dbt with their database of choice
dbt users who want to develop (or contribute to) an existing plugin

An obvious place to start is the dbt documentation. I've already added a link to the dbt-mssql plugin in the list of supported databases in the documentation. This sort of breaks down if a plurality of plugins exist for a given database - I think it would be unwise for us to try to keep track of the latest documentation for every community-contributed plugin out there! Maybe instead, we can add a list of community-contributed plugins, separated out by database type, then leave the documentation to the owner of the plugin.
We should also consider what the path looks like to graduate a plugin from "community supported" to "officially supported". I think this will mostly entail:

more complete documentation at docs.getdbt.com
contributing patches when dbt-core is updated
the level of support we (core dbt maintainers) are able to provide if users have questions / issues

I'm very curious to hear what the maintainers of dbt plugins think about all of this! Ultimately, this discussion is motivated by a desire to help you in your plugin-building endeavors, so please don't hesitate to let me know what you think!
cc @ichavez @jacobm001 @mikaelene @norton120

Hi,
Sorry for late replay. I've been on a long vacation :-).
I think it's great to be able to create our own adapters and that others can make use of them and contribute.
From my experience when developing my sqlserver plugin, there are a lot of missing test cases in dbt-integration-tests. An update to that to include newer features, like snapshots etc, would be great and make it easier.
There is of course problems with multiple adapters for the same database. From my perspective I will need to continue develop my sqlserver plugin. It supports features that we need in our version of sqlserver (2017) and @jacobm001s mssql support features he needs in sqlserver 2008. With that in mind maybe a list is better until we have an adapter that support a wider set of versions?
I think it is quite a large job to make a new adapter a first class citizen in dbt. For sqlserver you would need pretty much the same level of customization as you have done for Bigquery. I think you guys at Fishtown has to decide if it worth it on your end as it will need some of your time to coordinate and make sure we just don't ad a crappy adapter to the great dbt :-).
/Mikael

I really appreciate that perspective @mikaelene! Totally agree that improving dbt-integration-tests is a super high-leverage thing we can do here.
I think I agree with you, we should provide a list of "known" plugins, then contributors combine their efforts or develop code in parallel. I think your example of sqlserver 2008 vs 2017 is a really good one!

@drewbanin - is there a link somewhere to the in progress dbt-vertica plugin?

hey @jordan8037310 - I'm not aware of any public vertica plugins for dbt, but maybe you could do a quick google search and see if there are any out there on the net! If so, I'd love to document it!

@jordan8037310 I have been working on a dbt plugin for Vertica the past week or so. It is not quite ready yet, but you can check it out here https://github.com/mpcarter/dbt-vertica
I have successfully passed 2 of the 3 features of dbt-integration-tests.

@mpcarter that's great! we're not yet near an implementation phase but collecting some info at this point. happy to contribute once we get moving, too

@mpcarter just an FYI if you haven't seen it elsewhere: we're going to put some work in to build a better integration test suite that makes it easier to test community-supported plugins. I can follow up with more info on that here when we have some code in place!

@drewbanin Sounds good 👍  FYI the Vertica plugin seems to work. Have not really put it through the paces yet though.

Came across this thread today and thought I would add that we created a registry of dbt related plugins
https://datacoves.com/dbt-libs
There's a specific page for dbt database connectors
https://datacoves.com/dbt-ibraries-categories/database-adapters


Project folder

Hi @manowb ,
I try to write some details on the project modifications.

As I said you, we added a gitignore. Maybe it is too strict: keeping the original .classpath and .project files solves the installation issues that I was talking about last friday. So in next commit, we are going to restore these files. However, we agree that it is better that the bin folder continues to be excluded from commits.

In the 'suggestion' branch, we propose a different structure for directories. In this version, all Java sources are in the src folder, that is organized in 5 package (the 4 modules you have identified, plus the main package). Each package follows the naming convention org.doremus.marc2rdf.packageName. The main reason is aligning the project to what conventionally developers do. What do you think about it?

Currently in the project there are two folders, Data and data. Following conventions, we should use the lowercase version, or rename it resources.

Then, Eva still has some doubt about the new version of the code. It is ok for you using GitHub issues for questions about coding? So that, it will easier reference files or line of code.
Thank you very much

Hi @pasqLisena,
Thank you for the explanation about the project modifications. It is OK for gitignore. and for your suggestion about restructuring the directories.
I don't know why there are two folders "Data" and "data". In my eclipse project there is only one.
Is it OK for you if I make now another commit to resolve it and integrate suggestions?
Of course, we can use GitHub issues to communicate :)

I don't know why there are two folders "Data" and "data". In my eclipse project there is only one.

Yes, in mine too. I think that when pulling in local, git solves the path, automatically.

Is it OK for you if I make now another commit to resolve it and integrate suggestions?

Yes, of course. We will wait for that. :smile:

Of course, we can use GitHub issues to communicate :)

Very good. Soon @fernanev will contact you.
Thank you very much. I'm available for any problem or doubt

With my last commits, I have:

removed the Data, moving all their files in data. Please everyone, check if also locally you have the correct name; if not, manually correct it.
solved 1.

I am going to close the suggestion branch. The master branch could be now the starting point for any future editing. :+1:
Thank you


Vertically center tab icon if there is no “label”

This is a small 'fix' in order to vertically centre icons if there is a missing label. The outcome of is illustrated in the image below. Ideally this is probably going to require a new flag on each tab, but usually it's a UX decision to either have labels or not. This just centres icons if there is no label.

Cancelling due to pull req #60 . This is a duplicate of that.


add short flags

https://github.com/gchaincl/httplab/issues/49 - Support for short flags

@gchaincl Thanks! I updated the README.md and used govendor to add pflag


fix immunized

Multiply 'fraction' by the proportion of susceptible individual to be sure to subtract and add a value that does not exceeds the proportion of susceptible individials

I see the issue you are addressing -- this function is error-prone if fraction exceeds s. But the intention is that fraction is the fraction of the population that is immunized, so the responsibility is on the caller to make sure the argument is sensible. I'm inclined to leave it as is. What do you think?

BTW, ignore the failing test -- that has nothing to do with this PR.

Thank you for answering, and for asking about my opinion.
I think that sweeping over fraction would result in incorrect estimates of the count of overall infected individuals, since when s < fraction <= 1 the amount of susceptible individuals becomes negative. So, if we want to leave the function as it is then we should sweep between $0$ and the fraction of susceptible individuals s, not between $0$ and $1$.
However, leaving the code untouched would make it easier for a reader to internalize the concept (as it worked for me) even if the result is not 100% correct.


Don't copy imported module to filter out its dependencies

At the moment contents of the imported module are copied to filter out its dependencies. It makes modules depend heavily on some way of iterating through the space. Bigger downside is that it duplicates the space on import and makes import slower as mentioned in Luke's comment in the code.
In this PR ModuleSpace is introduced which keeps main space and its dependencies separately. This allows querying them separately and when dependent space is queried it doesn't queried recursively. Thus all transitive dependencies are queried only once from the top level space.
For example if we have space A which imports space B and C and D is imported from B and C then A contains B, C and D as dependencies in a separate collection. Query to A goes through its atoms first then goes to B, C and D but dependencies of these spaces are not queried.
ModuleSpace::atom_iter() iterates through main atomspace only. This breaks tests in f1_imports.metta which are disabled in this change. There are two ways of solving this: (1) adding dependency spaces as atoms into ModuleSpace iterator which resembles previous behavior, (2) add separate function which gets dependencies from space. I would like to implement solution after discussing a way.
Second big change in this PR is implementation of the Space::atom_iter for DynSpace. This change was necessary because now ModuleSpace is a top space and it contains DynSpace as a main space. When client gets atom_iter from ModuleSpace the call is forwarded to DynSpace which returns None. Any non-standard combination of the DynSpace and other spaces can fall into this issue. Thus I decided to implement it despite it cannot be implemented without unsafe.
I am raising this as a draft PR to discuss two questions.

@Necr0x0Der, @luketpeterson Do we really need to return dependent atomspaces as atoms when iterating through the space. Probably more general it should sound like: do we need to represent imported spaces as atoms? On the one hand it is implemented like this now but trying to add dependency manually putting it into space leads to the duplication of the results.
@luketpeterson Could you please share your opinion on Space::atom_iter implementation for DynSpace I implemented. Is it correct? Do you think it is needed?
@luketpeterson What do you think about ModuleSpace as a way of solving dependency duplication issue?

old_interpreter CI fails because Space::common for DynSpace has issue similar to the Space:atom_iter. DynSpace cannot return FlexRef if underlying space returns FlexRef::RefCell.

I'll take a look in the morning.

Should also update the docs/modules_internal_discussion.md file to indicate the discussed solution is implemented.

@luketpeterson What do you think about ModuleSpace as a way of solving dependency duplication issue?

I think, at the conceptual level, we both hit upon the same solution; so our thinking is very well aligned.  My thoughts were written up in the docs/modules_internal_discussion.md file.

@luketpeterson Could you please share your opinion on Space::atom_iter implementation for DynSpace I implemented. Is it correct? Do you think it is needed?

I have conflicting opinions on the question of "is it correct"?  I believe your implementation will be safe in practice.  In effect you take two parallel references to the same data.  One Ref<T> and also the iterator.  The Ref<T> is along for the ride and just guarantees the RefCell's rules are followed so the iterator's accesses will be ok.  So, on the upside, from the outside everything should be ok.
On the downside, this is relying on implementation details of RefCell, so I think it would not technically be considered sound.  Also, this approach would have a race condition if we changed the RefCell to a Mutex or RwLock in the future.
I am sad because, in a perfect world, this should not require unsafe.  I wish there were a Ref::map_into_box API that allowed the creation of a boxed object that had the lifetime of the borrowed ref.
I did not find a crate that already does this, but I have wished for one myself in the past. (in fact, this is why I left the method unimplemented for DynSpace earlier in the year.)
If I had a few spare days I would try and generalize this pattern to a stand-alone crate and get it code-reviewed by the wider community, but I don't have the spare time to do that right now.
Bottom line: I don't love it, but I don't see a better short-term solution so I am ok with it as a short-term fix and we can revisit it in the future

UPDATE on the DynSpace implementation of Space::atom_iter
I also found this discussion on the topic here - https://github.com/rust-lang/rust/issues/54776  But it looks like it's not a very active topic.
Sadly, it also looks like owning_ref (another possible solution) still has one remaining soundness hole as well.  https://users.rust-lang.org/t/whats-the-current-state-of-the-owning-ref-crate/102098

Ugh.  I realized that there is an unfixable safety hole in the implementation of DynSpace impl of Space::atom_iter.  It comes from the lifetimes of the Iterator::next function.  fn next(&mut self) -> Option<Self::Item>;
Say the user borrows an iterator from the space behind a RefCell, then uses the iterator to get references to some atoms, saves those references, and then drops the iterator.  The user could then get a mutable reference to the space and delete those atoms.  This would leave the original atom references pointing to garbage.
To make this safe, the lifetime of the returned references can't outlive the iterator itself.  So you'd need a method like fn next(&'i mut self) -> Option<&'i Self::Item>;  We have been calling this the "cursor" pattern.  Alternatively the iterator could clone the atoms as it goes.  Either way, the Space::atom_iter method will need to change or an alternative way to access the atoms is needed.

old_interpreter CI fails because Space::common for DynSpace has issue similar to the Space:atom_iter. DynSpace cannot return FlexRef if underlying space returns FlexRef::RefCell.

For FlexRef the issue is less fundamental. It could be reworked using a pattern similar to https://crates.io/crates/nested-ref  (or you could use that crate)

Ugh. I realized that there is an unfixable* safety hole in the implementation of DynSpace impl of Space::atom_iter.

Thanks Luke, it took a while to realize it but after reading RefVal related links I realized that implementing iterator on collection internal mutability will always has such flaw. Thus I agree the only way is to change atom_iter API and I would replace it by adding fold of the atomspace.

For FlexRef the issue is less fundamental. It could be reworked using a pattern similar to https://crates.io/crates/nested-ref (or you could use that crate)

What bother me here is that using DynSpace along with space implementations which can include other spaces can lead to the uncontrolled increasing level of the indirection. To simplify it a little I would pass specific space instance into MettaMod constructor instead of passing DynSpace. It doesn't fully remove indirection because ModuleSpace::deps anyway should keep shared pointers to the dependency modules, but should solve specific issue with old_interpreter behaviour.


wrong type inference for boolean

TypeScript Version:  3.8.3

Search Terms:
Code
// A *self-contained* demonstration of the problem follows...
// Test this by running `tsc` on the command-line, rather than through another build tool such as Gulp, Webpack, etc.

const x = true

Expected behavior:
the type of x should be boolean
Actual behavior:
the type of x is true
Playground Link:
https://www.typescriptlang.org/play/index.html#code/MYewdgzgLgBAHjAvDKAnArgUwFBA

Related Issues:

This is the intended behavior.

The feature is type narrowing: http://www.staging-typescript.org/play/?q=311#example/union-and-intersection-types

But in this case, The feature cause an error.
https://www.typescriptlang.org/play/index.html#code/KYDwDg9gTgLgBASwHY2FAZgQwMbDgIUwGcFsAVATzGABFgjs4BvAKDjgG0woIwiAuOERhRkAcwC6gpAFcAtgCM0cAD5CR41XAUQIAG2CYkAbhYBfFqEiw42PcSJxK1OgwA8ZOKFRIAJo8IScipaemwAPmY2OBgQgEkkdAhBMlN2CHR0UjRBAHlM7KgPSIBeOCRgAHc4fKzcKAAKGAALBCIASjTbCCRhKBlsGGgm+MTkp3ao9nYWtoA6WOoEpLgyxeBliC6LC0twaHg7BxqC+o8vEB9-AmJSZ1CGSNZ2bFa9XwBZeiJMMWAAfjypzQbiMFHCXXWrmwKRC0OKpmi2B6fQGQ0aULCsJcYWKk2e01mRAWcLCqxipIY2xY0UqolQDTkRDEgImzB2NORvXgAAVgLwDOSKtV7tCGgS2gAlBBiZowQRYPREYDmTo0vkC4BzDJ1NBzOkIBkSojS2XymL9FVmNVAA

But in this case, The feature cause an error.
https://www.typescriptlang.org/play/index.html#code/KYDwDg9gTgLgBASwHY2FAZgQwMbDgIUwGcFsAVATzGABFgjs4BvAKDjgG0woIwiAuOERhRkAcwC6gpAFcAtgCM0cAD5CR41XAUQIAG2CYkAbhYBfFqEiw42PcSJxK1OgwA8ZOKFRIAJo8IScipaemwAPmY2OBgQgEkkdAhBMlN2CHR0UjRBAHlM7KgPSIBeOCRgAHc4fKzcKAAKGAALBCIASjTbCCRhKBlsGGgm+MTkp3ao9nYWtoA6WOoEpLgyxeBliC6LC0twaHg7BxqC+o8vEB9-AmJSZ1CGSNZ2bFa9XwBZeiJMMWAAfjypzQbiMFHCXXWrmwKRC0OKpmi2B6fQGQ0aULCsJcYWKk2e01mRAWcLCqxipIY2xY0UqolQDTkRDEgImzB2NORvXgAAVgLwDOSKtV7tCGgS2gAlBBiZowQRYPREYDmTo0vkC4BzDJ1NBzOkIBkSojS2XymL9FVmNVAA

@Akimotorakiyu You can use a interface fix the issue:
interface IPeople extends BasicTypeDesc
{
    isRight: boolean;
}

const People = new TypeDesc<IPeople>({ isRight: false });

@Akimotorakiyu You can use a interface fix the issue:
interface IPeople extends BasicTypeDesc
{
    isRight: boolean;
}

const People = new TypeDesc<IPeople>({ isRight: false });

I have too many difference kind TypeDesc instance adn every kind instance is unique kind, enn... that's to say I need many extra interface to fix that?

You can write
    isRight: false as boolean

@RyanCavanaugh Yes, I did.


Improve Document Ingest UX

This PR fixes quite a few things

refresh() and update() now work (and use Document objects as input)
delete() works, and takes NODE doc_ids as input
added delete_ref_doc() which takes a REF_DOC_ID as input, and deletes all nodes associated with it
added optional kwarg to delete nodes from docstore (set it to false though, because if you share nodes across indexes this can cause problems 🤔)
added functionality to the docstore to track ref_doc info. For every node inserted, we keep track of which ref_doc_id that node belongs to + it's extra info
aligned all _delete() implementations to delete NODE doc_ids (some did, some didn't). As you can see, the terminology is starting to get a little wacky, But I want to avoid breaking changes right now

Todo:

[x] Add a page to the docs? Not sure where yet

Nice! Will take a look!

Todo:

[ ]  Add a page to the docs? Not sure where yet

I think we should just update https://gpt-index.readthedocs.io/en/latest/how_to/index_structs/update.html
I think some are no longer correct, e.g.
index.update(doc_chunks[0])

Wow that was a lot haha
Should be good to review again.
As I noted in slack, the biggest downside so far is that with 3rd party vector store integrations, there's no easy way to see what's been ingested without building that functionality into every integration 😢

Decided to also add ref_doc_info as a property to each index (and updated the docs guide as well)


Unable to install onnxsim in jetson containers on ORIN AGX

Thanks for the great work on setting up these containers!
Unfortunately, I am unable to install onnxsim on my Jetson Orin AGX in the l4t-pytorch containers.
My setup:
Jetson AGX Orin 64GB Developer Kit
Jetpack Version: 6.0-b52 (confirmed with sudo apt show nvidia-jetpack -a)
Error: Installing onnxsim fails; it ends up in an infinite loop when building the wheels for setup.py

@GrauleM IIRC I had also faced issue in the past with getting onnxsim to install, and ended up just running the model simplification on x86, and then copied the resulting ONNX to Jetson

Ok, thanks for the fast response!


Azure Activity Logs

Add an E2E Source test for the Azure Activity Logs.  The test itself is based on the Azure EventHubs E2E test and has the same environment variable requirements.
This test will:

Create two Azure Resource Groups: one to stream the Activity Log components and a second to perform an operation to track
Instantiate the eventhub components
Create the source
Wait for the events to flow

Due to the nature of the Activity Log, events are delayed and sometimes batched during sending which will require this test to have a very long polling interval for events (5 minutes). Additionally, a check is made to ensure that at least one event has been received while several could be received from the starting of an operation to the completion of an operation.
Lastly, this PR migrates the EventHub creation to the framework pkg to support component reuse.
Relates to #122

Thank you @FranBarrera and @JeffNeff. I pushed out the update for the eventhub name and converted the region constant to pull the environment variable instead.


Support Sqlite binary column

For Automerge and the other binary-based libs within Evolu.

Done in PR


Added FORMAT option to assert equal

@EGJ1996 The following test case should be useful for debugging:
GL 4.5

CREATE_BUFFER buf1 SIZE_BYTES 16 INIT_VALUES float 1.0 2.0 3.0 4.0

CREATE_BUFFER buf2 SIZE_BYTES 16 INIT_VALUES float 1.0 2.0 3.0 4.0

ASSERT_EQUAL BUFFERS buf1 buf2 FORMAT float 4

It should pass, but currently fails.
Also, separately, I would expect an error message from this one:
GL 4.5
CREATE_BUFFER buf1 SIZE_BYTES 16 INIT_VALUES float 1.0 2.0 3.0 4.0
CREATE_BUFFER buf2 SIZE_BYTES 16 INIT_VALUES float 1.0 2.0 3.0 4.0
ASSERT_EQUAL BUFFERS buf1 buf2 FORMAT

because `FORMAT` is used but then no format specifiers are given - i.e. 0 bytes' worth of formatting are specified.

Here is a patch for executor.cc:
patch.txt
Can you please apply it by doing:
git apply patch.txt
from the root of your repo?
If you can push it and then make the other changes, and then check that check_all.sh terminates with no errors, that would be great.


Two children with the same key

Warning: Encountered two children with the same key, %s. Keys should be unique so that components maintain their identity across updates. Non-unique keys may cause children to be duplicated and/or omitted — the behavior is unsupported and could change in a future version.%s, 0x0e30f72f435ee22b208f458f5bbd39435d33dea2e308edd9b8e96c4805a2b28c,
    in RCTScrollContentView (at ScrollView.js:852)
    in RCTScrollView (at ScrollView.js:977)
    in ScrollView (at VirtualizedList.js:1062)
    in VirtualizedList (at FlatList.js:662)
    in FlatList (at LandForSaleList.js:22)
    in LandForSaleList (at ListLandForSaleScreen.js:135)
    in RCTView (at View.js:44)
    in Root (at connectStyle.js:384)
    in Styled(Root) (at ListLandForSaleScreen.js:134)
    in ListLandForSaleScreen (created by Context.Consumer)
    in Connect(ListLandForSaleScreen) (at SceneView.js:9)
    in SceneView (at StackViewLayout.js:797)
    in RCTView (at View.js:44)
    in AnimatedComponent (at StackViewCard.js:69)
    in RCTView (at View.js:44)
    in AnimatedComponent (at screens.native.js:59)
    in Screen (at StackViewCard.js:57)
    in Card (at createPointerEventsContainer.js:27)
    in Container (at StackViewLayout.js:862)
    in RCTView (at View.js:44)
    in ScreenContainer (at StackViewLayout.js:313)
    in RCTView (at View.js:44)
    in AnimatedComponent (at StackViewLayout.js:309)
    in Handler (at StackViewLayout.js:302)
    in StackViewLayout (at withOrientation.js:30)
    in withOrientation (at StackView.js:79)
    in RCTView (at View.js:44)
    in Transitioner (at StackView.js:22)
    in StackView (at createNavigator.js:61)
    in Navigator (at createKeyboardAwareNavigator.js:12)
    in KeyboardAwareNavigator (at SceneView.js:9)
    in SceneView (at DrawerView.js:149)
    in RCTView (at View.js:44)
    in RCTView (at View.js:44)
    in ResourceSavingScene (at DrawerView.js:148)
    in RCTView (at View.js:44)
    in ScreenContainer (at DrawerView.js:138)
    in RCTView (at View.js:44)
    in AnimatedComponent (at DrawerLayout.js:440)
    in RCTView (at View.js:44)
    in AnimatedComponent (at DrawerLayout.js:439)
    in Handler (at DrawerLayout.js:495)
    in DrawerLayout (at DrawerView.js:165)
    in DrawerView (at createNavigator.js:61)
    in Navigator (at createAppContainer.js:387)
    in NavigationContainer (at App.js:114)
    in RCTView (at View.js:44)
    in Root (at connectStyle.js:384)
    in Styled(Root) (at App.js:113)
    in Provider (at App.js:112)
    in App (at withExpoRoot.js:22)
    in RootErrorBoundary (at withExpoRoot.js:21)
    in ExpoRootComponent (at renderApplication.js:34)
    in RCTView (at View.js:44)
    in RCTView (at View.js:44)
    in AppContainer (at renderApplication.js:33)

Hmm, actually it seems like the error might be related to duplicate land parcels/estates after running the npm start script from the Tasit SDK? Because LandForSaleList shows up in the stack trace above. I'll see if this issue comes up again.

Okay, I figured out how to reproduce the error. It happens after creating and funding an account and approving the marketplace, right when you click the Buy button

So the handling of state after the purchase seems to be the issue - when you first arrive on the ListLandForSaleScreen, it works fine.

Then again, the key 0x0e30f72f435ee22b208f458f5bbd39435d33dea2e308edd9b8e96c4805a2b28c sure seems like the ephemeral account address

It might be action actionId though. Checking with logs now.

It appears to be the landForSale id that's a duplicate. See 0x2f... in both of these images

Okay, I think I've got it. We're both appending and prepending the same piece of land for sale, 0x5e... in this case:
[14:25:59] append landForSale id 0x5e46f553729dc00672e376937b8bf89b5290bdf3d0690e0bf4d5dd33476591d3

[14:25:59] append landForSale id 0x2f1943de849df41a5322a39a1315340b7e174bf4ebb024d7f8f76abe71021a97

[14:25:59] append landForSale id 0xfc9ff5d7000ff1324ae5dcf973e74ac56d353a16575a718514d38e2284015490
...
[14:25:59] append landForSale id 0x7d43fcfc33ac534013a6b08b1796b8b265c90147447cf97c382a869235d257f9

[14:25:59] append landForSale id 0x2f1f6778bde1dad34ac9c425f8f341fa05b24105f1b11d819daa3e90b9f214fc

[14:26:02] landForSaleId 0x5e46f553729dc00672e376937b8bf89b5290bdf3d0690e0bf4d5dd33476591d3

[14:26:02] prepend landForSale id 0x5e46f553729dc00672e376937b8bf89b5290bdf3d0690e0bf4d5dd33476591d3

Here is the error that got us into the state of mismanaging local land for sale state in the error path:
triggered too many arguments in interface function safeExecuteOrder (count=5, expectedCount=4, version=4.0.27)


[BUG?] Not able to filter and load a partitioned parquet file using dask

I'm having a problem in filtering and loading a partitioned parquet file using dask, which I show below.
I'm including full code for reproducibility.
Versions
dask==0.17.0
pandas==0.22.0
fastparquet==0.1.4

Init code
Imports
import numpy as np
import pandas as pd
import dask.dataframe as dd
import random
import fastparquet

Generation and plotting functions
#generate dummy df
def generate_df(size):
    d = {
        'dates':
        pd.date_range('1980-01-01', periods=size, freq='1T'),
        'signal1':
        np.random.normal(0, 0.3, size=size).cumsum() + 50,
        'signal2':
        np.random.normal(0, 0.3, size=size).cumsum() + 50,
        'signal3':
        np.random.normal(0, 0.3, size=size).cumsum() + 50,
        'fake_categorical1':
        np.random.choice(['A', 'B', 'C'], size=size),
        'fake_categorical2':
        np.random.choice(['D', 'E', 'F'], size=size)
    }

    df = pd.DataFrame(d)

    return df.melt(id_vars=['dates', 'fake_categorical1', 'fake_categorical2'])

Generate df
Let's generate a small df:
small_df = generate_df(10)
small_df

dates
fake_categorical1
fake_categorical2
variable
value

0
1980-01-01 00:00:00
C
D
signal1
50.106769

1
1980-01-01 00:01:00
C
F
signal1
50.134314

2
1980-01-01 00:02:00
B
F
signal1
50.049454

3
1980-01-01 00:03:00
C
E
signal1
50.204364

4
1980-01-01 00:04:00
A
D
signal1
50.337799

5
1980-01-01 00:05:00
A
D
signal1
50.716083

6
1980-01-01 00:06:00
B
E
signal1
50.389717

7
1980-01-01 00:07:00
C
E
signal1
50.295345

8
1980-01-01 00:08:00
C
E
signal1
50.221086

9
1980-01-01 00:09:00
C
E
signal1
50.074217

10
1980-01-01 00:00:00
C
D
signal2
50.475801

11
1980-01-01 00:01:00
C
F
signal2
49.874271

12
1980-01-01 00:02:00
B
F
signal2
49.745380

13
1980-01-01 00:03:00
C
E
signal2
49.967490

14
1980-01-01 00:04:00
A
D
signal2
49.822340

15
1980-01-01 00:05:00
A
D
signal2
49.740567

16
1980-01-01 00:06:00
B
E
signal2
49.393413

17
1980-01-01 00:07:00
C
E
signal2
49.151365

18
1980-01-01 00:08:00
C
E
signal2
49.381807

19
1980-01-01 00:09:00
C
E
signal2
48.647324

20
1980-01-01 00:00:00
C
D
signal3
49.837980

21
1980-01-01 00:01:00
C
F
signal3
49.787703

22
1980-01-01 00:02:00
B
F
signal3
49.616496

23
1980-01-01 00:03:00
C
E
signal3
49.674050

24
1980-01-01 00:04:00
A
D
signal3
49.987540

25
1980-01-01 00:05:00
A
D
signal3
49.667552

26
1980-01-01 00:06:00
B
E
signal3
49.553683

27
1980-01-01 00:07:00
C
E
signal3
50.074661

28
1980-01-01 00:08:00
C
E
signal3
50.526340

29
1980-01-01 00:09:00
C
E
signal3
51.246639

Here is where the problem shows
Write and read non partitioned parquet:
Save a non partitioned parquet:
fastparquet.write('file_non_partitioned', small_df, compression='snappy',
                  file_scheme='hive', write_index=False)

Read it with dask:
df_non_partitioned = dd.read_parquet('file_non_partitioned')

No problems showing head():
df_non_partitioned.head()

dates
fake_categorical1
fake_categorical2
variable
value

0
1980-01-01 00:00:00
C
D
signal1
50.106769

1
1980-01-01 00:01:00
C
F
signal1
50.134314

2
1980-01-01 00:02:00
B
F
signal1
50.049454

3
1980-01-01 00:03:00
C
E
signal1
50.204364

4
1980-01-01 00:04:00
A
D
signal1
50.337799

And no problems filtering on a value of a column and computing... cool! The problem comes with partitioned parquet (see below).
df_non_partitioned[df_non_partitioned.fake_categorical1=='A'].compute()

dates
fake_categorical1
fake_categorical2
variable
value

4
1980-01-01 00:04:00
A
D
signal1
50.337799

5
1980-01-01 00:05:00
A
D
signal1
50.716083

14
1980-01-01 00:04:00
A
D
signal2
49.822340

15
1980-01-01 00:05:00
A
D
signal2
49.740567

24
1980-01-01 00:04:00
A
D
signal3
49.987540

25
1980-01-01 00:05:00
A
D
signal3
49.667552

# Write and read partitioned parquet
Save a partitioned parquet:
fastparquet.write('file_partitioned', small_df, compression='snappy',
                  file_scheme='hive', write_index=False,
                  partition_on=['fake_categorical1','fake_categorical2'])

Read it with dask:
df_partitioned = dd.read_parquet('file_partitioned')

No problems showing head():
df_partitioned.head()

dates
variable
value
fake_categorical1
fake_categorical2

0
1980-01-01 00:04:00
signal1
50.337799
A
D

1
1980-01-01 00:05:00
signal1
50.716083
A
D

2
1980-01-01 00:04:00
signal2
49.822340
A
D

3
1980-01-01 00:05:00
signal2
49.740567
A
D

4
1980-01-01 00:04:00
signal3
49.987540
A
D

No problems running compute() on the full dataframe:
df_partitioned.compute().head()

dates
variable
value
fake_categorical1
fake_categorical2

0
1980-01-01 00:04:00
signal1
50.337799
A
D

1
1980-01-01 00:05:00
signal1
50.716083
A
D

2
1980-01-01 00:04:00
signal2
49.822340
A
D

3
1980-01-01 00:05:00
signal2
49.740567
A
D

4
1980-01-01 00:04:00
signal3
49.987540
A
D

But when trying to filtering on a column before computing it doesnt work:
df_partitioned[df_partitioned.fake_categorical1=='A'].compute()

Error and more examples
---------------------------------------------------------------------------

ValueError                                Traceback (most recent call last)

<ipython-input-33-3a07acb2728f> in <module>()
----> 1 df_partitioned[df_partitioned.fake_categorical1=='A'].compute()

~/andre/yakushima-3env/lib/python3.5/site-packages/dask/base.py in compute(self, **kwargs)
    141         dask.base.compute
    142         """
--> 143         (result,) = compute(self, traverse=False, **kwargs)
    144         return result
    145

~/andre/yakushima-3env/lib/python3.5/site-packages/dask/base.py in compute(*args, **kwargs)
    390     postcomputes = [a.__dask_postcompute__() if is_dask_collection(a)
    391                     else (None, a) for a in args]
--> 392     results = get(dsk, keys, **kwargs)
    393     results_iter = iter(results)
    394     return tuple(a if f is None else f(next(results_iter), *a)

~/andre/yakushima-3env/lib/python3.5/site-packages/dask/threaded.py in get(dsk, result, cache, num_workers, **kwargs)
     73     results = get_async(pool.apply_async, len(pool._pool), dsk, result,
     74                         cache=cache, get_id=_thread_get_id,
---> 75                         pack_exception=pack_exception, **kwargs)
     76
     77     # Cleanup pools associated to dead threads

~/andre/yakushima-3env/lib/python3.5/site-packages/dask/local.py in get_async(apply_async, num_workers, dsk, result, cache, get_id, rerun_exceptions_locally, pack_exception, raise_exception, callbacks, dumps, loads, **kwargs)
    519                         _execute_task(task, data)  # Re-execute locally
    520                     else:
--> 521                         raise_exception(exc, tb)
    522                 res, worker_id = loads(res_info)
    523                 state['cache'][key] = res

~/andre/yakushima-3env/lib/python3.5/site-packages/dask/compatibility.py in reraise(exc, tb)
     65         if exc.__traceback__ is not tb:
     66             raise exc.with_traceback(tb)
---> 67         raise exc
     68
     69 else:

~/andre/yakushima-3env/lib/python3.5/site-packages/dask/local.py in execute_task(key, task_info, dumps, loads, get_id, pack_exception)
    288     try:
    289         task, data = loads(task_info)
--> 290         result = _execute_task(task, data)
    291         id = get_id()
    292         result = dumps((result, id))

~/andre/yakushima-3env/lib/python3.5/site-packages/dask/local.py in _execute_task(arg, cache, dsk)
    269         func, args = arg[0], arg[1:]
    270         args2 = [_execute_task(a, cache) for a in args]
--> 271         return func(*args2)
    272     elif not ishashable(arg):
    273         return arg

~/andre/yakushima-3env/lib/python3.5/site-packages/dask/dataframe/io/parquet.py in _read_parquet_row_group(fs, fn, index, columns, rg, series, categories, schema, cs, dt, scheme, storage_name_mapping, *args)
    364                               for name in index]
    365     df.columns = [storage_name_mapping.get(col, col)
--> 366                   for col in columns
    367                   if col != index]
    368

~/andre/yakushima-3env/lib/python3.5/site-packages/pandas/core/generic.py in __setattr__(self, name, value)
   3625         try:
   3626             object.__getattribute__(self, name)
-> 3627             return object.__setattr__(self, name, value)
   3628         except AttributeError:
   3629             pass

pandas/_libs/properties.pyx in pandas._libs.properties.AxisProperty.__set__()

~/andre/yakushima-3env/lib/python3.5/site-packages/pandas/core/generic.py in _set_axis(self, axis, labels)
    557
    558     def _set_axis(self, axis, labels):
--> 559         self._data.set_axis(axis, labels)
    560         self._clear_item_cache()
    561

~/andre/yakushima-3env/lib/python3.5/site-packages/pandas/core/internals.py in set_axis(self, axis, new_labels)
   3072             raise ValueError('Length mismatch: Expected axis has %d elements, '
   3073                              'new values have %d elements' %
-> 3074                              (old_len, new_len))
   3075
   3076         self.axes[axis] = new_labels

ValueError: Length mismatch: Expected axis has 2 elements, new values have 1 elements

I also noticed that I am not able to load one column (even without filtering):
df_partitioned[['fake_categorical1']].compute()

---------------------------------------------------------------------------

ValueError                                Traceback (most recent call last)

<ipython-input-34-a68c843edc71> in <module>()
----> 1 df_partitioned[['fake_categorical1']].compute()

~/andre/yakushima-3env/lib/python3.5/site-packages/dask/base.py in compute(self, **kwargs)
    141         dask.base.compute
    142         """
--> 143         (result,) = compute(self, traverse=False, **kwargs)
    144         return result
    145

~/andre/yakushima-3env/lib/python3.5/site-packages/dask/base.py in compute(*args, **kwargs)
    390     postcomputes = [a.__dask_postcompute__() if is_dask_collection(a)
    391                     else (None, a) for a in args]
--> 392     results = get(dsk, keys, **kwargs)
    393     results_iter = iter(results)
    394     return tuple(a if f is None else f(next(results_iter), *a)

~/andre/yakushima-3env/lib/python3.5/site-packages/dask/threaded.py in get(dsk, result, cache, num_workers, **kwargs)
     73     results = get_async(pool.apply_async, len(pool._pool), dsk, result,
     74                         cache=cache, get_id=_thread_get_id,
---> 75                         pack_exception=pack_exception, **kwargs)
     76
     77     # Cleanup pools associated to dead threads

~/andre/yakushima-3env/lib/python3.5/site-packages/dask/local.py in get_async(apply_async, num_workers, dsk, result, cache, get_id, rerun_exceptions_locally, pack_exception, raise_exception, callbacks, dumps, loads, **kwargs)
    519                         _execute_task(task, data)  # Re-execute locally
    520                     else:
--> 521                         raise_exception(exc, tb)
    522                 res, worker_id = loads(res_info)
    523                 state['cache'][key] = res

~/andre/yakushima-3env/lib/python3.5/site-packages/dask/compatibility.py in reraise(exc, tb)
     65         if exc.__traceback__ is not tb:
     66             raise exc.with_traceback(tb)
---> 67         raise exc
     68
     69 else:

~/andre/yakushima-3env/lib/python3.5/site-packages/dask/local.py in execute_task(key, task_info, dumps, loads, get_id, pack_exception)
    288     try:
    289         task, data = loads(task_info)
--> 290         result = _execute_task(task, data)
    291         id = get_id()
    292         result = dumps((result, id))

~/andre/yakushima-3env/lib/python3.5/site-packages/dask/local.py in _execute_task(arg, cache, dsk)
    269         func, args = arg[0], arg[1:]
    270         args2 = [_execute_task(a, cache) for a in args]
--> 271         return func(*args2)
    272     elif not ishashable(arg):
    273         return arg

~/andre/yakushima-3env/lib/python3.5/site-packages/dask/dataframe/io/parquet.py in _read_parquet_row_group(fs, fn, index, columns, rg, series, categories, schema, cs, dt, scheme, storage_name_mapping, *args)
    364                               for name in index]
    365     df.columns = [storage_name_mapping.get(col, col)
--> 366                   for col in columns
    367                   if col != index]
    368

~/andre/yakushima-3env/lib/python3.5/site-packages/pandas/core/generic.py in __setattr__(self, name, value)
   3625         try:
   3626             object.__getattribute__(self, name)
-> 3627             return object.__setattr__(self, name, value)
   3628         except AttributeError:
   3629             pass

pandas/_libs/properties.pyx in pandas._libs.properties.AxisProperty.__set__()

~/andre/yakushima-3env/lib/python3.5/site-packages/pandas/core/generic.py in _set_axis(self, axis, labels)
    557
    558     def _set_axis(self, axis, labels):
--> 559         self._data.set_axis(axis, labels)
    560         self._clear_item_cache()
    561

~/andre/yakushima-3env/lib/python3.5/site-packages/pandas/core/internals.py in set_axis(self, axis, new_labels)
   3072             raise ValueError('Length mismatch: Expected axis has %d elements, '
   3073                              'new values have %d elements' %
-> 3074                              (old_len, new_len))
   3075
   3076         self.axes[axis] = new_labels

ValueError: Length mismatch: Expected axis has 2 elements, new values have 1 elements

I also noticed that if I choose both columns that were used to do the partitioning it works!
If not, Am I loading and filtering the partitioned parquet correctly? How can I filter on a value of a specific column before mamking compute()?

Thank you in advance!

I found a way of how to do it. If I read the partitioned parquet as:
df_partitioned = dd.read_parquet('file_partitioned',
                                 filters=[('fake_categorical1', '==', 'A')])

I am able to get a DataFrame that only contains fake_categorical1=='A'. But still this is not very usable or practical.
Is there a way to do it with .loc or other way similar to pandas?

Thank you for a well-specified problem. This sounds like a bug, possibly in fastparquet.
Note that using filters= is a different mechanism altogether: you do not read the partitions that don't have any rows matching your criteria at all; whereas filtering with [..] does consider every row of every partition.
I suspect that the error is coming from the selection of one of the columns - there is a special optimization under the hood to only load those columns from the underlying data; but in this case, the special column does not correspond to any data, only directory names, so maybe nothing is being loaded.
One immediate workaround you could try:
df_partitioned.map_partitions(lambda df: df[df.variable=='signal1'])

which, if it works, will I think will prove what the problem is.

@martindurant Thank you for your reply.
I understand that filters= only causes the read of the relevant partitions. But it seems a bit surprising that filtering with [...] would always cause the read of all partitions.
I think it would make sense that when filtering on the column which the parquet file was partitioned on, the lib (Dask or Fastparquet, not sure which one should have such abstraction) should be clever enough to know the DataFrame was coming from a partitioned parquet file and to only 'touch' the relevant partitions.
For example:
fastparquet.write('file_partitioned', small_df, compression='snappy',
                  file_scheme='hive', write_index=False,
                  partition_on=['fake_categorical1','fake_categorical2'])

df_partitioned = dd.read_parquet('file_partitioned')

pandas_df = df_partitioned[(df_partitioned.fake_categorical2 =='A') &
                        ((df_partitioned.variable=='signal1'))].compute()

Should only read the partitions where fake_categorical2 =='A' and then filter those partitions on variable=='signal1'.
Do you agree?
As for your workaround I'll only be able to test it tomorrow. But if  am not mistaken, if it works, it will cause the load of all partitions, even if I am filtering on the column that was used to create the partitions, right?


Parts In Stock

Make sure parts are in stock with a supplier that can deliver quickly, in small quantities, and are also available at very large scale.

BOM created in octopart, and linked from readme. Parts are available @ volume of 25 devices (25x complete BOM) from a combination of TME + digikey.
https://octopart.com/bom-lookup/NXcyO8GW


PIL optimizer: The optimizer should remove any definition (not just columns) that is not referenced.

This is especially relevant since many utility functions are only used to be evaluated and return expressions / constraints and then do not occur in the constraints any more.

I think this is actually rather important because I think currently the entire lib is tossed into the generated PIL file which causes slower parsing later.


UCP: Migrate scalar function JsonDepthSig from TiDB

Description
Port the scalar function JsonDepthSig from TiDB to coprocessor.
Score

50

Mentor(s)

@mapleFU

Recommended Skills

Rust programming

Learning Materials
Already implemented expressions ported from TiDB

https://github.com/tikv/tikv/tree/master/components/tidb_query/src/rpn_expr)
https://github.com/tikv/tikv/tree/master/components/tidb_query/src/expr)

/pick-up-challenge

/give-up-challenge

@mapleFU It seems that this has been solved in https://github.com/tikv/tikv/pull/6167 . Maybe we can close this issue?


Change styles compiler

The styles are compiled by webpacker, sprockets should be used for this.

https://github.com/ita-social-projects/ZeroWaste/pull/364


Negative date containing an "X" will not validate

Trying to represent an uncertain date of 2nd century BC with unspecified digits
-01XX should validate but it does not.

So there's actually two things that are going on here. First, you are absolutely right, we should be accepting negative values for dates that have the unspecified character. I'm gonna work on that. But another thing of note is that it looks like you might be referring to the latest version of the spec, which is not what our current code is targeting. We actually built this to comply with the draft specification, which used 'u' instead of 'X' as the unspecified character. So, after I make a quick fix we should be able to validate -01uu, but we're not quite ready to support -01XX yet.

This was partially addressed in #16. Since we aren't changing our target spec yet, I'm going to close this.


HDDS-6129. OM has thread unsafe issues that can cause OM shutdown.

What changes were proposed in this pull request?
See: https://issues.apache.org/jira/browse/HDDS-6129
How was this patch tested?
Existing UT

I have modified this PR by referring to the OmMultipartKeyInfo modification in HDDS-2322. Also added UT. Can you help take antoher look? @bharatviswa504 @adoroszlai

Now with this change, not sure the added test makes sense.

Agreed, I posted the test method only as a simple way to repro the problem.

Now with this change, not sure the added test makes sense.

Agreed, I posted the test method only as a simple way to repro the problem.

Tnanks @adoroszlai and @bharatviswa504 for the review.
I changed the original test, where I passed subject.copyObject(). This will get an error before this patch,  after we re-new the object there will be no errors here.

Merged this. Thanks @adoroszlai and @bharatviswa504 for the review.


Allow players to change cosmetics mid-lobby

I'm not sure how feasible this would be but I would find myself a lot more willing to use cosmetics if I'd be able to remove them later on while playing.
Sometimes i want to switch suits but i have weird cosmetics and i can only switch them by leaving the lobby and changing them, and you can't re-join mid session so everyone else ALSO has to rejoin.
Thank you for your awesome mod <3

https://github.com/notnotnotswipez/MoreCompany/pull/95


How to change order options of auto correction prompt?

Currently zsh prompt as follows when typo in  command
zsh: correct 'sl' to 'ls' [nyae]?

is that possible to change the order to [ynae] so when I press enter it will be executed.

The prompt can be changed by tweaking the $SPROMPT variable (see docs, search for SPROMPT). Unfortunately, the behavior is baked in the zsh binary itself. As it says in the docs, previously SPACE or TAB where accepted as yes; now, they are invalid choices. See also: Parameters.


⚠ Processors has degraded performance

In 810e685, Processors ($STATUS_URL) experienced degraded performance:

HTTP code: 200
Response time: 56 ms

Resolved: Processors performance has improved in 0c985f2.


stm32, Unable to read UART while checking from Framing error.

I'm implementing DMX512 input on an F4.
Reading it's done at 250000 bauds, the Break and the Mark After Break it's at 100000 so it's common practice to detect a framing error instead of switching baudrate or edge detection.
I have an interrupt handler for the UART IRQ, inside it I have this piece of code:
while (uart_irq_update(dev) && uart_irq_is_pending(dev)) {
   if (uart_irq_rx_ready(dev)) {
        int err = uart_err_check(dev);
        if (err > 0)
        {
            if(err == UART_ERROR_OVERRUN ){
                printk("[UART0_RXTX] UART_ERROR_OVERRUN \n");
            }
            if(err == UART_ERROR_PARITY  ){
                printk("[UART0_RXTX] UART_ERROR_PARITY \n ");
            }
            if(err == UART_ERROR_FRAMING ){
                printk("[UART0_RXTX] UART_ERROR_FRAMING \n ");
            }
            if(err == UART_BREAK ){
                printk("[UART0_RXTX] UART_BREAK \n");
            }
            if(err == UART_ERROR_COLLISION ){
                printk("[UART0_RXTX] UART_ERROR_COLLISION \n");
            }
        }
        int ret = uart_fifo_read(dev,&_rx_buffer,1);
        if (ret > 0)
        {
            printk("read data %d\n"_rx_buffer);
        }
        if (ret < 0)
        {
            printk("error reading data\n");
        }
        else{
            printk("no data\n");
        }
    }

I should be able to detect the Break or the data but if I detect the Break , uart_fifo_read always returns 0,  this doesn't happen if I try to not detect the break.
I'm expecting to be able to check the error flags and be able to read from the buffer if no error has occured.

Seems that Line Break Detection is missing from uart_stm32_err_check;
I'm not sure about the reason why it is missing but we can try to fix this.
I can't test this myself right now, but maybe you can have a try to add the following, in drivers/uart_stm32.c, in function ``, just after the framing error detection and before clearing errors (here):
	if (LL_USART_IsActiveFlag_LBD(UartInstance)) {
		err |= UART_BREAK;
	}

	if (err & UART_BREAK) {
		LL_USART_ClearFlag_LBD(UartInstance);
	}

Can you try this and see if fixing your point ?
Please note that by default, line break detection is set for 10-bit break. You'll need to make some changes for 11-bit break detection.

Hey thanks about your answer, the modification seems to detect Breaks but I would like to test if futher, I will update when I have more data.
Do you want me to do a PR in case?

Hey thanks about your answer, the modification seems to detect Breaks but I would like to test if futher, I will update when I have more data.
Do you want me to do a PR in case?

Thanks for the tests. Yes, don't hesitate to post a PR once you believe this is good to merge.

I kept an eye on it all day and it's now behaved as expected, I created a PR.

@andreacampanella  can you please check comment https://github.com/zephyrproject-rtos/zephyr/issues/41818#issuecomment-1016218393  to see if it is linked to your pb ?


How to use zoneIdRef/accountIdRef

Hey,
under my<EMAIL_ADDRESS>account I have several sites: foo.com, bar.com with their IDs 123 and 456.
Using zoneId: 123 in<EMAIL_ADDRESS>works fine, but I would not like to use such magic values and use the domains instead, e.g. foo.bar. I tried
zoneIdRef:
  name: foo.bar

but I get:
status:
  atProvider: {}
  conditions:
  - lastTransitionTime: "2024-04-03T21:47:43Z"
    message: 'cannot resolve references: mg.Spec.ForProvider.ZoneID: cannot get referenced
      resource: Zone.zone.cloudflare.upbound.io "foo.bar" not found'
    reason: ReconcileError
    status: "False"
    type: Synced

Same goes for accountId and accountIdRef. Could you help me, what the "search term" for the corresponding values would be?

Oh right, I remember its already on k8s/crossplabe level, not cloudflare.


Can't Get Storybook to work for components with external dependencies.

Hello guys,
Sorry this is my first issue raised ever hence I don't know the proper way to do it.
The problem i'm facing is with this repo :-
https://github.com/deepaksslibra/ever
I'm trying to setup storybook in that and it works for components which don't have external dependencies.
It works for SwitcherComponent but not for ModalsComponent for example.
Sorry, I have no experience customizing the webpack config.
I get this error:-

Steps to Reproduce :-

git clone   -c core.symlinks=true --recursive https://github.com/deepaksslibra/ever.git
git checkout develop
yarn bootstrap
cd admin/website-angular
yarn storybook

The code is in the 'develop' branch of the repository.

@igor-dv @shilman Guys any update please ? Sorry for being nosey but I'm in urgent need of help

@deepaksslibra https://github.com/deepaksslibra/ever/blob/develop/admin/website-angular/src/tsconfig.app.json why do you include files from node_modules directly?
in .storybook/tsconfig.json storybook uses ../src/tsconfig.app.json - it might be misconfigured. You should not need any custom webpack config for your setup as far as I can see
I immediately got permission errors (locally) after using your checkout command and doing git checkout develop 🤔
Can you create a fresh angular app + sb init and compare your configuration with the fresh installation?

@deepaksslibra https://github.com/deepaksslibra/ever/blob/develop/admin/website-angular/src/tsconfig.app.json why do you include files from node_modules directly?
in .storybook/tsconfig.json storybook uses ../src/tsconfig.app.json - it might be misconfigured. You should not need any custom webpack config for your setup as far as I can see
I immediately got permission errors (locally) after using your checkout command and doing git checkout develop
Can you create a fresh angular app + sb init and compare your configuration with the fresh installation?

Hello @kroeder.. I can't understand why If I load storybook for SwitcherComponent for example, it works perfectly fine.
I would also like to add that I haven't been working on this project from the start and have no idea why the tsconfig files are structured that way. I've been asked to implement Storybook into this existing codebase.
What is the most surprising factor for me is that the app itself runs without no issue i.e. if you do
yar run:admin the angular cli doesn't break or throw an error. It just runs.
I don't know how to work around the config files and how do they import modules and stuff. That's why I'm asking for help.


Improving storybook typing and helper functions

Type createPropProvider properly.
Get rid of PropProviders.compose, and just have createPropProvider
take in a list of SelectorMaps.
Rename PropProviders.Common() to CommonProvider, and have a new
Common variable that is a map (so that createPropProvider can be called
with Common and something else).
Make Avatar typing more strict.
Simplify WaitingButton.

r? @keybase/react-hackers


added tests for beNil and beNonNil

Just a start, doesn't prove that #312 aren't failing.
Has to be done on a higher level

Hey @supermarin, I think a lot has change since this was opened, so I'm going to go ahead and close this for now. Let me know if you think otherwise!


Encodes lists with associated and non-associated values

Not a completely proper since elements may be out of order. Restructuring of DreamList may be required.
Basically build a list of assoc. value we saw, and then iterate through values to put ones we didn't see yet.
Fixes #90

Can also add a method to DreamList along the lines of DreamList.ContainsAssocativeValue(val) if that implementation is preferred - don't know your preference.


Supports parallel refreshing of partitions when executing refresh external table's partition

Enhancement
you could modify HiveMetaCache#refreshPartition(HiveMetaStoreTableInfo hmsTable, List partNames)

cc @adzfolc


Remittance calculation change

As a (Student) I want The remittance calculation to apply the institution offering cost rules across my total application So That The remittance rules are applied correctly and I have enough funding throughout my study period.
Acceptance Criteria

[x] Change the Inputs into the Maximum Allowable Tuition Remittance Calculation such that:

The Offering costs used as input (Tuition + Books and Supplies + Mandatory Fees) is equal to the offering costs across the entire offering, whereas

The award amounts used as input is equal to the award amounts assigned to each specific disbursement

[x] If disbursement has not been issued, use requested amount, if eCert was sent, use actual amount remitted.

[x] Apply the rule change to allow for the offering costs calculation (Tuition +books and supplies) to be a total amount limit for all disbursements in a single application.

[x] Update E2E tests around remittance

Additional Context

Current formula = the lessor of (tuition + books and supplies (formerly program related costs)) and (Canada Grant + CSL + BCSL – amount already disbursed)
Current formulas are correct but are not being applied the correct way.
Current formula is remittance limit is the lesser of total disbursement award or total cost from the institution.
The error in this logic is that the cost calculation is being applied in isolation for each disbursement when there is multiple in the same application
The costs should be cumulative between the two disbursements in the same application

Example
Current State
Student application has 2 disbursements and total institution costs for tuition and books/supplies are $7000 on the offering. (Canada student grant + CSL+BCSL = $9000)
Disbursement 1 = $5000
Disbursement 2 = $4000
Remittance allowed
D 1 = $5000 (Formula = lesser of total costs or total disbursement) Costs = $7000 so it would max out at the disbursement amount and allow $5000
D 2 = $4000 (Formula would be the same as above because its not cumulative so once again it would max out with the full $4000
Changed version
D1 would use $5000 the same way as above however it would allocate $5000 of the $7000 costs. The total costs remaining would be $2000
D2 would only allow remittance of $2000 as that is the lessor of the amounts since only $2000 was left of the costs for the second disbursement.
This is required for PT release but needs to be done for both PT and FT
Technical

[ ] Adjust the method getMaxTuitionRemittance to also consider the value from sims.disbursement_schedules.tuition_remittance_effective_amount, if present, or sims.disbursement_schedules.tuition_remittance_requested_amount if the e-Cert was not generated yet (it can be done as variable1 ?? variable2 to pick the one available).
[ ] The change will affect the maximum amount the institution can request during the COE approval and also the tuition_remittance_effective_amount calculated during the e-Cert generation. In both situations, the method getMaxTuitionRemittance should receive the existing tuition remittance already calculated.
[ ] E2E tests

[ ] Ensure there is at least on scenario validations the value maxTuitionRemittanceAllowed from the API endpoint institutions/location/:locationId/confirmation-of-enrollment/disbursement-schedule/:disbursementScheduleId.
[ ] Expand the current COE confirmation tests to validate the maximum tuition including the tuition already requested in a previous disbursement.
[ ] Expand the current e-Cert tests to have a scenario to ensure maximum tuition including the tuition already requested in a previous disbursement.

@Joshua-Lakusta Talking to Andrew about this ticket we thought that the logic for max tuition remittance should be:
Math.min((offeringTotalCosts - previousTuitionRemittance), totalAwards)
instead of
Math.min(offeringTotalCosts, totalAwards) - previousTuitionRemittance
Could you please confirm?

I agree, after reviewing yours makes more sense. We are taking the min of the costs minus what they have remitted or the total award. Thanks for reaching out and move forward with your calculation.


Setup Puppet 4 testing for rspec and acceptance

Manual testing shows it all works, but rather have it automated :-)

Has been implemented with #395 and #398


:sparkles: feat(tabs): support brand icons

Summary
Goal is to support brand icons and a subtitle
Use Case

Preferred Solution
No response
Additional Information
No response
Will this feature introduce breaking changes?
None
Code of Conduct

[X] I agree to follow this project's Code of Conduct

/cib


Problem displaying the image in the select element

In fact I use the attribute "data-img-src" and I assign to it the source path of the image, but it does not take it and the image does not appear next to each value of the select.
If you can help me please, thanks :)

Can you include the snippet of the code used?

I have the same problem.
I use laravel framework
This is my html code

Stores
Products
logistics

I initialize chosen plugin like this:
$(".my-select").chosen();
Can you help me too

same issue, i also downloaded your example code, and installed it as it is in my website, and it doesn't load the images,
its your code without modifying it


solve option according to requirements

according to requirements, the option section should use if let, while let, not match.

Sorry, this is a repository for my learning. You can see the official repository here.


Fixes/pico graphics

fixes build for external projects using pico_graphics

I think my preferred solution would be to include "bitmap_fonts/bitmap_fonts.cmake" in the consuming project, but that's because I'm wary of this pattern becoming the norm- since it duplicates the whole library config in two places.
Another possible approach could be a root level include for pimoroni-pico which includes all libraries and dependencies, since only the ones that are linked into a project will be used anyway. Linking them individually is just busywork, I suppose!

Hmm, I almost opened ~the same PR (but with less duplication):
diff --git a/libraries/pico_graphics/pico_graphics.cmake b/libraries/pico_graphics/pico_graphics.cmake
index 67fdd79a..07cc65e5 100644
--- a/libraries/pico_graphics/pico_graphics.cmake
+++ b/libraries/pico_graphics/pico_graphics.cmake
@@ -1,4 +1,12 @@
-add_library(pico_graphics
+if(NOT TARGET bitmap_fonts)
+    include(${CMAKE_CURRENT_LIST_DIR}/../bitmap_fonts/bitmap_fonts.cmake)
+endif()
+
+if(NOT TARGET hershey_fonts)
+    include(${CMAKE_CURRENT_LIST_DIR}/../hershey_fonts/hershey_fonts.cmake)
+endif()
+
+add_library(pico_graphics
     ${CMAKE_CURRENT_LIST_DIR}/types.cpp
     ${CMAKE_CURRENT_LIST_DIR}/pico_graphics.cpp
     ${CMAKE_CURRENT_LIST_DIR}/pico_graphics_pen_1bit.cpp
@@ -13,4 +21,4 @@ add_library(pico_graphics

 target_include_directories(pico_graphics INTERFACE ${CMAKE_CURRENT_LIST_DIR})

-target_link_libraries(pico_graphics bitmap_fonts hershey_fonts pico_stdlib)
\ No newline at end of file
+target_link_libraries(pico_graphics bitmap_fonts hershey_fonts pico_stdlib)

... since I didn't have the best experience trying to do unicorn things:

Tried to include(libraries/galactic_unicorn/galactic_unicorn) -> compile error due to missing pico_graphics
Include that -> linker errors due to missing *_fonts
recreate dependency tree in my code and hope it doesn't change

Having a top-level "all the libraries" include would also work, maybe even have the import file include that. The extra include step per-library seems a bit weird compared to the pico-sdk...

@Daft-Freak I guess that would be sort-of our equivalent of Pico SDK's pico_sdk_init() and it makes sense.

Replaced by #686
Thank you both!


[Scenario] Zaginięcie Emmy Evans

 https://github.com/Pancernik90/Zagini-cie-Emmy-Evans
What kind of mission do you want to add (mark with an X, e.g. [X] Descent)?

[ x ] Mansions of Madness
[ ] Descent

What expansions does the scenario require?

[ x ] Conversion Kit (CK)
[ ] First Edition (MoM1E) - Also Recurring Nightmares
[ ] Fatal Alchemy (FA) - Also Suppressed Memories
[ ] Call of the Wild (CotW) - Also Suppressed Memories
[ x ] Beyond the Threshold (BtT)
[ ] Streets of Arkham (SoA)
[ ] Sunctum of Twilight (SoT)
[ ] Horrific Journeys (HJ)
[ ] Path of the Serpent (PotS)

What expansions can the scenario optionally use?

[ x ] Conversion Kit (CK)
[ ] First Edition (MoM1E) - Also Recurring Nightmares
[ ] Fatal Alchemy (FA) - Also Suppressed Memories
[ ] Call of the Wild (CotW) - Also Suppressed Memories
[ x ] Beyond the Threshold (BtT)
[ ] Streets of Arkham (SoA)
[ ] Sunctum of Twilight (SoT)
[ ] Horrific Journeys (HJ)
[ ] Path of the Serpent (PotS)

[Please enter the GitHub Url to your scenario here]
(https://github.com/Pancernik90/Zagini-cie-Emmy-Evans)

Hello, @Pancernik90 , thanks for creating a scenario and share it with the Valkyrie's community.
Could you edit the repository name to remove the special character "-" (hyphen), please. I will add the scenario to the manifest after that.

Of course, it's done.
If i want to make changes in scenario, or get some update from people, i can just delete files in repository and upload updated versions?
Just asking for the future.

Great, thanks, @Pancernik90
Yes, you can update the scenario directly. Be aware and do not change the path or the file names to prevent Valkyrie to link them.
The scenario has beed added to the manifest. It should become available in an hour or less.
By the way, I would like to translate the scenario into Spanish as soon as you have a "stable" version.
@antontimmermans , would you mind to add it to the statistics, please?

Added to statistics sheets


Using service definition to obtain symbol service url

OK, I confirm this did not regress behavior in our 3 test builds. We'll have to deploy it where we had the repro to confirm the new behavior works there.

We'll have to deploy it to the organization where we have the repro to confirm new behavior works

And the test failures about Npm are unrelated


Terminal spam on windows

🐛 Bug description
Output is spammed when running wasm-pack build in git bash on windows.

🤔 Expected Behavior
Less spam.
👟 Steps to reproduce
Run wasm-pack build in a project from git bash on windows.
🌍 Your environment
Include the relevant details of your environment.
wasm-pack version: 0.5.1
rustc version: rustc 1.32.0-nightly (e6c5cf923 2018-11-04)

See also https://github.com/rustwasm/wasm-pack/issues/298

@fitzgen Ah, I scanned for existing issues but didn't pick up on that one, cheers.

In addition to #298 I think this is covered by https://github.com/rustwasm/wasm-pack/issues/284 as well, so I'm going to close this in favor of those two issues


Revert "Restore ruby 1.8 compatibility"

Now that Ruby :one:.:eight: support has been dropped since #90, it's time to generate the templates with the new Hash syntax. :wink:

:)


Jobs dashboard first iteration implementation

This is a Feature branch for https://x-team-internal.atlassian.net/browse/XP-3352
Ticket:

https://x-team-internal.atlassian.net/browse/XP-3356 PR: https://github.com/x-team/xp-ui/pull/448

🚀


Add sql query command

This pr is mainly to add SQL query commands, while compatible with previous PQL command.
bin/pinot-admin.sh PostQuery -brokerHost localhost -brokerPort 8000 -queryType sql -query "select * from baseballStats limit 10"
bin/pinot-admin.sh PostQuery -brokerHost localhost -brokerPort 8000 -queryType pql -query "select * from baseballStats limit 10"
bin/pinot-admin.sh PostQuery -brokerHost localhost -brokerPort 8000 -query "select * from baseballStats limit 10"

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@f245b5e). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master    #4726   +/-   ##
=========================================
  Coverage          ?   64.66%
  Complexity        ?        4
=========================================
  Files             ?     1061
  Lines             ?    56616
  Branches          ?     8415
=========================================
  Hits              ?    36609
  Misses            ?    17288
  Partials          ?     2719

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f245b5e...3bcc176. Read the comment docs.

overall lgtm, some minor comments


Bump Circe + Akka Http version

This bumps the version of Circe to 0.8.0. We also bump the version of akka-http to 10.0.7 (the akka-http change shouldn't effect anyone as its binary compatible)
Probably should make a release as 3.4.0

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@cd95eeb). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master      #14   +/-   ##
=========================================
  Coverage          ?   85.71%
=========================================
  Files             ?        4
  Lines             ?       63
  Branches          ?        4
=========================================
  Hits              ?       54
  Misses            ?        9
  Partials          ?        0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cd95eeb...25ed3a4. Read the comment docs.

release just left sonatype as 3.4.0


Fix yii\web\Model validate method for unknown attributes for current scenario and for unknown attributes.

How it's a problem
Once we had a list of attributes in a defined model: $model.
class AnotherModel extends yii\base\Model { # It could be ActiveRecord model too.
       public $knownProperty
       public $anotherKownProperty;

       ## rules and another methods...
}

$model = new AnotherModel();
$model->knownProperty = 'valid value';
$model->anotherKownProperty = 'valid value';

$model->validate(['not known attribute']) # True, We are validating what? A unknown attribute?
$model->validate('not known attribute') # True, We are validating what? A unknown attribute?

We could put validate method in a unknown state. Returning true or false based on a not set property.
How it's fixed
We just have to check every attribute passed as first argument to yii\base\Model validate method and then check if there is a unknown attribute to be validated  throws a InvalidArgumentException to alert the developers to correct the incorrect usage.

Q
A

Is bugfix?
✔ We should not call validate() passing $attributeNames with wrong parameters.

New feature?
✔It allows the developers to quickly fix the wrong usage of validate() method.

Breaks BC?
❌ It breaks only the wrong usage of validate() method with wrong attributes name, it throws a InvalidArgumentException so the developer can fix it.

Tests pass?
✔ Added two more tests and the previous works just fine.

Fixed issues
It fix misunderstanding of the usage of validate().

If approved i would like to add further testing to lock this feature.

Hello?

While this might be helpful in some ways, I'm not sure if we should change the current behavior. Validation tells us if the model has any errors - validating not existing attribute is not adding any errors to the model.

The intention was to avoid confusion in the method usage. But if the change is too abrupt maybe it's not worth the effort.
Thank you for your time!


chore: use the new LH webui when reporting status updates

related to https://github.com/jenkins-x/lighthouse/pull/1324
instead of using the pipelines visualizer, let's use the new lighthouse webui and its "merge status" page
it might help people understand why keeper is not merging a PR

Hi @vbehar. Thanks for your PR.
I'm waiting for a jenkins-x or todo member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.
Once the patch is verified, the new status will be reflected by the ok-to-test label.
I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the jenkins-x/lighthouse repository.

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To complete the pull request process, please assign hferentschik
You can assign the PR to them by writing /assign @hferentschik in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

/ok-to-test

/lgtm

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: jstrachan
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [jstrachan]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


[chore] Prepare v1.25.0 release

Changes

Updates spec version from 1.26 to 1.31 (hence all the spec link changes)
Merges auto and manual changelogs

Merge requirement checklist

[x] CONTRIBUTING.md guidelines followed.
[x] Change log entry added, according to the guidelines in When to add a changelog entry.

If your PR does not need a change log, start the PR title with [chore]

[x] schema-next.yaml updated with changes to existing conventions.

If you're interested, I can run semantic-conventions-java against this branch and spot check it for any issues. Don't have time today, but can do it tomorrow.

If you're interested, I can run semantic-conventions-java against this branch and spot check it for any issues. Don't have time today, but can do it tomorrow.

@jack-berg that'd be great, thanks!
I run the new compat-check tool and here's the list of breaking changes I see.
(The tool will be part of CI after this release and we'll now allow them anymore).
$ docker run --rm -v c:/repo/semantic-conventions/model:/source -v  c/repo/semantic-conventions/docs:/spec  otel/semconvgen:0.24.0 -f /source --continue-on-validation-errors compatibility --previous-version 1.24.0

Found 8 compatibility issues:
        enum attribute member 'messaging.system.azure_eventgrid' was removed
        enum attribute member 'messaging.system.azure_eventhubs' was removed
        enum attribute member 'messaging.system.azure_servicebus' was removed
        metric 'http.client.request.time_in_queue' was removed
        metric 'messaging.deliver.duration' was removed
        metric 'messaging.deliver.messages' was removed
        metric 'system.processes.count' was removed
        metric 'system.processes.created' was removed

In theory we can fix them, but I believe no one (including Java) generates metrics yet, so I'd rather not deprecate them this last time.

If you feel strongly about keeping them, I could live with them being deprecated.
Let me know if you see any other incompatibilities or want to fix these ones.

Here's a draft of what the changes look like in semantic-conventions-java: https://github.com/open-telemetry/semantic-conventions-java/pull/61
Looks good to me!


gnomeExtensions.custom-command-list: patch hardcoded path to bash

should fix #361603
Things done

Details

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

In the upstream, it has been fixed from /bin/bash to /usr/bin/env bash.