fix(useBatteryIsLow): return battery level only if low

Description
Fixes #1346
The hook useBatteryIsLow returns the current battery level on initial rendering - even if the current battery level is not considered as low (Android < 0.15, iOS < 0.20);
With removing the initial value from the useEffect hook on initial rendering, the returned value remains null until the listener RNDeviceInfo_batteryLevelIsLow  detects the first callback for a low battery status.
Testing
On iOS simulator the battery level is always -1.
On Android emulator the battery level is always 1.
At least for me.
I therefore tested it on an external physical device too. See the video in the linked ticket above.
I did not find dedicated tests for the hooks. Let me know if I missed something.
Compatibility

OS
Implemented

iOS
✅

Android
✅

Windows
✅

Checklist

[x] I have tested this on a device/simulator for each compatible OS
[ ] I added the documentation in README.md
[ ] I updated the typings files (privateTypes.ts, types.ts)
[ ] I added a sample use of the API (example/App.js)

As as aside, this hook seems very weird. From just reading the name of it, I was assuming it would just spit-out a boolean rather than a number, but ¯\_(ツ)_/¯

I added a helper function which does the check again if the current battery level is actually low:

Android < 0.15
iOS/web < 0.2

If true the initial value is set to the state and returned by the hook.
If false the default return value remains null.
Please note that testing on iOS simulators is not really helpful as the battery level is always -1.
I did not add an additional check for that (e.g. return level > 0 && level < 0.2).
On Android emulators the battery level is always 1.


Fixes API error when learner is enrolled in a courserun with no related page

Pre-Flight checklist

[X] Testing

[X] Code is tested
[X] Changes have been manually tested

What are the relevant tickets?
Fixes #920
What's this PR do?
Wraps the check for a courserun's related course page in a try/except - if the learner is enrolled in a course run that doesn't have a page (like, for instance, a proctored exam), the check for the page will end up throwing a RelatedObjectDoesNotExist exception.
How should this be manually tested?
Create and enroll in a course that does not have a CMS page. Then, view the dashboard. You should see course cards as per usual and you should not see the invite to enroll in courses.

Codecov Report

Merging #933 (eb6f6a5) into main (18d212a) will decrease coverage by 0.05%.
The diff coverage is 25.00%.

@@            Coverage Diff             @@
##             main     #933      +/-   ##
==========================================
- Coverage   84.50%   84.44%   -0.06%
==========================================
  Files         139      139
  Lines        6600     6603       +3
  Branches     1109     1109
==========================================
- Hits         5577     5576       -1
- Misses        841      846       +5
+ Partials      182      181       -1

Impacted Files
Coverage Δ

courses/serializers.py
95.26% <25.00%> (-1.12%)
:arrow_down:

ecommerce/discounts.py
78.94% <0.00%> (-3.51%)
:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Use new Data api to avoid coalescing operator and optionals

Warnings

:warning:

Any changes to library code should be reflected in the Changelog.
Please consider adding a note there and adhere to the Changelog Guidelines.

  Generated by :no_entry_sign: Danger Swift against 1e645715f36a348fafa27198783aa7ebadec1ae4


Simplify implementing GRXWriters

The code to handle state and properly nillify the GRXWriteable instance should be encapsulated in the GRXWriter superclass.
Depends on #2113.

Depends on #2109 too.

The four subclasses that can be simplified are GRXBufferedPipe, GRPCCall, GRXForwardingWriter, and GRXImmediateWriter.

-setState: can go to the superclass, and the subclasses hook in particular behavior by overriding -pause, -resume and -stop methods (implemented as noops in the superclass).
-startWithWriteable: and -finishWithError: can be implemented in the superclass with the code to manipulate _state and _writeable. A subclass can override the method with a call to the superclass. For example, GRPCCall can pass the superclass a wrapped writeable.
-finishOutputWithError: from GRXForwardingWriter could be useful for the other classes. Same as -start from ProtoRPC.

For future-proofing the API, I guess we'll want to do all this in a new GRXBaseWriter class, to leave GRXWriter pure abstract.
Beware that, for GRPCCall, we want the ability to cancel even if the completion has already been received and enqueued, so we don't release the writeable wrapper on finish, but when the completion has been delivered to the wrapped writeable.

Beta? Seems like an API finalization issue.

It's introducing a class to simplify the implementation of some others; it won't break any interfaces.

Too old an enhancement. closing.


[iOS] Launch on iOS is stuck the first time you run a project

Environment
Provide version numbers for the following components (information can be retrieved by running tns info in your project folder or by inspecting the package.json of the project):

CLI: 5.3.0
Cross-platform modules: 5.3.1
Android Runtime: 5.3.0
iOS Runtime: 5.3.0
VSCode: 1.32.3

Describe the bug
VS Code is stuck on launch on iOS the first time you run a project.
To Reproduce

Execute tns create anywhere and create a project.
Open VS Code with the project.
Configure launch.json for NativeScript
Click Launch on iOS.

The output is stuck at Running tns command....
Expected behavior
The app launches on iOS emulator/device.
Additional context
Restarting VS Code fixes the issue. The issue doesn't reproduce on Android.

related to https://github.com/NativeScript/nativescript-vscode-extension/issues/239


chore: highlight command print outs

Description
Solves: https://github.com/superfaceai/cli/issues/345
Motivation and Context

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[ ] I have updated the documentation accordingly. For updating Oclif commands documentation use oclif-dev.
[ ] I have read the CONTRIBUTING document.
[ ] I haven't repeated the code. (DRY)
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

I would like to take a look on this as well. E.g. the underline can evoke a clickable text.
We will have a discussion with Z and Martyn tomorrow so maybe even more feedback can be done here and probably I will have some spare time to take over and finish it if you are fine with that.

This is how it looks like for prepare
@julpat If you have time and ideas how to make this better you can take over this PR (or open new with more general changes)

Finally got to this (and have the repository running & explored, so next pair of hands for this kind of stuff).

Removed the underline to avoid confusion
Added missing command highlighting
Point out more at the execute command

For the purpose of this PR (and the issue), it seems sufficient to me.
There will be more changes (copy / ux) in the near future, so let's not block this one.

Looks OK, what do you think @kysely ?

I think it's still verbose but I'm fine either way :)


Migrate Flutter projects to remove ENABLE_BITCODE=YES

Ensure Flutter apps do not have ENABLE_BITCODE turned on.  It it is on, then give the user a warning and remove the build settings.  This will allow bitcode to be removed from the tool.
New migration would look a lot like:
https://github.com/flutter/flutter/blob/7f17098f97f5b2642ca405a400df93cd01079e40/packages/flutter_tools/lib/src/ios/migrations/ios_deployment_target_migration.dart#L66-L69
like:
  @override
  String? migrateLine(String line) {
    if (line.contains('ENABLE_BITCODE = YES;')) {
      if (!migrationRequired) {
        // Only print for the first discovered change found.
        logger.printWarning('Disabling deprecated bitcode. See https://github.com/flutter/flutter/issues/107887 for additional details.');
      }
      return null;
    }

    return line;
  }

Part of https://github.com/flutter/flutter/issues/107883

Also should only run if >Xcode 14 is installed.  We don't want to break an Xcode 13 watchOS submission somehow.


Add time boundation in the game

If there is no time limit then the user will get infinite time to solve the anagram. At least show time taken by user to solve the puzzle in the last.

solved the isuue.


Light Brush EEF Development

This issue is being created to track the integration of a dedicated light painting EEF which can be deployed on different robotic arms for demonstration.
Initial development should focus on deployment to the Motoman MH5L robot flange.

Summer 2022 Intern Project Update
AIMS sponsored intern began prototyping new EEF hardware.

EEF Integration Testing (Ver 4 - left) (Ver 2 - right)
Final Prototype (V4)Unpainted

Some initial testing was performed to characterize variables like exposure time and LED placement for the design. This analysis is incomplete.


Added new function for using HTTPS when checking to see if a bucket e…

Added a new function to see if a bucket exists, uses HTTPS rather than HTTP.
Motivation and Context
Security group won't allow HTTP protocol to validate S3 bucket.
Testing
Sadly I didn't see any unit tests for the original function (DoesS3BucketExistAsync) so I did not add any that are specific to this.
Screenshots (if appropriate)
N/A
Types of changes

[X] Bug fix (non-breaking change which fixes an issue)
[X] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist

[X] My code follows the code style of this project
[X] My change requires a change to the documentation
[ ] I have updated the documentation accordingly
[X] I have read the README document
[ ] I have added tests to cover my changes
[ ] All new and existing tests passed

License

[x] I confirm that this pull request can be released under the Apache 2 license

Resolves issue https://github.com/aws/aws-sdk-net/issues/1107. I hope.

Why do we need a new function? I think need to fix the existing one.

@dermeister0 Reason V2 was added is due to this comment: https://github.com/aws/aws-sdk-net/issues/1107#issuecomment-437535437

@dermeister0 Let me know if that change is acceptable or not. Thanks!

Hello all, any update on when this will be merged to aws:master and a new SDK package released to nuget?

Hello,
AWSSDK.S3 version <IP_ADDRESS> contains a DoesBucketExistsV2 version which addresses this issue (https://github.com/aws/aws-sdk-net/commit/208377db24731da445d23da7dd8bea60e33a4f17).


Integrate new popup menu (update to<EMAIL_ADDRESS>

What should we do?
Update to<EMAIL_ADDRESS>and make the adjustments necessary for the new popup menu.
Why should we do it?
To ensure the look of the color picker works with the new popop menu UI.

Closed via https://github.com/bpmn-io/bpmn-js-color-picker/pull/14.


Add Server Instance Param to New-BCContainer

Describe the issue
Would be nice to be able to set the instance name rather than being forced to use BC. Helpful when using multiple containers in a vm.

The only place where this would be needed is if you have multiple service tiers in one container - and that is not supported.
If you have multiple containers on a VM you would need to distinguish them (container name, IP address or using traefik).
But... - since NAV containers use NAV and BC containers use BC - it is possible.
If you share a file called ServiceSettings.ps1 in -myscripts with this format:
https://github.com/microsoft/nav-docker/blob/master/generic/Run/150-new/ServiceSettings.ps1

then that should be used.

Thanks Freddy. I'll close this issue.
I'm using multiple containers, but find it easier if the service names differ.


WIP: bump dependencies to align with cdi ginkgo test  framework

we want to use ginko testing framework ,
we want to reuse some of the CDI framework from https://github.com/kubevirt/containerized-data-importer/tree/main/tests/framework

@eslutsky the changes probably conflicted with those in #52


Feature: additional actions/links to be renderd in sink Findings

Is your feature request related to a problem?
Most of our alerts are enriched with additional annotations like dashboard_url and runbook_url to make debugging and resolving issues faster. Not everything can be automated, sadly...
Those annotations were previously used to render slack actions/links to the specified hyperlinks.
Describe the solution you'd like
Passing additional (external) links to the (slack_)sink configs which are then rendered when present on the alert.
Describe alternatives you've considered
Currently, only robusta-provided links to the UI are configured: https://github.com/robusta-dev/robusta/blob/master/src/robusta/integrations/slack/sender.py#L282-L303
Additional context
The example I am talking about is described here.

Thanks for reporting it @Dariusch
How would you configure those links?
Does all alerts has those links? (dashboard_url, runbook_url)
Would you want to configure it per alert?
Or, it's something that converts given annotations to buttons, if those are available?

Not all of the alerts have those links.

Or, it's something that converts given annotations to buttons, if those are available?
Yes

For example the PrometheusRules configured by the kube-prometheus-stack do not have those annotations.
A flux alert could link to a dashboard for example:
---
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: flux-rules
  namespace: monitoring
spec:
  groups:
    - name: flux
      rules:
        - alert: ReconciliationFailure
          expr: max(gotk_reconcile_condition{status="False",type="Ready"}) by (exported_namespace, name, kind) + on(exported_namespace, name, kind) (max(gotk_reconcile_condition{status="Deleted"}) by (exported_namespace, name, kind)) * 2 == 1
          for: 3m
          labels:
            severity: warning
            team: engineering
          annotations:
            summary: '{{ $labels.kind }} {{ $labels.exported_namespace }}/{{ $labels.name }} reconciliation failed'
            description: '{{ $labels.kind }} {{ $labels.exported_namespace }}/{{ $labels.name }} reconciliation failed for more than 10 minutes'
            dashboard_url: https://grafana.${domain}/d/flux-cluster/flux-cluster-stats?orgId=1&refresh=30s

Or add the runbook URL.
Annotation and labels are a kv map.
I guess a proposed solution would be to add a list to the slack sink config which is mutated into the slack schema like
- slack_sink:
    # Other values
    actions:
      - type: button
        text: 'Runbook :green_book:'
        url: '{{ (index .Alerts 0).Annotations.runbook_url }}'

See also #735

FYI, we made some improvements around including links in Slack messages here: https://github.com/robusta-dev/robusta/pull/967
It doesn't handle existing links from annotations/labels on the alert. But we're thinking about expanding this to your use case, as well as showing links from annotations in the Robusta UI.


Implemented test case for biceps:R0055_0

Checklist
The following aspects have been respected by the author of this pull request, confirmed by both pull request assignee and reviewer:

Adherence to coding conventions

[x] Pull Request Assignee
[ ] Reviewer

Adherence to javadoc conventions

[x] Pull Request Assignee
[ ] Reviewer

Changelog update (necessity checked and entry added or not added respectively)

[x] Pull Request Assignee
[ ] Reviewer

README update (necessity checked and entry added or not added respectively)

[x] Pull Request Assignee
[ ] Reviewer

config update (necessity checked and entry added or not added respectively)

[x] Pull Request Assignee
[ ] Reviewer

SDCcc executable ran against a test device (if necessary)

[x] Pull Request Assignee
[ ] Reviewer

Add a proper Description

Add a proper Description

Removed the line "the headline sais it all", which is indeed superfluous.


Update analytics endpoints

Remove Fathom, and update Matomo URL.
Related Commits:

https://github.com/bcgov/aries-vcr/commit/292ffe85661630f906c2faa31a34fa8c17d76be1#diff-210b772aef140a9affc38cff52efb7f876a14f4287a1835bf8e65a0f2797677f
https://github.com/bcgov/aries-vcr/commit/16fb5f59021a29a10d716ce7b779d8ee1fdd4cab#diff-210b772aef140a9affc38cff52efb7f876a14f4287a1835bf8e65a0f2797677f

@esune I believe Matomo is now pointing to the right URL, are we still using Fathom or can this issue be closed?

From a cursory search in index.html I don't see any references to Fathom anymore.


IE11 Limitations

After https://github.com/PolymerLabs/lit-html/pull/112 or something similar gets merged IE11 works pretty nice :)
Currently there are some limitations

Last node can't be a node with a variable

// works in IE11
return html`

${header}

ending

`;

// does not work in IE11
return html`

${header}

`;

You need a polyfill for template tag and ES6 Symbols
=> so this should be somewhere noted

So far no ShadyCSS support

I think we have multiple issues open on IE now, we can just keep one.


Appcast: Error: undefined local variable or method `cask_tokens'

General troubleshooting steps

[x] I have checked the instructions for reporting bugs (or making requests) before opening the issue.

[x] None of the templates was appropriate for my issue, or I’m not sure.

[x] I ran brew update-reset && brew update and retried my command.
[x] I ran brew doctor, fixed as many issues as possible and retried my command.
[x] I understand that if I ignore these instructions, my issue may be closed without review.

Description of issue
appcast errors after updating with https://github.com/Homebrew/brew/pull/2654
This also causes an error with cask-repair
Output of your command
brew cask _appcast_checkpoint --calculate https://git-fork.com/update/feed.xml
Error: undefined local variable or method `cask_tokens' for #<Hbc::CLI::InternalAppcastCheckpoint:0x007fa1c92be748>
Follow the instructions here:
  https://github.com/caskroom/homebrew-cask#reporting-bugs
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli/internal_appcast_checkpoint.rb:12:in `run'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli/abstract_command.rb:38:in `run'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:103:in `run_command'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:154:in `run'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:137:in `run'
/usr/local/Homebrew/Library/Homebrew/cmd/cask.rb:8:in `cask'
/usr/local/Homebrew/Library/Homebrew/brew.rb:93:in `<main>'

brew cask _appcast_checkpoint --calculate https://www.wireshark.org/download/osx/
Error: undefined local variable or method `cask_tokens' for #<Hbc::CLI::InternalAppcastCheckpoint:0x007fdc6820a638>
Follow the instructions here:
  https://github.com/caskroom/homebrew-cask#reporting-bugs
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli/internal_appcast_checkpoint.rb:12:in `run'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli/abstract_command.rb:38:in `run'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:103:in `run_command'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:154:in `run'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:137:in `run'
/usr/local/Homebrew/Library/Homebrew/cmd/cask.rb:8:in `cask'
/usr/local/Homebrew/Library/Homebrew/brew.rb:93:in `<main>'

Output of brew cask doctor
==> Homebrew-Cask Version
Homebrew-Cask 1.2.1-71-gd1f599f1
caskroom/homebrew-cask (git revision 6ee23; last commit 2017-05-23)
==> Homebrew-Cask Install Location
<NONE>
==> Homebrew-Cask Staging Location
/usr/local/Caskroom
==> Homebrew-Cask Cached Downloads
~/Library/Caches/Homebrew/Cask
==> Homebrew-Cask Taps:
/usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask (3616 casks)
/usr/local/Homebrew/Library/Taps/caskroom/homebrew-drivers (97 casks)
/usr/local/Homebrew/Library/Taps/caskroom/homebrew-eid (7 casks)
/usr/local/Homebrew/Library/Taps/caskroom/homebrew-fonts (1106 casks)
/usr/local/Homebrew/Library/Taps/caskroom/homebrew-versions (161 casks)
==> Contents of $LOAD_PATH
/usr/local/Homebrew/Library/Homebrew/cask/lib
/usr/local/Homebrew/Library/Homebrew
/Library/Ruby/Site/2.0.0
/Library/Ruby/Site/2.0.0/x86_64-darwin16
/Library/Ruby/Site/2.0.0/universal-darwin16
/Library/Ruby/Site
/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/vendor_ruby/2.0.0
/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/vendor_ruby/2.0.0/x86_64-darwin16
/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/vendor_ruby/2.0.0/universal-darwin16
/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/vendor_ruby
/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0
/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/x86_64-darwin16
/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/universal-darwin16
==> Environment Variables
LANG="en_AU.UTF-8"
LC_CTYPE="en_AU.UTF-8"
PATH="/usr/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/Homebrew/Library/Taps/homebrew/homebrew-bundle/cmd:/usr/local/Homebrew/Library/Homebrew/shims/scm"
SHELL="/usr/local/bin/zsh"

@reitermarkus Thanks!


[Feature Request] Add to playback queue

There is a queue endpoint in the Web API.
It's currently still in beta, but it would be nice to have.
https://developer.spotify.com/documentation/web-api/reference/player/add-to-queue/

Added in 307d69945e59d28ecbee87a87634bf070e5f2d8c
Released in 5.1.0


Issue with k8s.io/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm/

This is a Bug Report

Problem:
Proposed Solution:
Page to Update:
https://kubernetes.io/...

k8s节点加入集群的时候出现这个报错是怎么回事呢

hi, please log an issue in the kubernetes/kubernetes repository and follow the issue template.
the version of the kubelet has to be specified and also all the details about your setup!
thanks.
/close


chore: fix spellcheck

Related issue(s)

Resolves #
Description
Summary of changes
I've addressed any CSpell warnings which kept popping up in PRs.
Dependency additions / changes
N/A
Test additions / changes
N/A
Checklist

[x] I have tested the changes locally.
[x] I have formatted the changes with Prettier and/or cargo fmt with default settings.
[ ] I have linked this PR to the issue(s) that it resolves.
[x] I have reviewed the changes on GitHub, line by line.
[x] I have ensured all changes are covered in the description.
[ ] This PR requires documentation updates when merged.

Additional context

@kevaundray Looks like we ended up making the same PR at the same time. This one addresses some extra issues as well (intrinsics/Guillaume).

Yep, will close mine


HDFS includes and libraries NOT found when compiling spark xgboost with USE_HDFS on

The error info is as follow:
-- Architecture: x64
-- HDFS_LIB_PATHS: /lib/native
CMake Error at dmlc-core/cmake/Modules/FindHDFS.cmake:64 (message):
HDFS includes and libraries NOT found.(HDFS_INCLUDE_DIR-NOTFOUND,
HDFS_LIB-NOTFOUND)
Call Stack (most recent call first):
dmlc-core/CMakeLists.txt:32 (find_package)
But I can't find any libhdfs anywhere

Did you install Apache Hadoop?

Yes. I had. Hadoop and HDFS path are as follows: /usr/local/Cellar/hadoop/3.1.1//bin/hdfs; /usr/local/Cellar/hadoop/3.1.1//bin/hadoop . But I can't find hdfs lib path. Could you tell me how to set or compile hdfs ? Thank you very much.

@goodturing Did you use Homebrew to install Hadoop? I tried it and the Homebrew formula doesn't seem to include libhdfs. For best results, please download the source from Apache website and compile.

@goodturing For your information, here are the steps for compiling from the source:
# Compile HDFS native library from source
cd
wget http://mirrors.advancedhosters.com/apache/hadoop/common/hadoop-3.1.1/hadoop-3.1.1-src.tar.gz
tar xvf hadoop-3.1.1-src.tar.gz
cd hadoop-3.1.1-src/hadoop-hdfs-project/hadoop-hdfs-native-client
CC=gcc-8 CXX=g++-8 mvn compile -Pnative   # Use Homebrew GCC

# Create HDFS home under the home directory
cd
mkdir hdfs_home
cd hdfs_home
mkdir -p lib/native
# Copy libhdfs.so
cp -rv ~/hadoop-3.1.1-src/hadoop-hdfs-project/hadoop-hdfs-native-client/target/native/target/usr/local/lib/ ./lib/native/
# Copy hdfs.h
cp -rv ~/hadoop-3.1.1-src/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/include/hdfs/ ./include/

# Now build XGBoost with libhdfs
cd /path/to/xgboost
mkdir build
cd build
HADOOP_HDFS_HOME=${HOME}/hdfs_home CC=gcc-8 CXX=g++-8 cmake .. -DUSE_HDFS=ON
make

Yes, I installed hadoop by brew. Then I fall back to Hadoop 2.7.7. This problem has been solved. But I encountered another problem.
ld: warning: ignoring file /WKS/hadoop-2.7.7/lib/native/libhdfs.a, file was built for archive which is not the architecture being linked (x86_64): /WKS/hadoop-2.7.7/lib/native/libhdfs.a
Undefined symbols for architecture x86_64:
"_hdfsCloseFile", referenced from:
dmlc::io::HDFSStream::~HDFSStream() in libdmlc.a(hdfs_filesys.cc.o)
dmlc::io::HDFSStream::~HDFSStream() in libdmlc.a(hdfs_filesys.cc.o)
"_hdfsConfGetStr", referenced from:
dmlc::io::FileSystem::GetInstance(dmlc::io::URI const&) in libdmlc.a(io.cc.o)
"_hdfsConnect", referenced from:
dmlc::io::HDFSFileSystem::HDFSFileSystem(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) in libdmlc.a(hdfs_filesys.cc.o)
dmlc::io::HDFSFileSystem::ResetNamenode(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) in libdmlc.a(hdfs_filesys.cc.o)
"_hdfsDisconnect", referenced from:
dmlc::io::HDFSFileSystem::~HDFSFileSystem() in libdmlc.a(hdfs_filesys.cc.o)
dmlc::io::HDFSFileSystem::~HDFSFileSystem() in libdmlc.a(hdfs_filesys.cc.o)
dmlc::io::HDFSFileSystem::~HDFSFileSystem() in libdmlc.a(hdfs_filesys.cc.o)
dmlc::io::HDFSFileSystem::ResetNamenode(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) in libdmlc.a(hdfs_filesys.cc.o)
dmlc::io::HDFSStream::~HDFSStream() in libdmlc.a(hdfs_filesys.cc.o)
dmlc::io::HDFSStream::~HDFSStream() in libdmlc.a(hdfs_filesys.cc.o)
"_hdfsFreeFileInfo", referenced from:
dmlc::io::HDFSFileSystem::GetPathInfo(dmlc::io::URI const&) in libdmlc.a(hdfs_filesys.cc.o)
dmlc::io::HDFSFileSystem::ListDirectory(dmlc::io::URI const&, std::vector<dmlc::io::FileInfo, std::allocatordmlc::io::FileInfo >) in libdmlc.a(hdfs_filesys.cc.o)
"_hdfsGetPathInfo", referenced from:
dmlc::io::HDFSFileSystem::GetPathInfo(dmlc::io::URI const&) in libdmlc.a(hdfs_filesys.cc.o)
"_hdfsListDirectory", referenced from:
dmlc::io::HDFSFileSystem::ListDirectory(dmlc::io::URI const&, std::vector<dmlc::io::FileInfo, std::allocatordmlc::io::FileInfo >) in libdmlc.a(hdfs_filesys.cc.o)
"_hdfsOpenFile", referenced from:
dmlc::io::HDFSFileSystem::Open(dmlc::io::URI const&, char const*, bool) in libdmlc.a(hdfs_filesys.cc.o)
"_hdfsRead", referenced from:
dmlc::io::HDFSStream::Read(void*, unsigned long) in libdmlc.a(hdfs_filesys.cc.o)
"_hdfsSeek", referenced from:
dmlc::io::HDFSStream::Seek(unsigned long) in libdmlc.a(hdfs_filesys.cc.o)
"_hdfsTell", referenced from:
dmlc::io::HDFSStream::Tell()    in libdmlc.a(hdfs_filesys.cc.o)
"_hdfsWrite", referenced from:
dmlc::io::HDFSStream::Write(void const*, unsigned long) in libdmlc.a(hdfs_filesys.cc.o)
ld: symbol(s) not found for architecture x86_64
collect2: error: ld returned 1 exit status
make[2]: *** [../xgboost] Error 1
make[1]: *** [CMakeFiles/runxgboost.dir/all] Error 2
make: *** [all] Error 2
building Java wrapper
cd ..
mkdir -p build
cd build
cmake .. -DUSE_OPENMP:BOOL=OFF -DUSE_HDFS:BOOL=ON -DUSE_AZURE:BOOL=OFF -DUSE_S3:BOOL=ON -DPLUGIN_UPDATER_GPU:BOOL=OFF -DJVM_BINDINGS:BOOL=ON
cmake --build . --config Release
Traceback (most recent call last):
File "create_jni.py", line 89, in
run("cmake --build . --config Release")
File "create_jni.py", line 51, in run
subprocess.check_call(command, shell=True, **kwargs)
File "/anaconda3/lib/python3.7/subprocess.py", line 328, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'cmake --build . --config Release' returned non-zero exit status 2.
[ERROR] Command execution failed.
org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)

Did you compio Hadoop 2.7.7 from source? Don't use the binary tarball, since it contains Linux binaries and doesn't work with Mac OSX.
As an aside, if you plan to deploy your application to a Spark cluster, it may be best to use a Linux machine instead.

Yes I used binary tarball directly. I am used to view/edit codes on mac, Maybe I should use linux machine. Thanks a lot.


Create codeql-analysis.yml

Initial commit of CodeQL

Removed Autobuild temporarily until we actually get files up

Yeah, we'll chill on this until we actually have code.


Use rust 1.82 and bump build number.

Destination channel: defaults
Links

PKG-6207

Explanation of changes:

Bump build number and build with rust 1.82

Rust < 1.81 was affected by CVE-2024-43402.

Linter check found the following problems:
ERROR conda.cli.main_run:execute(125): `conda run conda-lint /tmp/abs_1f2ul_krzl/clone` failed. (See above for error)
The following problems have been found:
===== WARNINGS =====

clone/recipe/meta.yaml:60: invalid_url: https://ypy.readthedocs.io : Not reachable: 403
===== Final Report: =====
0 Errors and 1 Warning were found

Linter check found the following problems:
ERROR conda.cli.main_run:execute(125): `conda run conda-lint /tmp/abs_2bl7m8sg38/clone` failed. (See above for error)
The following problems have been found:
===== WARNINGS =====

clone/recipe/meta.yaml:59: invalid_url: https://ypy.readthedocs.io : Not reachable: 403
===== Final Report: =====
0 Errors and 1 Warning were found

Linter check found the following problems:
ERROR conda.cli.main_run:execute(125): `conda run conda-lint /tmp/abs_12br8pvf9q/clone` failed. (See above for error)
The following problems have been found:
===== WARNINGS =====

clone/recipe/meta.yaml:59: invalid_url: https://ypy.readthedocs.io : Not reachable: 403
===== Final Report: =====
0 Errors and 1 Warning were found


Internet issue.

In case there is an internet issue, the script fails and gives nasty error log. Fixed it with a small understandable sentence 'Please check your internet connection'

I had to modify the script to just spit a list of urls and then download images from the list using curl to overcome this issue. Perhaps this should be the default behaviour?

@NoSkillGuy thanks for your PR. can you please rebase this PR? when i merged a couple other pull requests, this went into conflicting mode.

closing this in favor of #138


Add deprecation warnings for renamed commands

Describe the feature
In #3554, we renamed source snapshot-freshness -> source freshness. Both commands are supported and no deprecation warning is showed. It would be ideal to show deprecation warnings when using the old command.
Describe alternatives you've considered
Support renamed commands forever via an alias and don't display any deprecation warnings. I'm perfectly happy with this answer as well, I'd happily close this issue if that is acceptable.

I agree! Let's do both: continue support source snapshot-freshness as an alias forever*, and raise a deprecation warning when it's used.
*At least until dbt v2.0, if not longer :)


Hashing the "same" individual produces different values

GEDCOM parser for Python version: 0.2.2
Python version: 3.6.2
Operating System: Ubuntu 16.04

Description
I'm trying to put individuals (i.e. ged4py.model.Individual instances) into hash-based data structures (dicts, sets). However, I'm running into problems because an individual's hash changes depending on how I access the individual.
What I Did
#!/usr/bin/env python

from io import BytesIO

from ged4py.parser import GedcomReader

GEDCOM = '''0 HEAD
1 GEDC
2 VERS 5.5.1
2 FORM LINEAGE-LINKED
1 CHAR UTF-8
1 LANG English
0 @I0000@ INDI
1 NAME Jack /Doe/
2 GIVN Jack
2 SURN Doe
1 FAMC @F0000@
0 @I0001@ INDI
1 NAME John /Doe/
2 GIVN John
2 SURN Doe
1 FAMS @F0000@
0 @F0000@ FAM
1 HUSB @I0001@
1 CHIL @I0000@
0 TRLR'''

if __name__ == '__main__':
    f = BytesIO(GEDCOM.encode('utf-8'))
    with GedcomReader(f, encoding='utf-8') as parser:
        for indi in parser.records0('INDI'):
            print(indi)
            print(f'Hash: {hash(indi)}')
            if indi.father:
                print(f'Father: {indi.father}')
                print(f'Father Hash: {hash(indi.father)}')
            print()

For me, this produces the following output:
Individual(level=0, xref_id=@I0000@, tag=INDI, value=None, offset=77, #subrec=2)
Hash: -9223363292767093237
Father: Individual(level=0, xref_id=@I0001@, tag=INDI, value=None, offset=148, #subrec=2)
Father Hash: -9223363292767093125

Individual(level=0, xref_id=@I0001@, tag=INDI, value=None, offset=148, #subrec=2)
Hash:<PHONE_NUMBER>729

The output shows that (in my case) the father's hash is<PHONE_NUMBER>729 when accessed "directly" (i.e. via the return value from parser.records0('INDI'), but -9223363292767093125 when accessed via the son's father attribute.
In my opinion, both of these Individual instances refer to the same person, and should therefore have the same hash. My example is for individuals, but I would expect this to be true for families, and probably any other record that has an xref_ID.

Thanks for report. It's a complicated issue and I don't think there is an acceptable solution for this. To have the same hash value objects also have to be compared as equal. I do not think there is a way to define equality rules for GEDCOM records in general. As an example individuals "John Smith" and "John Smith" could be two different individuals even though they have the same name, date of birth, etc. Potentially one could use record @XREF_ID@ to identify the and compare records but that assumes that records come from the same file and this assumption cannot be verified in general. I guess my mistake was not to disable hash for the records, in general records are mutable structures and mutable objects should not provide hash method. I will fix that mistake and disable hash for classes that are mutable or have no reasonable comparison method.
What that means for you is that you won't be able to hash Individuals at all. But you can hash xref_id attribute of the individual, again only if you are comparing records from the same file, this will not work across multiple files (or if you try to create record instance yourself).

These are valid points, and perhaps disabling hashing completely is indeed the best approach. A compromise might be to enable hashing and equality-checks only for records with an xref_id, but I'm not sure whether the added complexity (for you and the users) would be worth it. I'm already using the xref_id as a workaround.
I'll keep the ticket open, feel free to close it once it suits you.

Thanks for feedback. I have just published version 0.2.3, it disables hashing for Record types. I also took this opportunity to think about hashing/ordering for other classes and found that date classes are not quite consistent in that respect. I added hashing method to DateValue and CaendarDate classes and improved ordering for DateValue classes too.


Context
https://github.com/bazelbuild/rules_go is an alternative way to build Go application using Bazel. It does so by manually instrumenting the go tools such as go tool compile or go tool link in hermetic Bazel sandboxes. The intermediate artifacts is then cached by Bazel for subsequent reuse.
What I did
For the Compile step, we are currently calling go tool compile with both -o <compile-output>.x and -linkobj <link-output>.a. The .x file is then used for subsequent downstream package compilations while the .a file will be used for downstream executable linkings.
Here is the relevant code: https://github.com/bazelbuild/rules_go/blob/8b8294b6359a9c0a89af968fb7092f6e7194f420/go/tools/builders/compilepkg.go#L494-L495
What did you see happen?
When I upgraded gorm.io/driver/mysql, a direct dependency of db.go, to a newer version, I noticed that the new db.x was changed.
The binary diff between 2 .x files are just a few bytes small
> diffoscope ~/Downloads/db*
--- /Users/sluongng/Downloads/db (1).x
+++ /Users/sluongng/Downloads/db.x
│┄ Command `'nm -s {}'` failed with exit code 1. Standard output:
│┄     /Library/Developer/CommandLineTools/usr/bin/nm: error: for the -s option: bad number of arguments (must be two arguments)
│┄     /Library/Developer/CommandLineTools/usr/bin/nm: error: a.out: No such file or directory
@@ -34357,15 +34357,15 @@
 00086340: 0104 0203 0405 1000 8108 0082 0803 7b03  ..............{.
 00086350: 4303 0103 0203 4103 8001 0324 034a 031b  C.....A....$.J..
 00086360: 03d4 0303 d503 03dc 0303 d803 0081 080f  ................
 00086370: 010d 0203 0405 0607 0809 0a0b 0c0d 0e0b  ................
 00086380: 0083 0800 8408 0343 034a 03d4 0303 d503  .......C.J......
 00086390: 03d8 0303 da03 0341 0365 0083 080a 0108  .......A.e......
 000863a0: 0203 0405 0607 0809 1200 8508 0086 0803  ................
-000863b0: bd01 03d8 0403 d403 037b 0341 0380 0103  .........{.A....
+000863b0: bd01 03d4 0303 d804 037b 0341 0380 0103  .........{.A....
 000863c0: 4a03 ea04 03d5 0303 d504 03d8 0303 2403  J.............$.
 000863d0: ef04 03d6 0403 dc03 0085 0811 010f 0203  ................
 000863e0: 0405 0607 0809 0a0b 0c0d 0e0f 1026 0087  .............&..
 000863f0: 0800 8808 0328 036f 0355 0358 035a 0302  .....(.o.U.X.Z..
 00086400: 0341 0324 03d9 0403 dc04 03dd 0403 0103  .A.$............
 00086410: 2303 5703 5903 2903 2503 5f03 6a03 4403  #.W.Y.).%._.j.D.
 00086420: 6d03 4a03 1b03 e604 03e7 0403 d402 0380  m.J.............
@@ -113551,9 +113551,9 @@
 001bb8e0: 0806 0103 0004 0000 0117 0004 0d00 0017  ................
 001bb8f0: 0102 0100 0101 00d0 080f 0002 1700 0f01  ................
 001bb900: 0100 1701 040b 0b01 00d0 0815 1701 0001  ................
 001bb910: 0103 0000 0c3a 1700 0401 0105 0100 d008  .....:..........
 001bb920: 2014 0100 d008 2300 0400 0100 d008 270a   .....#.......'.
 001bb930: 0100 d108 0400 0117 0004 0101 0500 0100  ................
 001bb940: d208 0300 0a01 00d3 0803 0001 1401 00d3  ................
-001bb950: 080a 0004 0001 00d4 0802 8162 2d2d 5d3d  ...........b--]=
-001bb960: fb45 0a24 240a                           .E.$$.
+001bb950: 080a 0004 0001 00d4 0802 907f c3c9 9764  ...............d
+001bb960: 77e3 0a24 240a                           w..$$.
exit 1

I tried using different tools I could find in x/tools to better understand the content of the file but it seems like the content is pretty consistent between 2 files
<EMAIL_ADDRESS>go run main.go -- ~/Downloads/db.x > /tmp/1.txt
<EMAIL_ADDRESS>go run main.go -- ~/Downloads/db\ \(1\).x > /tmp/2.txt
<EMAIL_ADDRESS>diff -u /tmp/{1,2}.txt | wc -l
0

I also used a modified version of x/tools/internal/gcimporter/main.go to analyze the file
func main() {
    // Define the .x file input flag
    filePath := flag.String("file", "", "Path to the .x file containing the export data")
    flag.Parse()

    if *filePath == "" {
        log.Fatal("Please provide the path to a .x file using the -file flag")
    }

    // Create a new token.FileSet
    fset := token.NewFileSet()

    // Read the export data from the specified .x file
    export, err := os.ReadFile(*filePath)
    if err != nil {
        log.Fatalf("can't read %q export data: %v", *filePath, err)
    }

    // Use gcexportdata to read the data and create a *types.Package
    r, err := gcexportdata.NewReader(bytes.NewReader(export))
    if err != nil {
        log.Fatalf("reading export data %s: %v", *filePath, err)
    }

    tpkg1, err := gcexportdata.Read(r, fset, make(map[string]*types.Package), "")
    if err != nil {
        log.Fatalf("decoding export data: %v", err)
    }

    // Print the package information from the .x file
    fmt.Println("# Read from export data:")
    printPackage(tpkg1)

    // Now reexport as indexed (deep) export data, and reimport.
    var tpkg2 *types.Package
    {
        var out bytes.Buffer
        if err := gcimporter.IExportData(&out, fset, tpkg1); err != nil {
            log.Fatal(err)
        }
        var err error
        _, tpkg2, err = gcimporter.IImportData(fset, make(map[string]*types.Package), out.Bytes(), tpkg1.Path())
        if err != nil {
            log.Fatal(err)
        }
    }

    // Print the re-imported package information
    fmt.Println("# After round-tripping through indexed export data:")
    printPackage(tpkg2)
}

// printPackage prints out details of the package
func printPackage(pkg *types.Package) {
// ...taken from the original file
}

But still cannot see any diff between the 2 files.
Here are the 2 files attached.
db.tar.gz
What did you expect to see?
The expectation here is that if we were to compile package github.com/buildbuddy-io/buildbuddy/server/util/db/db.go, if there were no changes in exported data in db.go, then db.x will have reproducible content, and thus, downstream packages which depend on db.x will not need to recompile.

Related Issues and Documentation

x/tools/go/gcexportdata: can't import package from pkg/mod #52269
x/tools/go/gcexportdata: TestImportTestdata and TestImportStdLib failures #20121 (closed)
cmd/go: 'go list -export' doesn't report export data file path when GOROOT is wrong #32849
x/tools/go/packages: NeedExportFile causes error about invalid go.mod file to be dropped #67638
go vet --vettool panic: assertion failed in go 1.20.1 darwin/arm64 while it should say: export data is newer version - update tool #58644 (closed)
cmd/buildid: rewrite algorithm is broken #23181 (closed)
x/tools/cmd/gopls: fails on filepaths with spaces #34270 (closed)
dyld: malformed mach-o image in newest Go version #38765 (closed)
cmd/link: ld: B/BL out of range -134840372 (max +/-128MB) from 0x10809F954 #67854
cmd/go: doesn't preserve file permissions when modifying the go.mod and go.sum files #34634 (closed)

(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)

There are two changes in the diff:
// < 000863b0  bd 01 03[d4 03 03 d8 04] 03 7b 03 41 03 80 01 03  |.........{.A....|
// ---
// > 000863b0  bd 01 03[d8 04 03 d4 03]  03 7b 03 41 03 80 01 03  |.........{.A....|

// < 001bb950  08 0a 00 04 00 01 00 d4  08 02[90 7f c3 c9 97 64  |...............d|
// < 001bb960  77 e3] 0a 24 24 0a                                 |w..$$.|
// ---
// > 001bb950  08 0a 00 04 00 01 00 d4  08 02[81 62 2d 2d 5d 3d  |...........b--]=|
// > 001bb960  fb 45] 0a 24 24 0a                                 |.E.$$.|

The second change, in the last 8 bytes of the payload (before the final $$) is just the fingerprint of the whole file, so that will change if there is any other change in the data. The only significant change is the first one. If we include the previous byte (03) we notice that it is a reordering of two three-byte strings (03 d4 03) and (03 d8 04). This suggests a nondeterministic map iteration in the encoder.
So what are these bytes? Instrumenting the ureader, the pos value (base offset) is 301ec, which means the file-relative offset 863b2 of the reordered string is the section-relative offset 863b2 - 301ec = 561c6. Dumping all the elemEnds, we find that this one falls within the range of elemEnds[12207]:
i             elemEnds[i]
12205   56102
12206   5612b
12207   56170 <-- 561c6 is after this
12208   561f1
12209   5622b

However, a panic statement added to the decoding of that element is never executed, so I suspect is an element needed only by the compiler (e.g. escape, inlining, generics) and not part of the type information.
I may have made a mistake in my analysis above, but I think this is a problem for the compiler team.
@timothy-king

@sluongng Do you have a chance to check whether this problem happen with go1.22.x?

I briefly rolled back my repo to 1.22.7 to do the same test but the diff is much bigger
> diffoscope db{1,2}.x | wc -l
  223704

> diffoscope db{1,2}.x | head -20
--- db1.x
+++ db2.x
│┄ Command `'nm -s {}'` failed with exit code 1. Standard output:
│┄     /Library/Developer/CommandLineTools/usr/bin/nm: error: for the -s option: bad number of arguments (must be two arguments)
│┄     /Library/Developer/CommandLineTools/usr/bin/nm: error: a.out: No such file or directory
@@ -1,21 +1,21 @@
 00000000: 213c 6172 6368 3e0a 5f5f 2e50 4b47 4445  !<arch>.__.PKGDE
 00000010: 4620 2020 2020 2020 3020 2020 2020 2020  F       0
 00000020: 2020 2020 3020 2020 2020 3020 2020 2020      0     0
-00000030: 3634 3420 2020 2020 3138 3037 3534 3520<PHONE_NUMBER>
+00000030: 3634 3420 2020 2020 3138 3038 3531 3420<PHONE_NUMBER>
 00000040: 2020 600a 676f 206f 626a 6563 7420 6c69    `.go object li
 00000050: 6e75 7820 616d 6436 3420 676f 312e 3232  nux amd64 go1.22
 00000060: 2e37 2047 4f41 4d44 3634 3d76 3120 583a  .7 GOAMD64=v1 X:
 00000070: 7265 6761 6269 7772 6170 7065 7273 2c72  regabiwrappers,r
 00000080: 6567 6162 6961 7267 732c 616c 6c6f 6368  egabiargs,alloch
 00000090: 6561 6465 7273 2c65 7865 6374 7261 6365  eaders,exectrace
 000000a0: 7232 0a0a 0a24 2442 0a75 0100 0000 0000  r2...$$B.u......
-000000b0: 0000 232b 0000 252b 0000 fe2c 0000 cf2f  ..#+..%+...,.../
-000000c0: 0000 4036 0000 96ab 0000 07b2 0000 78b8<EMAIL_ADDRESS>
Here are the 2 .x files on 1.22.7.
db.tar.gz
Not sure what to make of it. I have yet to run any of the gcexports tools on them to see if I can make sense of the diff somehow.

If I understand correctly, you first compiled "db.go" with version ABC of a dependency gorm.io/driver/mysql to get db1.x, and then you upgraded  gorm.io/driver/mysql to version DEF and recompiled "db.go" to get db2.x. You then saw a difference in these files.
Looking through the files of the build files on https://github.com/bazelbuild/rules_go/issues/4111#issue-2538260273 . There are gorm '.x' files that are different. First I found is bazel-out/.../io_gorm_driver_mysql/mysql.x.

https://app.buildbuddy.io/invocation/0bd82434-6213-466b-aa76-e3c68250c92a?fileFilter=io_gorm_driver_mysql%2Fmysql.x#files
https://app.buildbuddy.io/invocation/a21c41a4-d2d3-49b5-9fa2-928cb7043d1e?fileFilter=io_gorm_driver_mysql%2Fmysql.x#files

I suspect this is WAI. I think the first thing to do is to examine the build cache after building gorm.io/driver/mysql at versions ABC and DEF. Unless these are identical, it is fair game for db1.x and db2.x to differ. This could be contents of inlined functions or instantiations of generic function bodies not being identical, or an internal type in export data that is transitively reachable changed a field name. If these gorm.io/driver/mysql builds produces all identical files that are the inputs to this build command for both versions, please give me detailed instructions for how to reproduce the builds. I think this will be more productive to try to catch this as it happens than parse the files.

If I understand correctly, you first compiled "db.go" with version ABC of a dependency gorm.io/driver/mysql to get db1.x, and then you upgraded gorm.io/driver/mysql to version DEF and recompiled "db.go" to get db2.x. You then saw a difference in these files.
Looking through the files of the build files on https://github.com/bazelbuild/rules_go/issues/4111#issue-2538260273 . There are gorm '.x' files that are different. First I found is bazel-out/.../io_gorm_driver_mysql/mysql.x.
https://app.buildbuddy.io/invocation/0bd82434-6213-466b-aa76-e3c68250c92a?fileFilter=io_gorm_driver_mysql%2Fmysql.x#files
https://app.buildbuddy.io/invocation/a21c41a4-d2d3-49b5-9fa2-928cb7043d1e?fileFilter=io_gorm_driver_mysql%2Fmysql.x#files

Correct.

I suspect this is WAI. I think the first thing to do is to examine the build cache after building gorm.io/driver/mysql at versions ABC and DEF. Unless these are identical, it is fair game for db1.x and db2.x to differ. This could be contents of inlined functions or instantiations of generic function bodies not being identical, or an internal type in export data that is transitively reachable changed a field name. If these gorm.io/driver/mysql builds produces all identical files that are the inputs to this build command for both versions, please give me detailed instructions for how to reproduce the builds. I think this will be more productive to try to catch this as it happens than parse the files.

That would be very disappointing to know.
We intentionally use -linkobj <pkg>.a and -o <pkg>.x when using go tool compile so that we could have a more stable .x output with only the exported information needed for downstream compilation. We were hoping that if only the transitive deps was updated, as long as the exported namespaces (const, var, func, struct etc...) did not change, then .x would remain stable and thus, more cache friendly in Bazel/Blaze setup.

If I understand correctly, you first compiled "db.go" with version ABC of a dependency gorm.io/driver/mysql to get db1.x, and then you upgraded gorm.io/driver/mysql to version DEF and recompiled "db.go" to get db2.x. You then saw a difference in these files. ... I suspect this is WAI.

In haste I completely missed the part where you (@sluongng) mentioned that you upgraded a dependency between the two builds. @timothy-king is right: there is more information in an export data file than just type information. Even the same type information can be (deterministically) written in multiple ways that will cause build cache misses.
The the Go tools (compiler, linker, etc) are intended to be fully deterministic pure functions (ignoring the special case of post-link executable stamping), and the go build system, like Bazel, is an evaluator of pure deterministic functions in which every bit of input matters. So if you present the same inputs, you should get the same outputs; anything else is a bug. It certainly should not be path dependent: in other words, the sequence of prior builds should have no effect on the output of the current one. However, in this case, you presented different inputs, presumably differing only in some minor detail, and the tool computed differing outputs, though functionally identical.
The key test would be to delete the output files (e.g. go/blaze clean), to re-execute the build (without any changes to the inputs), and to check whether it re-creates an identical output file. If so, then there is no determinism problem.

there is more information in an export data file than just type information. Even the same type information can be (deterministically) written in multiple ways that will cause build cache misses.

Hmm that's surprising and disappointing to hear. Our main use case in rules_go was to divide compile outputs into .x and .a, with hope that (a) downstream compile actions (not link actions) will download smaller inputs and (b) .x will change less often as exported data of a package does not always get updated.
It sounds like we will only benefit from (a) and not (b).
My question is: do you think our use case makes sense to warrant a feature request here? Or is it something the Go team is not willing to support?

Side note: I would love to read more about the gc export data information if there are any documents available. I tried parsing through x/tools repo code but it's rather a slow process.

Hmm that's surprising and disappointing to hear. Our main use case in rules_go was to divide compile outputs into .x and .a, with hope that (a) downstream compile actions (not link actions) will download smaller inputs and (b) .x will change less often as exported data of a package does not always get updated.
It sounds like we will only benefit from (a) and not (b).

Dividing compile outputs into export information and object code is a very worthwhile optimization; we do it in Blaze (the version of Blaze used internally at Google), and analogous things in other languages (for example, Java's JVM class files contain both type information and function bodies, and only the former is needed for downstream compilation, so it makes sense to split the files). I confess I thought Bazel's rules_go already did this.
Nonetheless, the issue @timothy-king is describing is that the exported information used by downstream compilation is more than meets the eye. It contains not just the types of public declarations; it also holds information used by escape analysis, inlining, and generics, and all of these can be very sensitive to small changes in indirect dependencies. So it will change even the types do not. But it is still a pure function of the transitive closure of dependencies---not of the previous states.

My question is: do you think our use case makes sense to warrant a feature request here? Or is it something the Go team is not willing to support?

What exactly are you asking for? I think it might be instructive if there were a way to inspect every byte of the export data file so that you could understand how it depends on the inputs. But I think the toolchain already does everything you actually need.

Side note: I would love to read more about the gc export data information if there are any documents available. I tried parsing through x/tools repo code but it's rather a slow process.

There isn't a whole lot of documentation on it besides these two items:

https://go.dev/cl/578055 - a general overview
https://cs.opensource.google/go/go/+/master:src/cmd/compile/internal/typecheck/iexport.go - documents the older 'indexed' format used by x/tools

but I did an internal presentation a while back that  doesn't contain any confidential information, and though it's quite rough, you might find it useful. I have attached a PDF below.

Separate compilation.pdf

I am going to close this as I am not sure there is anything we can do. If you have a more specific feature request, please file a new issue. You may also want to ask questions in the #compilers channel of the Gophers slack.

We were hoping that if only the transitive deps was updated, as long as the exported namespaces (const, var, func, struct etc...) did not change, then .x would remain stable and thus, more cache friendly in Bazel/Blaze setup.

The .x files for dependencies should still be fine to cache on for a Bazel/Blaze setup. These will change less often, but they may still change. These files are off by a few bytes. You may end up with .x cache hits fairly often if the information in the namespaces does not change. It may be interesting to measure this at some point.

downstream compile actions (not link actions) will download smaller inputs

You may end up downloading 2 copies of a lot of the information most compilations with such a strategy.  Both copies would need coherent descriptions of many [most] of the types. For really big go programs, what I have seen is that the types can be much larger than the other data, and these need to be in both parts, and even more importantly are often required transitively in multiple packages. (Think big generated protos that are passed around.) This is the kind of thing you have to measure to know if it helps, and it may not help uniformly.

I tried parsing through x/tools repo code but it's rather a slow process.

Yes it is. The documentation is lacking and we have recently lost some institutional knowledge. The best documentation is the source for the writer at the moment: https://github.com/golang/go/blob/master/src/cmd/compile/internal/noder/noder.go . That is even slower going, but it is the most accurate.

Thank you both for the great insights. This is valuable information for us to improve Bazel's rules_go further. 🙇
Let me do some reading and jump on Gophers Slack with more questions.


Use PSR-7 to accommodate more frameworks

Use PSR-7 message interfaces instead of Symfony HTTP foundation classes and PSR-17 HTTP message factories to create responses.
I created this branch for myself and wanted to propose this change so more implementations of HTTP messages can use this library to implement CORS.

I understand this feature, but this is targeted mainly at Laravel, hence the Symfony HttpFoundation. Changing to PSR-7 would just create overhead I'm afraid.

As you can see in the changed files, the overhead mainly increases in the tests, because of the necessity to generate a PSR-17 response factory. I think this can be refactored to be less overhead.
Otherwise, the API stays the same, as Symfony and Laravel Request/Response objects all implement PSR-7.
Merging this would be a win-win IMHO, as more people would be able to contribute to this package.

I'm not sure if that's correct, I don't think Symfony Request implements PSR-7, it needs the Symfony PSR-7 Bridge? https://symfony.com/doc/current/components/psr7.html
So a request would be
Symfony Request -> Convert to PSR-7 Request -> Check CORS -> Send PSR-7 Response -> Convert to Symfony Reponse .
I do agree that PSR-7 would be cleaner for standalone libs.

Oh, my bad! I haven't used Symfony all that much and only including the PSR-7 bridge, so I assumed it was compatible.
Well this makes it indeed a bit tedious to use CORS with this package.
I'd really like to have a CORS package that works with PSR-7, but sadly, I can't maintain a fork of this package on my own. Its not your responsibility to provide it and that's okay.
Any idea what I can do?

Did you try https://github.com/tuupola/cors-middleware ? That seems it looks similar to this?
Or https://github.com/tuupola/cors-middleware otherwise.


Added an accessbility question in accessibility-quiz

Summary of changes
Added an accessibility question after self research on the official website of W3C .
According to W3C,which is not the valid guideline for creating style sheets that promote accessibility?
Checklist

[ x] I have read and followed the contribution guidelines.
[x ] I have read through the Code of Conduct and agree to abide by the rules.
[ x] This PR is for one of the available issues and is not a PR for an issue already assigned to someone else.
[x ] My PR title has a short descriptive name so the maintainers can get an idea of what the PR is about.
[ x] I have provided a summary of my changes.

closes #XXXXX

Hi @Rishav9911 !
It looks like you have closed a few of your previous PR's.
I would suggest updating your forked copy of the repo to the latest changes and then creating a branch with your new changes.
That should help fix the issues you are having plus it is best practice to work off of branches instead of the main branch when implementing changes.
Hope that helps

freeCodeCamp has an article on branches that can help
https://www.freecodecamp.org/news/how-to-use-branches-in-git/

sorry mam, I was having little difficulty as I am new to open source but now I have got the idea and has send another updated PR and has resolved all the issues and added a new question after thorough research.


removeformatPasted “middle ground”

I love this plugin, does just what I've always wanted, almost.
removeformatPasted: true obviously removes everything  and false leaves way too much (like other editors) — often allowing clients to ruins your front-end design via the CMS. :)
What I personally have always wanted is for most of the dross to get removed but for some of the basic markup left in. I have managed to get this functionality working with some code tweaked from a couple of online code samples (referenced below).
So, what it does is on paste strip tags against a white list and then removes all attributes. So pasting results in nice clean syntax that is unlikely to interfere with your front-end design.
What about incorporating this into the source and having something like pasteMode: 'plain' | 'basic' | 'rich' and an extra option for allowedTags: '

...

'
Here's the relevant part of my code, not yet thoroughly tested.
on('tbwpaste', function() {

    var $self = this;

    // Strip tags - http://locutus.io/php/strings/strip_tags/
    var html = $self.trumbowyg('html');

    var allowed = '

';

    allowed = (((allowed || "") + "").toLowerCase().match(/<[a-z][a-z0-9]*>/g) || []).join('');

    var tags = /<\/?([a-z][a-z0-9]*)\b[^>]*>/gi;

    html = html.replace(tags, function($0, $1) {

        return allowed.indexOf('<' + $1.toLowerCase() + '>') > -1 ? $0 : '';

    });

    // Remove attributes: http://stackoverflow.com/a/12360574
    var walk_the_DOM = function walk(node, func) {

        func(node);

        node = node.firstChild;

        while (node) {

            walk(node, func);

            node = node.nextSibling;

        }
    };

    var wrapper = document.createElement('div');
    wrapper.innerHTML= html;
    walk_the_DOM(wrapper, function(el) {

        if(el.removeAttribute) {

            el.removeAttribute('id');
            el.removeAttribute('style');
            el.removeAttribute('class');

        }

    });

    html = wrapper.innerHTML;

    $self.trumbowyg('html', html);

});

I'm currently improving this code; I'll post when it's finished.
One question: is it possible to get just the pasted text on tbwpaste before it's added to the textarea. I'm just thinking it'll be more performant when working with large amounts of text.

You should use pasteHandlers, not tbwpaste event.
https://github.com/Alex-D/Trumbowyg/blob/develop/plugins/cleanpaste/trumbowyg.cleanpaste.js#L162
You get the event from here:
https://github.com/Alex-D/Trumbowyg/blob/develop/src/trumbowyg.js#L622
So I think you can get the content via this code.
Feel free to make a PR to the core if you think it can be better :)
Take a look at this plugin too: https://github.com/Alex-D/Trumbowyg/blob/develop/plugins/pasteimage/trumbowyg.pasteimage.js#L18

I'd +1 this.  I've got people populating forms with content copied from Word documents that obviously add their own markup, etc.  I don't want to paste anything that would over-ride the site's stylesheet, but I do want to let basic formatting through (paragraphs, bold, italic, etc.).
At the moment I'm running a function against the field when I save it to strip the extraneous content back out, but it's a bit of trial and error and hacky.  It also only really applies to the document that they're converting from.

I pretty much have this working but haven't had time to finish it off yet, sorry. You can use the code I posted above though in the interim.

Should be resolved with the new allowTagsFromPaste plugin.


Support generic arguments to operators

I have a code like this:
class GetterLens<A, B> {
...
    GetterLens<A, C> andThen<C>(GetterLens<B, C> that) => that.compose(this);
    GetterLens operator >> (GetterLens lens) {
      return this.andThen(lens);
    }
}

instead of
  var plusElevenLens = plusOneStrLens.andThen(plusTenLens);

I want to receive something like:
  var plusElevenLens = plusOneStrLens >> plusTenLens;

But operator overriding does not support generic method syntax so with implicit dynamic I got error from analyzer for andThen method.
How I can achieve such functionality?

The language doesn't support generic arguments to operators. There is no good place (syntactically) to put the arguments.
In some places, as in your examples, the type inference could infer the types and insert them for you, but there are cases, where that's not possible, or where the user wants to pick a different type. In that case there wouldn't be any way to fix the type.

Any workaround for this?

Any workaround for this?

not really. You either need to use a function, or make the types less strong.

with less strong types - won't be auto inference. With function it'll be as hard to read as direct function invocation.

I agree that fewer types only work in some places (and probably not yours).
A direct function call is ok, but there are advantages to member functions: they can be nicely code-completed. It's, obviously, up to you to decide if it's worth here.
To be clear: I can see how your code could benefit from generic operators, and we are going to think about this use-case in the future. In particular, we would consider it, when we discuss different syntactical accesses to operators (as mentioned in my example above). These could come up in other situation, like tear-offs. At the moment, we are not working on these, though.

Ok. I see. Thanks.

Has there been any additional thoughts or plans on this point?  I think I'd like to have type inference on the generic operator and no way to be explicit about the generic (or maybe just an awkward syntax) as opposed to no generics on operators at all?  I have some code whereby the type inference would always work and the syntax would look nice (of course, I don't know how possible this would be though)

There is no reasonable way to add generics to operators, because there is no way to provide the type argument when using the operator.

Could it work if the type was inferred?  If you can't infer the type then just set as dynamic.

@polux would it be something that could be added to the spec, maybe kept open as a possible in the future?  (I thought type inference was quite good in Dart2 anyway, I think it would already work in a lot of situations I'm thinking without any changes but I could be missing something as I've only been playing around with the language for a short while).
var fn = <T1>(T1 x) => x;
fn(5) is int; //true

@twistedinferno my point is that if you default to dynamic when the type can't be inferred, and the body of your operator inspects the type argument it is passed, then the behavior of your program depends on whether type inference succeeded or not. The alternative is to fail type checking when the inference fails, but without syntactic support for explicitly passing type arguments this is not acceptable.

ah, sorry, I think I understand what you mean.  That example isn't right, it is not generic!

  var fn = <TInt>(TInt x) => x;
  var result = fn(5);
  print(result is dynamic); // true

print(result is int); // true

What I mean is something like this:
f<A>(A x) {
   return (A == int) ? 1 : 2;
}

Let's say f was an operator: then when calling f(foo), depending on the ability of the compiler to infer the type of foo the expression would evaluate to either 1 or 2.

Many thanks for clarifying and thinking more on the topic, the type inference is clever than it seems :-)  My editor in vscode shows variables of type dynamic but when the program is compiled Dart seems to assign a type if it needs to.  I ran this example trying to trick the compiler but it shows a runtime error when I uncomment the print value line.  Very interesting and thanks for your time :-)
var list = new List();
 list.add(5);
 list.add("5");

 for (var i = 0; i < 10; i++) {
   var result = list[Random().nextInt(1)];
   // print("list:" + result == int ? "1" : "2");
 }

I think I'll add some world to this thread, just to clarify a few things that do not seem to be self-evident. ;-)
@polux wrote

.. the inference algorithm would have to be part of the spec ..

Already simple stuff like Iterable<int> xs = []; depends on type inference, both during static analysis and at run time, so it should certainly be specified. We'll get there.
@twistedinferno wrote

I think it would already work in a lot of situations [For example:]

var fn = <T1>(T1 x) => x;
fn(5) is int; //true

What happens here is that you are testing that the result returned by fn is an int at run time, and that's not relying on any inferred types (that would succeed also if you gave fn the actual type argument Object or anything else that still allows for the invocation):
var fn = <T1>(T1 x) => x;

main() {
  print(fn<Object>(5) is int); // Prints 'true'.
}

But the inferred type argument will actually have the value int when you call fn(5), and you can see that in a different way:
var fn = <T1>(T1 x) {
  print(T1);
  return x;
};

X fnNamed<X>(X x) => x;

main() {
  fn(5); // Prints 'int'.
  print(fn.runtimeType); // Prints '<T1>(T1) => T1'.
  int Function(int) fnInt = fnNamed;
  print(fnInt.runtimeType); // Prints '(int) => int'.
  print(fnInt(5)); // Prints '5'.
}

You can also inspect the type of fn via its runtimeType getter, and this shows that it is a generic function. If you use a function declaration (like fnNamed) such that you are accessing the function directly (rather than passing around a function object and obtaining it by evaluation of an expression like fn), then you can also get a generic instantiation of a generic function, when it's assigned to a target whose type is a non-generic function. That's how you get a function of type int Function(int) called fnInt. This is similar to currying, in that fnInt is derived from fnNamed by passing an inferred type argument (with the value int), and the resulting object is a non-generic function, i.e., it is still ready to accept its value parameters.
So there are a lot of situations where inference takes place, and matters. So when you test the following it doesn't mean that there were no types:
print(result is dynamic); // true

It just means that you have confirmed that result has a type which is a subtype of the type dynamic. That's not hard to achieve, because every object has that property (because every type is a subtype of dynamic).
@polux wrote

What I mean is something like this:

f<A>(A x) {
   return (A == int) ? 1 : 2;
}

Let's say f was an operator: then when calling f(foo), depending
on the ability of the compiler to infer the type of foo the expression
would evaluate to either 1 or 2.

But that's certainly also the case:
f<X>(X x) {
   return (X == int) ? 1 : 2;
}

main() {
  print(f(42) == 1); // 'true'.
  print(f("Hello!") == 2); // 'true'.
  print(f<Object>(42) == 2); // 'true'.
}

But the features that you are relying on here are (1) reification of type arguments (such that there is a representation of them at run time), and (2) type inference (implicitly providing some type arguments).
So I think you can get all the things you are referring to, with regular methods.
But this issue had a different focus from the outset, namely syntax: An operator is no different from any other instance method, except for syntax, and that's exactly the reason why it is not trivial to enable invocations of operators to take type arguments explicitly, and the only reason why we don't support generic operators is the syntax.
So when you want a generic operator you can always work around it by writing a regular generic instance method and calling that. You get everything you've requested except the operator syntax.

But the features that you are relying on here are (1) reification of type arguments (such that there is a representation of them at run time), and (2) type inference (implicitly providing some type arguments).
So I think you can get all the things you are referring to, with regular methods.

Thanks, I had not realized that was already the case with "regular" methods. I assumed that in case the type inference of the type argument of a function failed, the program would be rejected. Even if that was the case, your example indeed demonstrates that the program's behavior depends on the result of the type inference, even when it succeeds.

I assumed that in case the type inference of the type argument of a
function failed, the program would be rejected

But that is indeed the case:

void f<X extends int>(X x) {}

main() {
  f("Splut!"); // Compile-time error.
}

And the run-time behavior of a program with no errors certainly depends on the outcome of type inference, in various ways.

Ok. To me that was type checking because it doesn't require unification but I guess that's still type inference indeed.

What I was getting at is: there are cases where Java's or Haskell type inference give up not because of a type mismatch but because there aren't enough constraints to infer a type and/or the inference algorithm is not smart enough to generalize in the right place. In these cases these languages don't fall back on "dynamic".

many thanks for explaining so well @eernstg (and for reminding me that is is not the same as .runtimeType and also for ignoring that List explicitly returns List()!)  I think the following example shows that if a generic type is not specified then it defaults to dynamic.
class MyGen<T1> {}
print(MyGen().runtimeType); //MyGen<dynamic>

Going back to the operators, the problem with the syntax seems to be with where to explicitly specify the generic type if it cannot be inferred, I can understand the difficulty here.  Would it be acceptable to omit the ability of explicitly specifying the generic type?  If the type can't be inferred, could the type default to dynamic as in the example above?

@polux wrote:

.. thanks ..

Thanks!

it doesn't require unification

Dart type inference does not use unification; it's inference rather than (mere) type checking here, because the selection of an actual type argument for the invocation of f may rely on information from subterms (like actual arguments) as well as information from the context.
@twistedinferno wrote:

if a generic type is not specified then it defaults to dynamic.

Not quite, if no type arguments are provided, and there is no information which may be used for inference, the chosen type arguments are computed using an algorithm that we call instantiate-to-bound. This is a bit complex, but the basic idea is that we'll use the declared bounds as far as possible, and then approximate from above (that is, using a supertype) in the cases where the bound cannot be expressed precisely (e.g., X extends C<X>).

Would it be acceptable to omit the ability of explicitly specifying the generic type?

That's a slippery slope, because developers would then be unable to solve the problem in the "natural" manner if they write invocations where inference cannot make a decision (but a developer might be able to help). So I suspect that it would be better to ask people to use a regular instance method (as opposed to an operator) if they need a generic function. That's possible today.

@Aetet I think this could be a workaround

class Foo<A, B> {
    Foo<A, C> andThen<C>(Foo<B, C> that) => null;

    Bar<A, B, C> call<C>() => Bar<A, B, C>(this);
}

class Bar<A, B, C> {
    final Foo<A, B> source;

    const Bar(this.source);

    Foo<A, C> operator >>(Foo<B, C> lens) {
        return source.andThen<C>(lens);
    }
}

Foo<int, double> x;
Foo<int, String> y;
Foo<double, String> z;

Foo<int, String> res = x.andThen<String>(z);
Foo<int, String> res2 = x<String>() >> z;

and a simpler example for anyone looking for how to get rid of method parentheses and also use generic types
class FooGenericOperator<T> {
    final Foo f;

    const FooGenericOperator(this.f);

    T operator >(T b) => f.operate(b);
}

class Foo {
    R operate<R>(R b) => b;

    FooGenericOperator<T> call<T>() => FooGenericOperator(this);
}

class Bar {}

Bar x = Foo().operate(Bar());

Bar y = Foo()<Bar>() > Bar();

@modulovalue Thanks for your response. It's interesting solution.
Every year I make presentation about little dart puzzlers for local conference.
Old Presentation
I use your solution at new presentation I won't mind and mention you as author?

@Aetet No I don't mind, I'd be honored. Thanks!


Bluetooth: controller: Release of padding PDUs for framed production

Implemented:

Released framed padding PDUs when data received in SDUs for a given event do not fully utilise BN
Implemented tests to check release of framed padding PDUs as the end of the event

@cvinayak
The changes I made dismissed your previous approval. Would appreciate it if you could approve again.
Thanks.


Import data using CLI

I'd be cool to have a possibility to just import data from another project (refill all data).
What feature are you missing?
Importing data from existing project.
I have three projects - dev, staging and prod. I don't want to manually copy all data form 'prod' to 'staging' in the admin panel.
How could this feature look like in detail?
graphcool import 'Project to copy from name'

or
graphcool import 'path to folder with json data which was exported form another project'

closing as duplicate of #610


SSH Connection Error - unix_listener: cannot bind to path

Not working SSH connection with deployment.

For the directory /var/www/.ssh/ need to specify the owner of www-data

thanks for pointing out the problem.
The permissions on the .ssh folder have been fixed in the commit 5016e21


feat: add boosted edges for rotated pages

This PR adds boosted edges for rotated polygons: we successively draw a dilated polygon and fill the inside of the shrank polygon with 0, so that we drew the edges. We can choose the edges thickness with 2 parameters, which are for the moment set to 1.5 and 0.4 (such as in the DB paper).
Any feedback is welcome!

Closing this since we will remove the boosted edge feature in the loss and add a dice loss term


Add facets support

While doing #11 I noticed that the project was missing the support of faceted search. There is a PR ready for this issue waiting to be merged.

I see:

attributesForFaceting in the settings
facetsDistribution during search
facetFilter during search

Missing facetsDistribution in SearchResult model. Closing this issue in favor of #58 that is more accurate.


chore: Add sherif

Sherif is a tool designed to detect issues with monorepo config.
Implemented to query in https://github.com/TanStack/query/pull/6376

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (77509d0) 87.78% compared to head (6cb85c6) 87.78%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff           @@
##             main     #603   +/-   ##
=======================================
  Coverage   87.78%   87.78%
=======================================
  Files          31       31
  Lines         819      819
  Branches      184      184
=======================================
  Hits          719      719
  Misses         95       95
  Partials        5        5

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


How to round to the nearest even number?

When displaying a currency amount I want to use the bankersround: always round to the closes even value when rounding a 5. So like this:
0.255 => 0.26
0.265 => 0.26
0.275 => 0.28
0.285 => 0.28

Does anybody know if this is possible with accounting.js? All tips are welcome!

Unfortunately, at the moment you will have to round your numbers before you pass them into accounting.js.

Ah, in that case I'll round them in Big.js before I pass them on to accounting.js.
Thanks a million for the info!


fix: added missing dependencies for templates

Summary

Adds the missing changeset from #3035

Checklist

[X] All changes are covered by tests (or not applicable)
[X] All changes are documented (or not applicable)
[X] I reviewed the entire PR myself (preferably, on GH UI)
[X] I described all Breaking Changes (or there's none)

Why is this needed?

Missed the changeset from #3035 which mean't it didn't release to NPM - so the integration test is still failing.


[RFC] AA support for signed image verification

As @stevenhorsman mentioned in issue#2682 of kata-containers, the function of verifying the container image signature will be added in the PoC of ccv0. It is also mentioned in the proposal that in the long term approach, the attestation-agent (AA for short) will cooperate to complete the verification of the container image signature based on the attestation. This proposal will give specific schemes to support verifying container image signature in AA.
Summary
Based on the red hat simple signing architecture, three contents are required to successfully verify the container image signature: signature, public key and policy.json file. Therefore, how to obtain these three contents has become the key for the confidential containers to support the verification of container image signature. In addition, the usage scenario of the confidential containers requires that the container image must be encrypted (paritial layers containing sensitive info) and signed. Therefore, this scheme needs to organically combine the two mechanisms, rather than treat them as independent and irrelevant features.
We currently envision two options:
Option 1：Boot Image scheme
In this scheme, the signature (or signature store server), public key (or public key server) and policy.json required for signature verification need to be directly built into the boot image by the tenant when building it.
In fact, in this scheme, the process of verifying the signature does not require the participation of AA, ocicrypt-rs can complete all signature verification as long as it supports simple signing. We propose this option only to facilitate comparison with the option 2 scheme introduced later.
Since AA is not required to participate, this scheme is almost indistinguishable from the original red hat simple signing architecture. However, must pay attention to the following two points:

To ensure the security of signature, public key and policy.json content, the boot image must be provided by the tenant. In the process of implementing the commercialization of confidential containers, it is optional whether to require tenants to provide boot images, but the scheme of "must provide boot images by tenants" will undoubtedly increase the use and maintenance costs of tenants and reduce the flexibility of the scheme.

Although the signature store server and public key server can be configured to obtain from the remote server, this adds additional trustors other than the trusted container registry. In fact, when practicing confidential computing, it is a best practice to reduce the number of trustors as much as possible.

Option 2：Key Broker Service scheme
In order to enhance security, it is not enough to simply configure the signature store server and public key server into the trusted domain controlled by the tenant, because these two communication channels are not attested. At present, the communication between KBS and AA is attested.
In this scheme, the tenant needs to deploy the key broker service (KBS for short) in the trusted domain under his control, and save the signature, public key and policy.json on the KBS side (we still use the word KBS to facilitate description and establish association with the KBS used in the encryption container image scheme). This scheme requires some cooperation from AA, ocicrypt-rs and KBS. We describe the whole process as follows:

KBS side:

Manage signing key and assign unique key-id.
Formulate the policy.json file and save it in the specified local directory.
Configure the yaml file in the /etc/containers/registries.d local directory and specify the signature store.

Signing: sign the container image and save the signature in the specified signature store.
Startup of AA: in the startup phase of AA, connect to KBS through the key broker client (KBC for short) and download the policy.json file through the attested channel.
Image pulling: the kata-agent in pod pulls the image.
Signature verification:

skopeo/image mgmt service calls ocicrypt-rs, and then ocicrypt-rs calls AA to send the manifest digest and docker reference to AA.
AA uses the informations such as manifest digest and docker reference to obtain the signature and the public key used to verify the signature from KBS and returns them to ocicrypt-rs.

Under this scheme, a best practice on the KBS side is to uniformly manage the signature key and decryption key, and establish a mapping relationship with docker reference/manifest digest. In this way, docker reference/manifest digest becomes the actual "key ID". If this practice is not adopted, KBS needs to separately consider where to place the key ID so that it can be read by AA and sent to itself along with the manifest digest and docker reference.
Since only option 2 requires AA's participation and cooperation, the following text only gives the requirements and design for option 2.
Requirements
KBS scheme requires the cooperation of KBS, ocicrypt-rs and AA. Therefore, the following requirements are proposed for these three components:
ocicrypt-rs

[ ] Extend keyprovider protocol. When pulling the container image, call the new grpc service API provided by AA to obtain the signature and public key.

[ ] Use the public key to verify the signature.

[ ] Can read the policy.json file and decide whether to accept the container image.

FYI: @arronwy
Attestation Agent

[ ] Implement the grpc service API newly added in the keyprovider protocol to provide signature and public key for ocicrypt-rs.

[ ] Expand KBC interface trait, add a new KBC interface, and call it when initializing KBC in AA startup phase. Its function is: KBC starts attestation to KBS, negotiates a key to establish encryption channel after attestation, and then downloads policy.json from KBS.

[ ] Advance the instantiation of KBC module to AA startup phase (the function of instantiating KBC when receiving UnWrapKey request for the first time is reserved), which requires AA to be able to read the configuration files of KBC name and KBS URI from the environment during startup phase.

Key Broker Service

[ ] Provide the management function of signing public key and allocate unique key-ID.

[ ] Provide the management function and download interface of policy.json file (download is allowed only after KBC attestation and establishes an encrypted channel).

[ ] Can read and parse the yaml file in the /etc/containers/registries.d directory. The format and content of the file are defined by the red hat simple signing scheme.

[ ] Can get the signature from the specified address.

[ ] Provide a unified and standardized service interface to enable AA to connect it and obtain signature and public key.

If the option2 scheme is finally adopted, we will issue a new proposal for the detailed definitions and standards of the above functions of KBS in the near future, and provide code libs that implement these new functions of KBS in sample KBS of AA's repository in the future(or exists as a separate library project under the CC organization).
Design (Attestation Agent)
We now give the design of the attachment agent supporting container image signature option2 scheme: AA will add a new KBC interface to download the policy.json file, and a new keyprovider grpc API to provide signature and public key to ocicrypt-rs.
KBC Interface
We will add a new interface in KBC interface trait, its name is get_sign_policy_file:
pub trait KbcInterface {
    ...
    fn get_sign_policy_file(&mut self) -> Result<std::fs::File> {
        ...(default implement)
    };
}

This trait will be called immediately after KBC initialization is completed in the startup phase of AA. The functions it realized are: connect to KBS, start an attestation, negotiate the key after attestation, establish an encryption channel, then download the policy.json file from KBS, and then return the file descriptor to the caller.
In order to maintain compatibility with previous KBCs and pre-attestation KBCs, this trait will provide a default implementation (generate the simplest policy.json) to ensure that KBCs that do not implement it can still run normally.
KeyProvider API
We will extend the keyprovider protocol and add a new grpc API:
syntax = "proto3";

package keyprovider;

...
message keyProviderVerifySignatureInput {
    bytes KeyProviderVerifySignatureInput = 1;
}

message keyProviderVerifySignatureOutput {
    bytes KeyProviderVerifySignatureOutput = 1;
}

service KeyProviderService {
    ...
    rpc VerifySignature(keyProviderVerifySignatureInput) returns (keyProviderVerifySignatureOutput) {};
}

The input and output messages are JSON strings. The format definition is given below:
KeyProviderVerifySignatureInput:
{
    "image": {
        "manifest-digest": "sha256:817a12c32a39bbe394944ba49de563e085f1d3c5266eb8e9723256bc4448680e"
    },
    "identity": {
        "docker-reference": "docker.io/library/busybox:latest"
    }
}

"manifest-digest" represents the digest of the image manifest blob of the signed container image, but in the specific implementation, please pay attention to:

The image manifest of a single container image may contain the digest value of multiple image manifest using different digest algorithms.

If the manifest contained in the container image is signed manifest defined by docker(including image manifest and additional manifest signatures), then this manifest digest field represents the "JSON web signature" part of the manifest signature in the signed manifest, rather than the digest of the image manifest blob in the container image.

"docker-reference" is a way to describe container image. In the currently used red hat simple signaling scheme, only this way is supported.
KeyProviderVerifySignatureOutput:
{
    "algorithm": "",
    "key-length": "",
    "public-key": "<pub-key>",
    "signature": "<signature-content>"
}

"algorithm" and "key-length" field indicates the public key length and signature algorithm used to verify the signature. The content of the "public key" field contains the public key required to verify the signature.
"signature" is the signature of the container image, which is embodied as a character stream here. Please refer to here for its specific content format.

I will have some more detailed comments soon, but for now a general question: do we want to have a generic secret endpoint for the AA? As CC continues to grow, it's somewhat likely that we will want to use the AA to provision more and more secrets. For example, we might need to provide secrets for authenticating with a container repository. As opposed to extending the keyprovider api every time we have to get another secret, maybe we should just have another endpoint where any component can request a secret from the AA? This would widen the scope of the project somewhat and might have security implications.

I will have some more detailed comments soon, but for now a general question: do we want to have a generic secret endpoint for the AA? As CC continues to grow, it's somewhat likely that we will want to use the AA to provision more and more secrets. For example, we might need to provide secrets for authenticating with a container repository. As opposed to extending the keyprovider api every time we have to get another secret, maybe we should just have another endpoint where any component can request a secret from the AA? This would widen the scope of the project somewhat and might have security implications.

@fitzthum This is a great idea. As you said, we can add a new grpc endpoint (for example, secretprovider) in AA to provide APIs for obtaining various secrets, right?

@jialez0 is this issue still relevant or can be closed?
If it's still relevant to what release do you think we should map it to (mid-November, end-December, mid-February etc...)?


ViewFetcher, getRecentDecorView fails on Samsung S4 with MultiWindow support

From<EMAIL_ADDRESS>on May 05, 2014 00:01:47
What steps will reproduce the problem? 1. Application manifest should contains
<uses-library android:name="com.sec.android.app.multiwindow" android:required="false"/>
<meta-data android:name="com.sec.android.support.multiwindow" android:value="true" />

Devices should support Samsung MultiWindows libraries (i.e. Galaxy S3, S4, ...; Galaxy Note) What is the expected output? What do you see instead? 1. ViewFetcher.getRecentDecorView return null, instead of view.

ViewFetcher.getRecentDecorView fails on lines 126-127, due to view object has another class, i.e.
com.android.internal.policy.impl.MultiPhoneWindow$MultiPhoneDecorView instead of com.android.internal.policy.impl.PhoneWindow$DecorView What version of the product are you using? On what operating system? Samsung S4, Android API 19, KitKat; Robotium 5.1

Original issue: http://code.google.com/p/robotium/issues/detail?id=600

From<EMAIL_ADDRESS>on May 05, 2014 01:12:30
Thanks for reporting this. It will be fixed in the next release.
Status: Accepted
Owner<EMAIL_ADDRESS>

From<EMAIL_ADDRESS>on June 18, 2014 08:12:16
Thanks again for reporting this. This has been fixed in Robotium 5.2.1
Status: Verified


Create SwerveSubsystem instance in RobotContainer

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

Current dependencies on/for this PR:

main

PR #14

PR #15

PR #16

PR #17   👈

PR #18

This stack of pull requests is managed by Graphite.


Quickstart Python Webapp up ResourceGroup issue

Just running this tutorial - https://docs.microsoft.com/en-us/azure/app-service/containers/quickstart-python
Fails on the "az webapp up" steps
This is autogenerated. Please review and update as needed.
Describe the bug
Command Name
az webapp up
Errors:
'NoneType' object has no attribute 'resource_group'
Traceback (most recent call last):
python3.6/site-packages/knack/cli.py, ln 206, in invoke
    cmd_result = self.invocation.execute(args)
cli/core/commands/__init__.py, ln 603, in execute
    raise ex
cli/core/commands/__init__.py, ln 661, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
cli/core/commands/__init__.py, ln 652, in _run_job
    cmd_copy.exception_handler(ex)
cli/command_modules/appservice/commands.py, ln 55, in _polish_bad_errors
    raise ex
cli/core/commands/__init__.py, ln 631, in _run_job
    result = cmd_copy(params)
cli/core/commands/__init__.py, ln 306, in __call__
    return self.handler(*args, **kwargs)
azure/cli/core/__init__.py, ln 485, in default_command_handler
    return op(**command_args)
cli/command_modules/appservice/custom.py, ln 2793, in webapp_up
    current_rg = app_details.resource_group
AttributeError: 'NoneType' object has no attribute 'resource_group'

To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

Put any pre-requisite steps here...
az webapp up -n {} --debug

Expected Behavior
Environment Summary
Linux-4.15.0-1059-azure-x86_64-with-debian-stretch-sid
Python 3.6.5
Shell: bash

azure-cli 2.0.74

Additional Context

Close this one as it is duplicated with #10713

Please use version 2.0.77 & let us know if this still repros. Thanks!


Get the following list of a public user

General Question
Read the Notes and fill out the form.
Switch to Preview for reading.
Form
Put an [x] if you meet the condition, else leave [ ].

[ x] I've searched the Issues
[ ] I've read the basic concepts
[x ] I'm using the latest version

Question
get the a following list of a user whose id is public. Like eminem following list.

            const targetUsername = "eminem";
            // Get target user ID
            const targetUser = await retry(() => ig.user.searchExact(targetUsername), { retries: 3 });
            if (!targetUser) {
                throw new Error(`User ${targetUsername} not found`);
            }

            // Get target user followers
            const followersFeed = ig.feed.accountFollowers(targetUser.pk);
            const followers = await retry(() => followersFeed.items(), { retries: 3 });


Feature/servo angle

Adds new celery_script node

{kind: "set_servo_angle", args: {pin_number: 4 | 5, pin_value: 0..360}}
new function to Farmbot.Firmware behaviour

set_servo_angle(self, pin, value)

New CeleryScript code and test generator.

This will fix #371 and #372 once released.

let's wait for formal release. thanks,


⚠ Quirk Club Argentina (API endpoint) has degraded performance

In 77656ca, Quirk Club Argentina (API endpoint) (https://us-central1-quirkclub-dev.cloudfunctions.net/api/check/api) experienced degraded performance:

HTTP code: 200
Response time: 6931 ms

Resolved: Quirk Club Argentina (API endpoint) performance has improved in ab6b6ce.


[0.5pt] Find external API - group task

Find an API to base the development of the webapp around it. The API should allow you to

Get a list of items with a unique item id (or generate the unique id).
For a given item, get detailed information about it.
choose an APIs that doesn't need authentication. if you select an API that requires authentication, you should implement it on your own. Also, if you select an API that provides image resources your webapp will be more visual.

The Free-To-Play Games Database API seems like a good option for our webapp. It provides a list of free-to-play games along with detailed information about each game, including its unique ID. The API is easy to integrate with your webapp using simple HTTP GET requests, and there are no limitations on the number of requests you can make or the amount of data you can access. Additionally, the API is free forever, and no authentication is required.


Merge netlify-dev-plugin in CLI

- Summary
Fixes: https://github.com/netlify/cli/issues/394

Since netlify-dev-plugin can only be used with cli. We're going to merge netlify-dev-plugin int CLI code base for sake of maintainability and ease of release.
- Test plan

netlify dev in your project

- Description for the changelog

Copy the source of netlify-dev-plugin to CLI
Install dependencies and devDependencies from netlify-dev-plugin

- A picture of a cute animal (not mandatory but encouraged)

I'm guessing the issue in that repository will be moved here too, right?

@calavera Yes of course.


Saving tuples as keys using sqlitedict

This issue is more a new feature idea.
It would be nice to be able to use Python tuples as keys as you can do it in standard Python dicts. My idea is to do something like serialisation from tuple to string (for example via user_defined_character.join(tuple) or via some user defined conversion function).
For querying there would be used just the same transformation function and for keys reconstruction there might be something like, either key.split(user_defined_character) or some other user defined conversion function.

I believe this would be solved by #19, which believe it or not I plan to get to very soon.

I believe it! :)
Now that I think of it, why not just pickle.dumps the keys as well?
That is, keys are internally always strings (=pickle dumps), and encode/decode of the keys that user gives happens transparently.
That will allow arbitraty objects as keys (anything pickleable). Strings or tuples or whatever will no longer a special case, and will require no special action from users.
Or am I missing something?

Handled in separate pull requests, using pickle for encoding arbitrary key objects.


Implement generic address watcher for WebApi

Have methods to add address to watch.
The method to add address to watch accept address, target amount, timeout, callback URL and timeout status.
The method to add address will start timer to watch for timeout. When timeout it will invoke callback with data then remove that watch.
When total received amount reached target it will invoke callback with data then remove that watch.
Callback data will contains address and total amount received.
Must implement IHostedService.

Blocked by #46.

We don't need this anymore. What we need is #80.