Parameterise aggregator function for DateValgator and DateValNNer

currently, implementation uses median. add maximum, minimum, and mean for possible usage in the future.

PR #50


Edited some channels

Added new channels from mncvision.id and removed deleted ones.

This branch has conflicts that must be resolved


Run Docker image using rkt:
# dpkg -i rkt_1.19.0-1_amd64.deb
# rkt --insecure-options=image fetch docker://nginx
# rkt --insecure-options=image --port=80-tcp:9080 run docker://nginx &
# PID=$!
# curl -I http://localhost:9080/
HTTP/1.1 200 OK
...
# kill $PID

Run Docker image using rkt from systemd-run:
# systemd-run --slice=machine rkt --insecure-options=image --port=80-tcp:9080 run docker://nginx
# curl -I http://localhost:9080/
curl: (7) Failed to connect to localhost port 9080: Connection refused

What did you expect to see?
I expected the Docker image to be active and running using the systemd-run and rkt combination.
What did you see instead?
# systemctl status run-rd6edbc1901c64bcdb43ee56b0c8d3dab.service
● run-rd6edbc1901c64bcdb43ee56b0c8d3dab.service - /usr/bin/rkt --insecure-options=image --port=80-tcp:9080 run docker://nginx
   Loaded: loaded
Transient: yes
  Drop-In: /run/systemd/system/run-rd6edbc1901c64bcdb43ee56b0c8d3dab.service.d
           └─50-Description.conf, 50-ExecStart.conf, 50-Slice.conf
   Active: failed (Result: exit-code) since Sat 2016-11-19 20:43:43 CET; 7min ago
  Process: 28170 ExecStart=/usr/bin/rkt --insecure-options=image --port=80-tcp:9080 run docker://nginx (code=exited, status=254)
 Main PID: 28170 (code=exited, status=254)

Nov 19 20:43:33 moon systemd[1]: Started /usr/bin/rkt --insecure-options=image --port=80-tcp:9080 run docker://nginx.
Nov 19 20:43:43 moon rkt[28170]: stage1: error getting container subcgroup: error calling sd_pid_get_unit: no such file or directory
Nov 19 20:43:43 moon systemd[1]: run-rd6edbc1901c64bcdb43ee56b0c8d3dab.service: Main process exited, code=exited, status=254/n/a
Nov 19 20:43:43 moon systemd[1]: run-rd6edbc1901c64bcdb43ee56b0c8d3dab.service: Unit entered failed state.
Nov 19 20:43:43 moon systemd[1]: run-rd6edbc1901c64bcdb43ee56b0c8d3dab.service: Failed with result 'exit-code'.

I think @s-urbaniak was already tracking down a similar regression, which should be related to cgroups. If I remember correctly, this appeared around 1.16. I'm however surprised that this broke under systemd-run; are systemd units affected as well?

@namsral can you check if systemd-machined is running, i.e. by issuing systemctl status systemd-machined?

I can confirm the container service is running.
Output of systemctl status systemd-machined:
# systemctl status systemd-machined                                                                                │··································································
● systemd-machined.service - Virtual Machine and Container Registration Service                                                                             │··································································
   Loaded: loaded (/lib/systemd/system/systemd-machined.service; static; vendor preset: enabled)                                                            │··································································
   Active: active (running) since Sat 2016-11-19 00:58:54 CET; 2 days ago                                                                                │··································································
     Docs: man:systemd-machined.service(8)                                                                                │··································································
           http://www.freedesktop.org/wiki/Software/systemd/machined                                                                                │··································································
 Main PID: 5572 (systemd-machine)                                                                                │··································································
   Status: "Processing requests..."                                                                                │··································································
   CGroup: /system.slice/systemd-machined.service                                                                                │··································································
           └─5572 /lib/systemd/systemd-machined                                                                                │··································································
                                                                                │··································································

@lucab systemd unit are also affected.

Ahh, I rechecked if this really was the same error (I have to correct the commit message in the r/w PR), and it is not! I got error calling sd_pid_get_owner_uid which is different from what we see here.
The util.IsRunningSystemd already passed successfully, and the reported error in this issue originates from util.CurrentUnitName() inside stage1/init: https://github.com/coreos/rkt/blob/master/stage1/init/init.go#L783

@namsral I just tried on a debian testing and on a coreos alpha, and it looks like I'm not able to reproduce the issue locally, so we may need a couple more information from your side:

the output of systemd-cgls
the content of /run/systemd/system/<unit>.service.d/50-Slice.conf
does it work if you remove --slice=machine?
can you check if 1.15 works?

Q: the output of systemd-cgls
A: See https://gist.github.com/namsral/c535fd258ddf94c272c4824244065986
Q: the content of /run/systemd/system/.service.d/50-Slice.conf
A: From the container mentioned the OP:
# cat /run/systemd/system/run-rd6edbc1901c64bcdb43ee56b0c8d3dab.service.d/50-Slice.conf
[Service]
Slice=machine.slice

Q: does it work if you remove --slice=machine?
A: No, same error
Q: can you check if 1.15 works?
A: The container runs with rkt v1.15 despite the same error:
# systemd-run --slice=machine rkt --insecure-options=image --port=80-tcp:9080 run docker://nginx
Running as unit run-r9b619bfc9eec497487234909657b3a96.service.
● run-r9b619bfc9eec497487234909657b3a96.service - /usr/bin/rkt --insecure-options=image --port=80-tcp:9080 run docker://nginx
   Loaded: loaded
Transient: yes
  Drop-In: /run/systemd/system/run-r9b619bfc9eec497487234909657b3a96.service.d
           └─50-Description.conf, 50-ExecStart.conf, 50-Slice.conf
   Active: active (running) since Mon 2016-11-21 12:10:27 CET; 19s ago
 Main PID: 8376 (ld-linux-x86-64)
    Tasks: 5
   Memory: 8.2M
      CPU: 4.241s
   CGroup: /machine.slice/run-r9b619bfc9eec497487234909657b3a96.service
           ├─8376 stage1/rootfs/usr/lib/ld-linux-x86-64.so.2 stage1/rootfs/usr/bin/systemd-nspawn --boot --notify-ready=yes --register=true --link-journal=try-guest --keep-unit --quiet --uuid=f401068e-6b0d-4ecd-88ca-4fe90a8
           ├─init.scope
           │ └─8456 /usr/lib/systemd/systemd --default-standard-output=tty --log-target=null --show-status=0
           └─system.slice
             ├─nginx.service
             │ ├─8464 nginx: master process /usr/sbin/nginx -g daemon off
             │ └─8465 nginx: worker process
             └─systemd-journald.service
               └─8459 /usr/lib/systemd/systemd-journald

Nov 21 12:10:27 moon systemd[1]: Started /usr/bin/rkt --insecure-options=image --port=80-tcp:9080 run docker://nginx.
Nov 21 12:10:27 moon rkt[8376]: image: using image from local store for image name coreos.com/rkt/stage1-coreos:1.15.0
Nov 21 12:10:27 moon rkt[8376]: image: using image from local store for url docker://nginx
Nov 21 12:10:30 moon rkt[8376]: networking: loading networks from /etc/rkt/net.d
Nov 21 12:10:30 moon rkt[8376]: networking: loading network default with type ptp
Nov 21 12:10:31 moon rkt[8376]: stage1: continuing with per-app isolators disabled: error calling sd_pid_get_unit: no such file or directory

# curl -I http://localhost:9080/
HTTP/1.1 200 OK

Confirming that this regression was introduced in 1.16.0. PR 3032 dropped the fallback case for the error that @namsral is hitting. https://github.com/coreos/rkt/pull/3389 is doing some cleanup around that code, we'll likely re-introduce the fallback once that is merged.
However, I'm failing to reproduce the issue even on a plain Ubuntu 16.04.1 VM. I noticed that you are running a 4.8.0 kernel, which doesn't seem to be available on xenial. Where is that coming from?

The 4.8.0 kernel is available at http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.8/.

Same issue with kernel 4.6.7, my server must be jinxed!
http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.6.7/

I'm postponing this investigation a bit as it seems pretty much specific to the mixed environment described above. @namsral can you see if it is reproducible with the stock kernel?
Also paging @martinpitt: is there something special when combining Xenial kernel and mainline kernel from PPA which can make sd_pid_get_unit() cgroup detection failing under systemd-run?

Also, same issue with older stock kernel 3.13.0-35-generic.

This is also happening when running functional tests make functional-check on Ubuntu 16.04, with the 4.8.0-46-generic #49-Ubuntu kernel.
I used ./configure --enable-functional-tests --enable-incremental-build --with-stage1-flavors=coreos.

I was running into the same problem on Ubuntu 16.04. Was not kernel related (updated from 4.4.0-98-generic to 4.10.0-38-generic). The following commands solved it for me (can't tell you which of those were relevant):
apt purge systemd-shim libsystemd-daemon0 libsystemd-login0
apt install ubuntu-server
systemctl daemon-reload
systemctl start rkt-jenkins.service

Here's my unit file:
[Unit]
Description="jenkins rkt instance"
Requires=network-online.target
After=network-online.target

[Service]
Slice=machine.slice
# Resource limits
Delegate=true
# CPUShares is a weight, between 2 and 262144. Available resources are shared within the same slice according to weight
CPUShares=512
MemoryLimit=4G
# Intentionally left out --insecure-options=image so it won't fetch the image automatically. ansible takes care of that
ExecStart=/usr/bin/rkt --debug=true run --volume volume-var-jenkins-home,kind=host,source=/virtual_machines/acoveo-jenkins/data/jenkins docker://jenkins/jenkins:lts
ExecStopPost=/usr/bin/rkt gc
KillMode=mixed
Restart=always

[Install]
WantedBy=multi-user.target

and my environment:
# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.3 LTS
Release:        16.04
Codename:       xenial
# uname -r
4.10.0-38-generic


Fix object Without() and a rule skipping issue.

It looks like Without() was incorrectly checking the length of contains.
This PR also has a patch which closes #5.

@faceair ping?


[5.8] No CSRF token mismatch when trying to login again

When you've two tabs open and login in tab 1, next login in tab 2 you'll get a CSRF token mismatch which is logical but should not happen.

Thanks for the PR, but I don't think this is a good solution, mainly because of the hard coded assumption that there is exactly one login entrypoint, and it's uri is login.

Re-reading the description, I think the correct behaviour is actually to get an exception, if you have two tabs open. I don't think this is a bug?

Agree about the url but I don't think there is a way to know the login url(s) but this covers a 99% use case. About the exception; yes it's logical but unwanted and totally illogical for the end user.

If you decide not to show the user an error page, that decision should be taken in the exception handler in your app, not in the framework's middleware IMO.

Will a PR to laravel/laravel accepted to have that by default? Getting a 419 error in this scenario is unwanted.

Thanks for your pull request to Laravel!
Unfortunately, I'm going to delay merging this code for now. To preserve our ability to adequately maintain the framework, we need to be very careful regarding the amount of code we include.
If possible, please consider releasing your code as a package so that the community can still take advantage of your contributions!
If you feel absolutely certain that this code corrects a bug in the framework, please "@" mention me in a follow-up comment with further explanation so that GitHub will send me a notification of your response.


Replace "AccessControl" with "Ownable" in ModuleFactoryBase

Is there an explicit reason why OZ's AccessControl is used in here, instead of the simpler Ownable? I ask because I only see one role that's being used. So why not just make it Ownable (which is also, effectively, "one role").

Yes, I believe that is the only functional difference between the two.
This brings up the question, do we actually want more than one address to be able to call the addVersion function.
While it may not be necessary, we (and module developers) may find this extra flexibility to be useful in the future.

Yeah, we can leave it as is 👍


API change: GetT -> GetT, GetTDefault

Previously the generated functions of the form GetT took a second argument with a default value, which proved to be noise in most cases, as the most widely useful behavior here is to just return a zero value of T. This breaking change modifies GetT to only take a single argument and return the zero value of T if not found or wrong type, and adds a new class of GetTDefault functions with the old behavior.
Addresses #12.

Ah very clever using the named return value to act as the zero value in the way that you did...
https://play.golang.org/p/GlbZnnCXTpE
I totally did not realize that would be possible. TIL. Nice


Show beginning and end dates on event page

It already does this on the event card, but not on the individual event page.

@keelyjay, this is what you were asking about. It shouldn't be too hard to do. I guess I just overlooked this because we haven't had a lot of events with start and end dates.

This is obsolete because the site has been over to SquareSpace.


Vite build options not available

Describe the bug
Build options passed to the cli is not passed on to the underlying vite build.
Options are found here
Ideally we would like to all options to be available, but the most important ones is :
 --config,  alternatively --sourcemap
Environment

OS: All
Node Version: 18

To Reproduce
  "scripts": {     "build": "fusion-framework-cli app build",   }, &&  npm run build -- --config vite.config.prod.js
Expected behavior
Ability to choose which config file to use as per vite docs
Stacktrace
error: unknown option '--config'

seems like a fair request, will add to backlog


[Desktop] It takes too long to generate a thumbnail, causing UI blocking.

Please refer to https://github.com/flutter-webrtc/flutter-webrtc/issues/1042

fixed https://github.com/flutter-webrtc/flutter-webrtc/pull/1049


[Binding 6.1.0 release] Fix timeline isn't accurate

The timeline should follow backend rule in the dashboard. Don't calculate them by frontend.

fixed 762ac380939225f195be3a657dde4f70bceee157


Added balancers.PreferNearestDC[WithFallback] balancers + Marked as deprecated balancers.PreferLocalDC[WithFallback] balancers

Pull request type

Please check the type of change your PR introduces:

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, renaming)
[x] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] Documentation content changes
[ ] Other (please describe):

What is the current behavior?

Issue Number: N/A
What is the new behavior?

Other information

Codecov Report
Attention: Patch coverage is 17.64706% with 14 lines in your changes missing coverage. Please review.

Project coverage is 53.12%. Comparing base (ab4aa77) to head (3d43e1e).

Files with missing lines
Patch %
Lines

balancers/balancers.go
0.00%
10 Missing :warning:

balancers/config.go
0.00%
3 Missing :warning:

internal/balancer/balancer.go
50.00%
0 Missing and 1 partial :warning:

:exclamation:  There is a different number of reports uploaded between BASE (ab4aa77) and HEAD (3d43e1e). Click for more details.
HEAD has 501 uploads less than BASE

Flag
BASE (ab4aa77)
HEAD (3d43e1e)

unit
87
0

go-1.21.x
47
2

ubuntu
93
3

macOS
29
0

go-1.22.x
48
1

go-1.23.x
55
0

integration
54
3

ydb-23.3
27
2

ydb-24.1
27
1

windows
28
0

ydb-nightly
9
0

experiment
9
0

Additional details and impacted files
@@             Coverage Diff             @@
##           master    #1450       +/-   ##
===========================================
- Coverage   67.75%   53.12%   -14.63%
===========================================
  Files         349      342        -7
  Lines       36234    29327     -6907
===========================================
- Hits        24551    15581     -8970
- Misses      10600    12530     +1930
- Partials     1083     1216      +133

Flag
Coverage Δ

experiment
?

go-1.21.x
53.07% <17.64%> (-15.06%)
:arrow_down:

go-1.22.x
47.46% <17.64%> (-18.25%)
:arrow_down:

go-1.23.x
?

integration
53.12% <17.64%> (-0.40%)
:arrow_down:

macOS
?

ubuntu
53.12% <17.64%> (-19.05%)
:arrow_down:

unit
?

windows
?

ydb-23.3
47.60% <17.64%> (-0.34%)
:arrow_down:

ydb-24.1
52.97% <17.64%> (-0.48%)
:arrow_down:

ydb-nightly
?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


gnu(17640)=4066

gnu(17640)=4066
GAP      4.8.3
GrpConst 2.5
Runtime: 6867018 ms (1.91 hrs)
Isomorphic groups eliminated
In case this value is new, add the next line to data/gnudata.g
GNU_SAVE( 17640, 4066, WITH_GC );

gap>
System : Windows Vista

Thank you for submitting new gnu(n). We will recalculate it and add to the database if we get the same result.
Checklist for the reviewer:

[ ] checked and assigned labels "new (n<50000)" or "recomputation", if needed
[ ] set "replication needed" label (unless it starts immediately)
[ ] changed it to "replication in progress" label when the replication has been started
[ ] changed it to "replicated" after successful replication review
[ ] reproducible value of gnu(n)
[ ] no essential runtime differences
[ ] sufficient level of technical details

Checklist for the maintainer:

[ ] correct arguments of command to store data
[ ] correct location of new line in the data file
[ ] mention submitter and reviewer in the commit message
[ ] Close corresponding issue from the commit message

gnu(17640)=4066
GAP      4.8.4
GrpConst 2.5
Runtime: 4276900 ms (1.19 hrs)
Isomorphic groups eliminated
Replicated by @alex-konovalov on Ubuntu 16.04 on Azure cloud standard DS13 v2 instance (8 cores, 56 GB memory)
To maintainers: for new results, add to data/gnudata.g this line:
GNU_SAVE( 17640, 4066, WITH_GC );


fix, ci: explicitely set up all jobs to run on tags

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 85.52%. Comparing base (9d199c0) to head (2ffeb12).
Report is 1 commits behind head on main.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2952   +/-   ##
=======================================
  Coverage   85.52%   85.52%
=======================================
  Files          58       58
  Lines        3413     3413
=======================================
  Hits         2919     2919
  Misses        494      494

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Display a loading gif when image and sound is bufferring

I just tried this, loved the desktop app..
I would like to work on this.

Need to clarify few things before starting:

Look of loading: What kind of loading animation are you expecting, do you have something in mind ? or I can try to build some CSS based loading to match your current UI & we can see from there
Behaviour of loading:
Show the loading when image/sound is loading
The loader should replace play/pause button (this is how youtube does it)

Hey, @PezCoder! welcome onboard 😄
I'm glad you liked the app.
As for the animation, I have no such preferences. You can approach it the way it looks good to you.
Also, Behavior of loading is exactly how you stated it.

@anshumanv I've done the loader based on just the music, There could be times when music is loaded & since the image is heavy it might not load.
Show both loader & play buttons might confuse the user.
Also I think the major functionality is the audio in the website & that's what the loader should be for hence I kept it just for the audio.


Training datasets with 2 classes (with style of pascal voc)

Thanx for ur excellent work!
I need a little help please. @ruinmessi  @yuyijie1995
I am trying to train RFBNet on a dataset with only 2 classes (with style of pascal voc)
I changed ~/data/voc0712.py like this:
26 VOC_CLASSES = ( 'background', # always index 0
27     'person')
28 ###    'eroplane', 'bicycle', 'bird', 'boat',
29 ###    'bottle', 'bus', 'car', 'cat', 'chair',
30 ###    'cow', 'diningtable', 'dog', 'horse',
31 ###    'motorbike', 'person', 'pottedplant',
32 ###    'sheep', 'sofa', 'train', 'tvmonitor')
and I changed train_RFB.py like this:
82 num_classes = (2, 2)[args.dataset == 'COCO']
83 ###num_classes = (21, 81)[args.dataset == 'COCO']
But I got an issue like this:
Loading base network...
Initializing weights...
Loading Dataset...
Training RFB_vgg on VOC0712
Traceback (most recent call last):
File "train_RFB.py", line 258, in
train()
File "train_RFB.py", line 209, in train
images, targets = next(batch_iterator)
File "/home/ydm/anaconda2/envs/pytorchyp3/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 336, in next
return self._process_next_batch(batch)
File "/home/ydm/anaconda2/envs/pytorchyp3/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 357, in _process_next_batch
raise batch.exc_type(batch.exc_msg)
KeyError: 'Traceback (most recent call last):\n  File "/home/ydm/anaconda2/envs/pytorchyp3/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 106, in _worker_loop\n    samples = collate_fn([dataset[i] for i in batch_indices])\n  File "/home/ydm/anaconda2/envs/pytorchyp3/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 106, in \n    samples = collate_fn([dataset[i] for i in batch_indices])\n  File "/home/ydm/project-multiscale/RFBNet-master/data/voc0712.py", line 185, in getitem\n    target = self.target_transform(target)\n  File "/home/ydm/project-multiscale/RFBNet-master/data/voc0712.py", line 136, in call\n    label_idx = self.class_to_ind[name]\nKeyError: 'car'\n'
Could u please tell how can I fix this issue?
Thanx again!

It‘’s Octo. 22 in 2018. Github crashed today, which caused that  i opened two same issues here.
So i am going to close this one.
See another one here.  #53


ATTEND-1011: Deny PUT of SYSEMP Attendee Type

ATTEND-1011
Description

Deny PUT SYSEMP Attendee Type

@chandra-nadukula The email you used to create the commit  is not associated with your GitHub account. Make sure to add (and verify) your email to your GitHub account.
Afterwards you can trigger a recheck via https://cla-assistant.io/check/SAP-docs/preview.developer.concur.com?pullRequest=794

Thanks @RoAnnC1 , done.


k8s-client might read an incorrect secret from a different namespace

This is more of a question, then a bug; but it's a bit of both...
I am looking at this line. Notice the logic here:

if there is a namespace defined, use it via listNamespacedSecret.

if there is not, read all namespaces via listSecretForAllNamespaces.

This is an interesting choice. We can obviously read the wrong thing easily and expose data from a secret that users never intended for (this is also easily reproducible).
On the other hand, if we take a step back, we only call listSecretForAllNamespaces because we think the namespace might be null. The future of this bug and discussion should really happen after this one.
For the time being, I am leaving it here as a place-holder and will get back to it.

I think the reason why I wrote it that way was because if we didn't know the namespace we were out of luck.  I suppose we could throw and error at that point.  You are right it could mean we expose something we shouldn't be.

You're right and I totally understand that, but it's tricky. Let's take each at a time.

For the fabric8 client, we think we have a namespace, always, because we default as a last resort to KubernetesClient::getNamespace and hey, the client is really suppose to know where it runs, right? Well, not really.

Basically fabric8 says : I am going to try a read a few properties and env variables, if not set, I will try to read /var/run/secrets/kubernetes.io/serviceaccount/namespace. Not present? Oh well...

k8s-client does not have such a method to find out the namespace from the current client. I have asked about this, no response yet. Judging by the python client issue, I am 99% of the answer I am going to get.

So neither of the clients is really safe, it's just that for the fabric8 because it has dedicated method, we trusted it. And it seems over the years no one complained. As said in that issue/discussion I have no idea how it would be possible to have a pod without that path. And if we ever get a bug report around that - sure, let's analyze how that is possible, first.
As such, if we agree on this, neither the k8s-client is going to have that chance to get a null namespace; because we query /var/run/secrets/kubernetes.io/serviceaccount/namespace via KubernetesNamespaceProvider anyway.
I am sorry for bringing so many details in and being so pedantic about this, but its just the way I function.

Yeah that is part of the reason why I created KubernetesNamespaceProvider.
I agree in a Pod we should never run into the case where we can't get the namespace, but we should probably account for that condition and handle it appropriately here.

I see, most probably throwing an Exception would be the most appropriate thing to do. I could easily fix this, as I am already working on some re-factoring simplifications, but before that I need also your input on the other issue.

I agree in a Pod we should never run into the case where we can't get the namespace, but we should probably account for that condition and handle it appropriately here.

just for clarification, mounting service-account is almost mandatory in a kubernetes cluster for a pod, while it can still opt-out from mounting service-account in a few cases:

static pod wont mount service-account
kube-apiserver can explicitly disable the ServiceAccount admission controller then mounting service-account will be optional for a pod
the service is deployed out of a kubernetes cluster.


Module system: detect and report erroneous diamond dependency

Describe the bug
A correct diamond dependency looks like this
    A
  /   \         |  depends on; module A depends on B, etc
 B     C        v
  \   /
    D

The module system handles that just fine, deduplicating D, based on key or imported file name.
However, in a more distributed setting, it may happen that the intermediate dependencies don't reference the same D.
    A
  /   \
 B     C
 |     |
 D1    D2

Depending on the contents of D1 and the very similar D2, this may or may not result in an error.
Specifically, this is a bug when D1 and D2 happen to be mergeable at the module system level, but not in practice; for example if both add package d to an list of packages to be added to a single environment.
In other cases, where the definitions can not be merged, the problem is merely a hard to understand error message. The error will be about some specific option, whereas the root causes is D1 and D2 were expected to be the same, but were not.
Steps To Reproduce
Steps to reproduce the behavior:

Use the module system in a distributed context, such as flakes
Have a common flake that multiple other flakes will want to use in their modules
Combine those other flakes into a single configuration; forget to use follows, or run into the flakes bug where updates don't propagate the way they should.

Expected behavior
A clear error message describing the root cause: D1 and D2 were expected to be the same module but were not. Also suggest a solution or link to a document that explains it.
Towards a solution
It seems that we don't have the right data to act on this today. A module can declare its "role" (D in the example) with key, but the deduplication logic can't detect the conflict, because file could refer to B or C when D happens to be an "anonymous module with key". Maybe this is a module system implementation artifact and not a consequence of the established syntax for modules and therefore solvable entirely within the context of the current module system <-> module interface, but I'm not sure about it.
Screenshots
Additional context

https://github.com/hercules-ci/flake-parts/issues/104
https://github.com/NixOS/nix/issues/5570

Notify maintainers

Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
output here

A workaround is to let D to be a path instead of a function or a set. It seems that the module system can deduplicate a module that is a path but cannot deduplicate it if it is a function or a set.

Correct.

The module system handles that just fine, deduplicating D, based on key or imported file name.

However, there's a good chance two versions of D will be referenced, in which case the file locations are different and the deduplication will not occur, which leads to errors that do not describe the root cause.

I think it's the final application flake's responsibility to ensure diamond dependencies reference the same copy of the module by specifying follows. However, when the module is a function, it cannot deduplicate even if the application properly uses follows.

I think the hashed file name approach works as expected and is consistent with other Nix practices, having different paths of the same version of glibc with different settings.

If the user "properly" uses .follows, what is there to deduplicate?

@mightyiam and I just ran into a flavor of this diamond dependency issue: our flake has 3 NixOS modules structured like this:

Module Common.
Module A, which imports common
Module B, which imports common

The way we have our flake structured results in Common being an "anonymous module with no key". We're getting errors from the nixpkgs module system complaining that Common is declaring options that already exist. The problem goes away if we declare a key in our shared module.  I put together a SSCCE here: https://github.com/jfly/nixos-modules-diamond-dependency.
I am left with some questions:

Should all "anonymous modules" explicitly declare a key?

If yes:

Is this documented anywhere?
Could we make this more discoverable? Perhaps by changing the nixpkgs module system to error out if a key is not specified?

If no: what is the right way to solve the issue described in this sample repo?
Or perhaps the answer is "ideally no, but it's an unsolved problem"? I think that might be what was discussed over here: https://github.com/NixOS/nixpkgs/pull/230588#discussion_r1334294197.

The downside of adding a key is that it could erroneously deduplicate modules that are actually distinct, leading to only one of them taking effect, silently ignore the other(s). (ie the problem at the start of the issue description)
So it's a tradeoff where we have to try and pick the lesser evil on a case by case basis until we have a mechanism by which this conflict can be detected.

Should all "anonymous modules" explicitly declare a key?

No, anonymous modules that are only part of the imports of a named or keyed module don't need a key if they are "inline" and not in a let binding somewhere. For example:
# file.nix
{ someFlake }: {
  imports = [
    # These don't need keys
    (mkRemovedModuleOption <...>)
    ({ ... }: { foo = true; })

    # These do
    someFlake.modules.foo.bar
  ];
}

this sample repo?

import anonymizes; it consumes the path and doesn't give it back. Never use it on a module, unless you want to apply a module's effect twice, which would be strange. Just never use it. It's bad for deduplication and bad for error message quality.
importApply does not degrade error message quality, but still does not deduplicate:
# flake.nix outputs, bad
nixosModules.foo = {
  imports = [
    # This might actually work if foo.nix only has mergeable things in it
    # It'd be best to turn foo.nix into a function that's in an attribute, and name it like mkRemovedOptionModule etc.
    (importApply ./foo.nix { inherit self; })
    (importApply ./foo.nix { self = inputs.nixpkgs; })
  ];
}

For flakes in particular, I would recommend to always set the file and key generically.
If you use flake-parts, this module sets a file and key for you.

Is this documented anywhere?

TBD. The reference documentation is already incomplete, and this is an emergent behavior, so yeah. But this needs to be documented :+1:
I think what needs to be done is that the module system gets documented with the same tooling as the rest of lib. That way at least the importApply docs are more visible.
$ nix run nix/2.24.8 repl nixpkgs
nix-repl> :doc lib.modules.importApply
Function importApply
[...]

This would at least give a hint, but a section about deduplication or perhaps even best practices would be in order.

Thank you for the quick, and wonderfully complete answer.

TBD. The reference documentation is already incomplete, and this is an emergent behavior, so yeah. But this needs to be documented 👍

Can I help with this? Would https://nix.dev/tutorials/module-system/ be the right place to document it?

As far as I know, we don't have any resources yet for designing and implementing a new module system application, let alone an ecosystem, which is where this comes in.
I guess we could do it there, and perhaps some of it could be in tutorial form, but it's mostly not in actionable form, but rather just informative. This suggests that perhaps the Nixpkgs manual would be a better place for it.
Currently the module system's reference documentation is split between Nixpkgs and NixOS manuals, with little more than evalModules being documented in the Nixpkgs manual. In all fairness an effort to move the docs to Nixpkgs didn't get off the ground, and I'm probably to blame for it, so if you would prefer to position it along the NixOS-based reference content for now, then that's also ok.


Special navigator for numbers to work on maps, lists, or vectors

For the implicit nav for numbers, should consider making a non-public navigator that will work on maps, lists, or vectors (currently uses keypath which works on just maps and vectors). This navigator would have to do an instance check on the collection type to determine how to access it.

Decided against this as it's better to keep the implicit behavior as simple as possible.


Unable to find the requested Boost libraries.

cmake ..
-- The C compiler identification is GNU 9.2.1
-- The CXX compiler identification is GNU 9.2.1
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at /usr/share/cmake-3.13/Modules/FindBoost.cmake:2100 (message):
Unable to find the requested Boost libraries.
Boost version: 1.67.0
Boost include path: /usr/include
Could not find the following Boost libraries:
      boost_program_options
      boost_system
      boost_regex

No Boost libraries were found.  You may need to set BOOST_LIBRARYDIR to the
directory containing Boost libraries or BOOST_ROOT to the location of
Boost.
Call Stack (most recent call first):
CMakeLists.txt:8 (find_package)
-- Configuring incomplete, errors occurred!
See also "/home/evantoday/routeros/poc/cve_2019_3924/build/CMakeFiles/CMakeOutput.log".

Do you have those boost libraries installed?


HTMX extension installation

Required check list:

[x] I didn't find in the repository's issues section a similar bug.
[x] I understand, this is an Open Source and not-for-profit product.
[x] This is not about a third-party project, framework, or technology.

My environment:
│ Gowebly CLI: v2.3.2 (build with go1.22.0 on darwin/arm64)
│
│ Go: 1.22.0 darwin/arm64
│ Node.js version: 20.11.1 (npm 8.1.4)
│ Bun version: 1.0.30
│
│ Is Air tool installed? true
│ Is Templ package installed? true
Describe the bug:
HTMX extensions don't work with bundlers :
https://github.com/bigskysoftware/htmx/issues/1690
So I tried to install HTMX and the extension from script tags in head but that doesn't work either:
<script<EMAIL_ADDRESS>integrity="sha384-0gxUXCCR8yv9FM2b+U3FDbsKthCI66oH5IA9fHppQq9DDMHuMauqq1ZHBpJxQ0J0" crossorigin="anonymous" defer></script>
<script<EMAIL_ADDRESS>defer></script>

How to install an HTMX extension?

My bad, installing from script tags work


Twidere support

Twidere (GitHub, F-Droid) was a Twitter client I used back in high school, so it holds a special place in my heart! I continue using it for Mastodon now and would love to use it for Bluesky, too.
Currently, only user profiles seem to load (metadata and sometimes users posts), feeds are empty, and posts are failing to send.

With 1b3489d Twidere for iOS loads the timeline properly, I can't seem to replicate posts failing to send however. Let me know if it's all working for you!

Hi, I'm running Twidere for Android and the behaviour I see is identical to that described by @notpushkin above; is there any chance of getting this fixed? Thanks!

@videah Thanks for looking into it! Twidere for iOS is quite different from the Android version, so the way it calls the API is probably different, too. I've given up on it though (no social media on phone = marginally better mental health, haha).
@andrewjohnlowe maybe you could run Twidere through mitmproxy and collect the request logs, so that @videah can investigate it some more without buying an Android phone?

I'll try mitmproxy later, but in the meantime it may be helpful to drop a link to Bluestacks android emulator; I've been using it to run Android apps on my Windows PC with great success — it runs out of the box with no problems. I have installed Twidere on it and replicated the same behaviour that I see on my physical Android phone. An earlier version (v4) exists for Mac (download page).

Update: Was SkyBridge patched recently? I noticed that SkyBridge now works fine on Twidere running on Bluestacks, but not on my physical Android (v9) phone. I'm running the exact same version of Twidere (4.1.7) with what I believe is the same login config. Does this information help?


GDScript parsing causes large editor stalls

Godot version
v4.1.stable.official [970459615]
System information
Windows 10
Issue description
Some part of the GDScript parsing routine is run on EditorSettings change, after idle parse, and when offering code completion suggestions. In Godot 3.5 this either isn't run or is significantly faster as to not severely impact editor performance.
In Godot 4.1 editor performance is heavily impacted by "code complexity." Below is a video of I/O and CPU usage from changing EditorSettings and typing with a code_completion_delay of 0.01.
NOTE: My CPU is an i7-10700K so the 6.2% CPU usage is Godot maxing out 1 core.
C is an autoload file with just a few properties that point to "complex" Class instances

The complex classes are just a bunch of properties

EditorSettings changes cause this stalling
https://github.com/godotengine/godot/assets/3820082/6d530713-270b-4a1a-a792-5463fa3b3bbd

Code completion suggestions cause this stalling
https://github.com/godotengine/godot/assets/3820082/589b1a54-5a02-4e93-bdf7-924f6e253a2c

While these examples are contrived, the performance on my actual project is quite bad.
https://github.com/godotengine/godot/assets/3820082/99e6cf75-93e0-46de-8ba7-3b4d8ab90464

The same codebase in Godot 3.5 has no performance issues (it might be stalling only after pressing close)
https://github.com/godotengine/godot/assets/3820082/aabd88c5-bc83-4d21-b3cf-f565ae6a8765
https://github.com/godotengine/godot/assets/3820082/3395d9b8-dc37-4741-9a47-06f5a700f697

This stalling also causes input events to be dropped or late, so if I do things that aren't typing about idle_parse_delay seconds after writing code they can get messed up during this stall, like using my mouse to select text or hitting ctrl + f. I can't reliably reproduce these for a video but they're common.
Semi-relevant discussion thread https://github.com/godotengine/godot-proposals/discussions/7487
Profiling
I downloaded a version of 4.0.3 with debugging symbols and confirmed moving the code completion delay slider around (changing EditorSettings) causes a validate call if any scripts are open in the script editor, but code completion seems to just be particularly slow.
VerySleepy call stacks
When changing EditorSettings

When triggering code completion with 0.01 delay

Steps to reproduce
Have a relatively "complex" codebase and attempt to change code complete delay or idle parse delay EditorSettings, or set code completion delay to 0.01 and start typing to trigger it.
MRPs attached.
Minimal reproduction project
Here's an MRP for Godot 4.1: gdscript-parse-mrp.zip
Here's an MRP for Godot 3.5: gdscript-parse-mrp3.zip
I get basically no CPU usage or disk thrashing from the Godot 3.5 MRP.

I can reproduce the stalling in 4.x (current master).
About the stalling during completion I created a flamegraph for that:

Interactive svg
We spend way to much time parsing the dependencies. In theory those should be cached, but since the parser for completion is bound to the GDScriptLanguage::complete_code and is ref counted, it gets destroyed after every completion request, and with it all dependent parsers. (Although it seems this was introduced in #90601 and #92616 so I will probably take another look at the situation before that.)

Apparently when calling GDScriptParser.parse the parser recreates itself, which will destruct it as well and therefor remove cache entries. So apparently there is no way to cache parse results at the moment, if I'm not missing something essential.

possibly fixed by https://github.com/godotengine/godot/pull/95115
i can't actually reproduce this, but my PR does avoid invalidating parsers in this specific situation where a GDScriptParserRef is created, used, then destroyed
id appreciate if someone else could test it

Doesn't appear to have any impact on the issue. Reproduction in this MRP is a lot more obvious if Idle Parse Delay and Code Completion are set to the lowest possible values since it will retrigger a parse with each character instead of once, but here's a more complete visual example of the issue and how to recreate it.
https://github.com/user-attachments/assets/171a6ef4-dc07-4212-9ffd-d6fc3420fda6
As a bonus, setting this up also informed me that there's definitely a problem when hovering over something that doesn't resolve while holding control (shown at 1:04 in the video) but I don't know if that's related or if I should open a new PR for it.

i see, im reproducing it now, after profiling i think that https://github.com/godotengine/godot/pull/95115 either doesn't change this issue or only helps a small amount (as in it fixes a performance regression from https://github.com/godotengine/godot/pull/92616 that might be contributing to the stalling)


Posparm

Opening this  to get some feedback on the testing part / the idea in general.
This introduces a new pass that improves that handling of positional params by replacing them with their value. This way we can drop dynamic_cast checks from the semantic phase (still need to do that) and generate more efficient code. The way it mutates the ast:
bpftrace -e 'i:s:1 { print(str($1 + 1)); }' "blaat" -dd

AST after: parser
-------------------
Program
 interval:s:1
  call: print
   call: str
    +
     param: $1
     int: 1

AST after: RemovePositionalParams
-------------------
Program
 interval:s:1
  call: print
   string: laat

But this makes testing a bit harder to do, e.g.
/vagrant/tests/semantic_analyser.cpp:114: Failure
Expected equality of these values:
  expected_result
    Which is: 10
  res
    Which is: 1 <------ return code from posparam pass

Input:
kprobe:f { printf("%d", $2); }

Output:
stdin:1:25-27: ERROR: $2 used numerically but given "hello". Try using str($2).
kprobe:f { printf("%d", $2); }
                        ~~

It used to be the semantic analyser that caught this, but now its the new pass. So where do we want to test it? Should the pass get its own test suite (I think it should) and should we then move this case there. Or do we keep it in both and have the semant test then test the whole chain of passes that have to happen before it (it kinda already does with field analyser) as well. We would need a way to indicate that we expect the failure to happen in some named pass, so test_expect_fail('invalid prog', 'PassName')
I think it makes sense to have a test suite per pass but also to duplicate the test to the semantic phase so we can make sure everything stil fails the way we expect it to. I don't think we should have to indicate which pass it failed in, we just want to be sure that it fails.

Overall looks good to me - the code is much nicer and cleaner than before. I scanned through the code, but it surely needs some more reviews.
I agree with the testing approach - we should have a separate test suite for each pass, but also duplicate (at least some of) the tests into the semantic analyser to see that it all works together. It shouldn't be necessary to indicate which pass failed - the only reason I can see here would be for cases when a pass changes the result of another pass.

this would break posparam + aot and I'm not convinced its the way to go. It does have some useful commits so might pick them into seperate MRs at some point


  This should be correct, but its unclear whether we have sufficient testing (particularly on logging & recovery side) to ensure this still produces correct results.

Also just to make these more visible:
pass 9.246e+05 8.222e+05 10     +12.   LargeTransactionBenchmark    SingleStatementUpdate
pass 1.391e+06 1.244e+06 10     +11    LargeTransactionBenchmark    SingleStatementSelect
pass 9.045e+05 7.908e+05 10     +14    LargeTransactionBenchmark    SingleStatementInsert


Fix CustomShuffleReader replacement when decimal types enabled

This pull request is to fix the bug of AQE with decimal, which is reported in issue #1670.

build

@sperlingxx Can this impact other types too?  We support shuffling nested types like arrays, maps, and structs, but GpuCustomShuffleReader does not.

Thanks @sperlingxx . I applied this change to my branch and ran quite a few queries with decimal + AQE enabled and it worked well.

@sperlingxx Can this impact other types too? We support shuffling nested types like arrays, maps, and structs, but GpuCustomShuffleReader does not.

Hi @revans2, I added ArrayType and StructType as supported types. And MapType is not supported in ShuffleExchangeExec.

Is there a way to make this fail in a way that we can fallback to the cpu?

I believe when we configure spark.rapids.sql.decimalType.enabled with false, it will fallback to cpu. The fallback strategy will fail at here because parent plan(CustomShuffleReaderExec) fallbacks but child plan does not.

build

build


failed to build: Missing label

failed to build: invalid builder paketocommunity/rust: builder index.docker.io/paketocommunity/rust:latest missing label io.buildpacks.builder.metadata -- try recreating builder

New to this stuff, so not sure if this is the right place to start.
I get this error trying to use this builder with controlplane cli.
Any tips would be greatly appreciated

No worries. The problem is happening because docker.io/paketocommunity/rust isn't a builder. It's a buildpack.
The easiest way to kick off a rust build is to pack build <image-name> -b docker.io/paketocommunity/rust. The lowercase -b says use this specific buildpack, which is the Rust buildpack.
If you need a builder, you can create one but we're not publishing a builder at this time.
There is a builder definition file in the repository though, and instructions here for customizing and creating a builder from that definition.
Hope that helps!

Ah yes, and the control plane CLI doesn't have a buildpack option : 😩
Made it harder to tell that builder !== Buildpack.
I tried using pack with the paketo builder and this buildpack and it fails for other reasons...something about openssl. That made me hit the "okay, not a problem for tonight" wall

I tried using pack with the paketo builder and this buildpack and it fails for other reasons...something about openssl.

Let me know specifically what command you're running and the error you're getting. The Paketo builders should have openssl & dev libraries installed, but things can happen. Happy to help sort it out.

command is
pack build myapp --builder paketobuildpacks/builder:base --buildpack<EMAIL_ADDRESS>
This is basically a little like 20 line webserver using warp and tokio but it seems to be related to
Compiling openssl-sys v0.9.87

and the error is
error: failed to run custom build command for `openssl-sys v0.9.87`

      Caused by:
        process didn't exit successfully: `/workspace/target/release/build/openssl-sys-f98e335d55700c71/build-script-main` (exit status: 101)
        --- stdout
        cargo:rerun-if-env-changed=X86_64_UNKNOWN_LINUX_GNU_OPENSSL_LIB_DIR
        X86_64_UNKNOWN_LINUX_GNU_OPENSSL_LIB_DIR unset
        cargo:rerun-if-env-changed=OPENSSL_LIB_DIR
        OPENSSL_LIB_DIR unset
        cargo:rerun-if-env-changed=X86_64_UNKNOWN_LINUX_GNU_OPENSSL_INCLUDE_DIR
        X86_64_UNKNOWN_LINUX_GNU_OPENSSL_INCLUDE_DIR unset
        cargo:rerun-if-env-changed=OPENSSL_INCLUDE_DIR
        OPENSSL_INCLUDE_DIR unset
        cargo:rerun-if-env-changed=X86_64_UNKNOWN_LINUX_GNU_OPENSSL_DIR
        X86_64_UNKNOWN_LINUX_GNU_OPENSSL_DIR unset
        cargo:rerun-if-env-changed=OPENSSL_DIR
        OPENSSL_DIR unset
        cargo:rerun-if-env-changed=OPENSSL_NO_PKG_CONFIG
        cargo:rerun-if-env-changed=PKG_CONFIG_x86_64-unknown-linux-gnu
        cargo:rerun-if-env-changed=PKG_CONFIG_x86_64_unknown_linux_gnu
        cargo:rerun-if-env-changed=HOST_PKG_CONFIG
        cargo:rerun-if-env-changed=PKG_CONFIG
        cargo:rerun-if-env-changed=OPENSSL_STATIC
        cargo:rerun-if-env-changed=OPENSSL_DYNAMIC
        cargo:rerun-if-env-changed=PKG_CONFIG_ALL_STATIC
        cargo:rerun-if-env-changed=PKG_CONFIG_ALL_DYNAMIC
        cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64-unknown-linux-gnu
        cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64_unknown_linux_gnu
        cargo:rerun-if-env-changed=HOST_PKG_CONFIG_PATH
        cargo:rerun-if-env-changed=PKG_CONFIG_PATH
        cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64-unknown-linux-gnu
        cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64_unknown_linux_gnu
        cargo:rerun-if-env-changed=HOST_PKG_CONFIG_LIBDIR
        cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR
        cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64-unknown-linux-gnu
        cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64_unknown_linux_gnu
        cargo:rerun-if-env-changed=HOST_PKG_CONFIG_SYSROOT_DIR
        cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR
        run pkg_config fail: `PKG_CONFIG_ALLOW_SYSTEM_CFLAGS="1" "pkg-config" "--libs" "--cflags" "openssl"` did not exit successfully: exit status: 127
        error: could not find system library 'openssl' required by the 'openssl-sys' crate

        --- stderr
        thread 'main' panicked at '

        Could not find directory of OpenSSL installation, and this `-sys` crate cannot
        proceed without this knowledge. If OpenSSL is installed and this crate had
        trouble finding it,  you can set the `OPENSSL_DIR` environment variable for the
        compilation process.

        Make sure you also have the development packages of openssl installed.
        For example, `libssl-dev` on Ubuntu or `openssl-devel` on Fedora.

        If you're in a situation where you think the directory *should* be found
        automatically, please open a bug at https://github.com/sfackler/rust-openssl
        and include information about your system as well as this message.

        $HOST = x86_64-unknown-linux-gnu
        $TARGET = x86_64-unknown-linux-gnu
        openssl-sys = 0.9.87

        ', /layers/paketo-community_rustup/Cargo/registry/src/index.crates.io-6f17d22bba15001f/openssl-sys-0.9.87/build/find_normal.rs:190:5
        note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Thanks!

That code seems to fail with
ERROR: failed to build: invalid run-image 'paketobuildpacks/run-jammy-base:latest': run-image stack id 'io.buildpacks.stacks.jammy' does not match builder stack 'io.buildpacks.stacks.bionic'

So maybe I just need to use the full until the base is updated.

Oh, sorry. I missed that you are still using the Bionic builder, which goes out of support at the end of this month. We're in the process of switching documentation & things to point to the Jammy builders.
Use paketobuildpacks/builder-jammy-full:latest as your builder.
pack build myapp --builder paketobuildpacks/builder-jammy-full:latest --buildpack<EMAIL_ADDRESS>--run-image paketobuildpacks/run-jammy-base:latest

I'm seeing that the images are updated, and using builder-jammy-base still has the same error.

Hmm, that's odd. The stack build image has the library:
→ docker run --rm -it index.docker.io/paketobuildpacks/build-jammy-base:latest /bin/bash -c "dpkg -l | grep -i ssl-dev"
ii  libssl-dev:amd64          3.0.2-0ubuntu1.9                        amd64        Secure Sockets Layer toolkit - development files

Maybe we need to add other libraries? I haven't had chance to dig into this. cc @dmikusa

Hmm, I think I overlooked this originally and just assumed the dev files were missing:
run pkg_config fail: `PKG_CONFIG_ALLOW_SYSTEM_CFLAGS="1" "pkg-config" "--libs" "--cflags" "openssl"` did not exit successfully: exit status: 127

I believe this 127 is telling us that pkg-config isn't installed. Digging into the image, I don't see it either. I think we need that to be added too. Doing a manual test, after I install that, I can run the command Rust is complaining about. I'll send a PR.


After application deployment sudden increase in load resulting in slow connection creation

Hello,
I have a Spring Boot application with hikari connection pool. The application was deployed in two instances(Say pod A and pod B).After deployment 3-4 requests were executed in pod A for warmup and the application was able to process 84 concurrent requests efficiently with average response time in milliseconds. But Pod B where no prior requests were sent for warmup,43 requests came and all had huge latency and 3-4 seconds was being spent on average in Hikari.getConnection() method.My Hikari pool configuration is as below :
hikari:
  maximum-pool-size: 100
  max-lifetime: 30000
  minimum-idle: 10

Is this a expected behaviour without any warmup under load?

Here is the screenshot of code level time consumed

This doesn't seem like an issue with Hikari. Why have you set your minimum connections to 0, when your application needs to deal with sudden spikes in activity?
You should look into the database technology you're using to see what can be done to improve the time to create a connection.
Ultimately, creating new connections is an expensive operation, and you may find that a non-zero minimum pool size is necessary for your use.

+1

@mdtamim Set initializationFailTimeout to some reasonable value, like 30000 (i.e. 30 seconds), and set system property com.zaxxer.hikari.blockUntilFilled=true. This will block the application startup for 30 seconds or until minimumIdle connections are added to the pool, whichever comes first.


[SMALLFIX] Make worker not crash if a storage location cannot be accessed

Currently Alluxio worker will fail to start if one of the storage disk is failing. This change makes Alluxio worker disregard the failed location and start normally with an error message logged.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/118/
Test PASSed.

Merged build finished. Test PASSed.

can you cherry-pick this patch to 1.8 branch after merged?

@bf8086 Thanks for this fix! Can you create a ticket so we can track this bug?

@gene I created a new ticket to track the issue.
@apc999 Will do.


Update the swagger file

Fix #5168 by correcting the types of properties used in ProjectMetadata

Pull Request Test Coverage Report for Build 7487

0 of 0   changed or added relevant lines in 0 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage remained the same at 64.53%

Files with Coverage Reduction
New Missed Lines
%

src/common/scheduler/scheduler.go
1
81.87%

Totals

Change from base Build 7483:
0.0%

Covered Lines:
11076

Relevant Lines:
17164

💛  - Coveralls


[cri] use 'UserSpecifiedImage' to set the image-name annotation

#5002 image annotation can helpful for debugging and auditing the workload. However, when an image has multiple tags, the image originally obtained may not be the one actually specified by the user. Starting from cri-api v0.28.0, a UserSpecifiedImage field is added to ImageSpec. It is more appropriate to use UserSpecifiedImage.

/cc @fuweid


Convert git commit summary to valid UTF8.

The summary string ends up in the database, and (at least) MySQL & PostgreSQL require valid UTF8 strings.
Fixes #28178

Why validateUTF8 is better than try to convert them into UTF8?

Why validateUTF8 is better than try to convert them into UTF8?

Because you do not know what encoding it was, then how could you "convert" it?

Why validateUTF8 is better than try to convert them into UTF8?

Because you do not know what encoding it was, then how could you "convert" it?

We can detect it and convert it like we did on git files if the content is long enough.

Why validateUTF8 is better than try to convert them into UTF8?

Because you do not know what encoding it was, then how could you "convert" it?

We can detect it and convert it like we did on git files if the content is long enough.

If you mean guessing the content encoding by something like chardet package, I think it's feasible while not necessary in my mind, because the git commit will be displayed by the client in the end, many git clients also use "UTF-8" (I doubt whether there are enough clients do encoding guessing). And since the commit summary (the first line) is very short, not sure whether the guessing algorithm is accurate enough.
It's also OK to do the encoding guessing if most people like, I am fine with either approach.

Some references. According to git official document, using non-UTF-8 encoding is not encouraged, and Gitea server itself doesn't (and should never) use the encoding config option IMO .....
https://git-scm.com/docs/git-commit-tree


Fixed generation of ERD with composite keys. Added possibility to generate diagram with relation fields (Closes #135)

Fixed composite primary keys on diagram
Added possibility to generate relation fields

@all-contributors please add @Vitalii4as for code


Code completion for Deployment Config

What would you like to be added:
It would be nice to suggest the candidate fields and its description when editing Deployment Config on the web.
Like Github Actions:

Why is this needed:

/close


Fix pandas requirement.

Issue, if available:

Pandas 1.1 introduces breaking changes.

Description of changes:

limited pandas version to <=1.1

Fixes

https://github.com/awslabs/gluon-ts/issues/965
https://github.com/awslabs/gluon-ts/issues/958
https://github.com/awslabs/gluon-ts/issues/959

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Note: This was apparently an unintended change. I've opened an issue over at https://github.com/pandas-dev/pandas/issues/37267


Add setInterval

It would be awesome if there were support for setInterval.

Hi, thank you, I'm not using this project personally anymore, but pull requests are very much welcome and I would be happy to accept such change :)


fix: reenable reflinks

This PR reenables the use of reflinks. Although this might require some additional testing on different platforms I thought I'd create a PR to be able to start the process. :)
Since this is merged (https://github.com/mamba-org/rattler/pull/485) I think it should "just work"™

Although Im not sure how to properly check this, on my windows dev-drive reflinks seem to work.

To test this I propose:

Install all pixi examples
remove part of the cache
Reinstall all examples
Run all pixi run starts in the examples
Try to edit a python file in the cache to see if that breaks the environments and visa-versa
If this succeeds on all platforms I would be fine merging this. If there is any other tests you can come up with please share! Last time this failed in a really hard to understand way.

Reinstall all examples

Do you mean to first remove the .pixi folder

Do you mean to first remove the .pixi folder

Maybe that after first trying to reinstall and see if that broke anything.

Here is a script that does the above.
# Navigate to the examples
> cd examples

# Clear all existing `.pixi` directories
> glob **/.pixi | each { rm -rf $in }

# Install all examples
> glob **/pixi.toml | each { pixi i --manifest-path $in }

# Delete content from cache directory
> rm -rf $"($env.PIXI_CACHE_DIR)/pkgs"

# Run a few examples
> pixi run --manifest-path ctypes-factorial/pixi.toml start
> pixi run --manifest-path cpp-sdl/pixi.toml start
> pixi run --manifest-path flask-hello-world/pixi.toml start
> pixi run --manifest-path opencv/pixi.toml start
> pixi run --manifest-path pypi/pixi.toml start
> pixi run --manifest-path rerun_example/pixi.toml start
> pixi run --manifest-path turtlesim/pixi.toml start

# Clear all existing `.pixi` directories
> glob **/.pixi | each { rm -rf $in } | null

# Reinstall flask-hello-world and corrupt the cache.
> pixi install --manifest-path flask-hello-world/pixi.toml
>  echo 'assert(False, "BoOm I Br0k3 1t")' o>> $"($env.PIXI_CACHE_DIR)/pkgs/flask-2.3.3-pyhd8ed1ab_0/site-packages/flask/__init__.py"
> pixi run --manifest-path flask-hello-world/pixi.toml start

Essentially the last part checks if you are using reflinks or hardlinks by installing flask from cache, then corrupting a file in the cache. If hardlinks are used then running the command again should show you an error. Something like:
F:\examples\flask-hello-world\.pixi\envs\default\Lib\site-packages\flask\__init__.py:103: SyntaxWarning: assertion is always true, perhaps remove parentheses?
  assert(False, "BoOm I Br0k3 1t")

If reflinks are used this error should not be there because the file is copied when its edited.
This however does not check if the file is not copied..

I'm pushing this one to the next release.


ENYO-2521: Make viewport a spotlight container without remembering last focus

Issue:
When there is a disabled item between items on scroller, there is a
chance to return paging control as a best match even when there is
focusable item below. This is because paging control is having smaller
y distance than below item.
Fix:
It is risky to alter _getPrecedenceValue logic on spotlight.
But, we have new option that is not remembering last focus on container.
We set viewport of scroller as spotlight container and set option
spotlightRememberFocus as false. So, spotlight can consider items first
on 5way focus move.
Enyo-DCO-1.1-Signed-off-by: Kunmyon Choi<EMAIL_ADDRESS>

Closing in favor of https://github.com/enyojs/moonstone/pull/2548


Captcha

I assume this means the project is pretty much dead?
If that's the case, I just wanted to thank everyone for all the hard work

I'm not sure, to me library it's still working correctly, are you having problems?

I'd assume the patch hasn't hit you yet
You now recieve the error message: "[Error: There was an error validating the code you entered. Please try again.]"
Due to a new captcha placement in the trade window. It seems to be on both the send and accept of the offer.

http://steamcommunity.com/groups/tradingcards/discussions/1/622954023422884592/#p4
"We’ve excluded a few of the existing third-party trading services from this requirement so they can continue to function." Might be because of that then.
Still I don't think this is a long term solution, I'm pretty sure there will be some more changes on this (bad) update

Facepalm..
My opinion on this is stated here, if you care about the future of automated betting. Better hope valve actually does something.
http://www.reddit.com/r/Steam/comments/2rw4f9/steam_offers_require_you_to_enter_captcha_now/cnjwc3v

"We’ve excluded a few of the existing third-party trading services from this requirement so they can continue to function."
Hoping they provide a way to opt-out.  The number is only estimate, we still don't know the exact number.
*Account with 5401 trades has captcha. Account with 7530 trades does not.
edit. Market trades might also be used to determine this. It's still very unclear how the whitelist works."
If this is true it may be possible to use captcha solvers at first then trade normally.

I'm not sure if the whitelist is account-related or even machine-related. I tried making a trade offer with my bot account from browser and I had to type in the captcha, still, automatic trade offers are working fine.

Strange. Do your bots have a lot of trades?

Yes, for sure it passes the 7000 cap

Oh. You can't reproduce this behavior with a newer account that has less trades then?

"The captcha isn't even that effective.
http://i.imgur.com/a6JuQPA.png
From here the OCR is easy. Might stop your average bot dev, but the scam organizations? Please, Valve. Don't kid yourself."
-cdhowie from reddit
Combined with something like https://www.npmjs.com/package/nodecr
We may be able to bypass it though it would increase load. It's just an idea but one you could play with.

It would work (DeathByCaptcha, for example, reportedly solves these captchas), but it's not a permanent solution. Eventually they will change the captchas and we will have to start all over.
If the whitelist is automated and based on number of trades, it would be easy to set up two bots trading items to each other all day. But Steam would probably notice and block these accounts.

@filsmick No, I have another bot account that does some interaction with the main bot (the working one) and it has a lot less trades and it's getting the captcha error when making offers.
@pmk010 as jessecar said, in my opinion bypassing captcha it's not a good solution for this.

@Ax6 it's not a good solution but for many small time bot owners it's the only solution for now. Getting steam to whitelist a few small bots isn't going to be possible as of yet, perhaps we can hope that they implement a proper system for people to get bots whitelisted, or even better, a simple option to disable captchas.

It is account specific and bypassing captcha isn't a good idea to stay on good terms with them.
Just contact them, I got my bots whitelisted in around 30 minutes.

@Kuba77 You did? Really?
Did you contact them via a support ticket?

@Kuba77 Well I didn't get my bots, or my clients' bots whitelisted and I submitted a support ticket a good 18 hours ago.

http://www.valvesoftware.com/email.php
Try Tony Paloma, he should help you @filsmick

Thank you a lot @Kuba77, I will try! It's definitely the best option, if they answer.

Thanks, I'll definitely try that and let you know the results!

@Kuba77 Do you mean the bots' usernames or also the passwords?

Guys, do you know what is "v" parameter in tradeoffer/new/captcha?v=... and how I can find it? I know only that it's always growing up.

I believe it is the current timestamp

Only usernames

Hi all!
How to send trade without captcha?
Or how to give image captcha?

@pmk010 interestingly enough, they have switched to reCACPTHA

As a workaround, Do you think it could be possible to get an image of the captcha and have one of the real users of an automated trade solve it for the bot?

On Jan 10, 2015, at 8:09 PM, jwigley<EMAIL_ADDRESS>wrote:
@pmk010 interestingly enough, they have switched to reCACPTHA
—
Reply to this email directly or view it on GitHub.

https://twitter.com/SteamDB/status/554045816323985408/photo/1
Valve's own captcha now replaced with Google's No CAPTCHA. And I think Valve will continue to change this in the near future.
So here is my decision: node-steam-tradeoffers is not going to include any changes involving captcha. Any PRs that feature such changes are not going to be merged, so don't waste your time.
Work with Valve on getting your bots whitelisted.

@Alex7Kom, Может все-таки реализовать какой-нибудь функционал, чтобы можно было как-то брать изображение капчи, и передавать введеный код в Ваш модуль?

@Maxpain177 Please use English.
And as I said NO changes will be made to the library regarding captcha. Better use your energy to pursuit Valve to remove captcha or whitelist your bots.

@Kuba77 Drunken_F00l denied any "manual" whitelist. So, who is lying? :grinning:

@scholtzm Can I see the source?

@filsmick Updated my previous comment.

Thanks. Then the only way would be full auto trading of crates back and forth with node-steam-trade I guess.

http://steamcommunity.com/groups/tradingcards/discussions/1/622954023422884592/

Interesting info about the auto-whitelisting trade count: http://www.reddit.com/r/SteamBot/comments/2rwq7l/psa_trade_offers_now_require_captcha/cnkhmfb

This is only very rough estimate. Me and some people (all with high amount of trades) do not have the captcha. Talking with others, we have noticed that account with 5401 trades has captcha. Account with 7530 trades does not.
However, another guy with ~4950 trades also does not have the captcha, but he has a lot of market purchases.
Anyway, take this information lightly as it's all based on very small data sample.

I tried contacting Tony Paloma as suggested by @Kuba77 however have yet to receive a reply. As for the suggestion made by @filsmick, is currently seems to be the only possible option, though I'm not sure whether it would be "above board" as such. With reCaptcha there is not much that can be done to bypass it.
For now I may consider doing some testing between two alts, see if we can find the threshold for becoming whitelisted. At the moment my assumption is that it's a combination of market transactions and trades, perhaps around 6000 in total, but that's just a guess.

Also tried contacting Valve as suggested by @Kuba77 no response yet.
2799 trades + 3934 market transactions here, still have captcha.

If someone has time, create a farming bot that trades from one bot to another non-stop (using regular trades, not trade offers) until the captcha disappears ^_^

@srabouin I'm doing it now, I'll let you know the results.

@filsmick feel free to tell me your results too.

Each trade takes about 13 seconds, I have 2500 successful trades in my account, making another 2500 will take about 11 hours. I don't know how to make trades faster, so I neet to wait at least 11 hours :-1:
Hope this will switch off captcha and my accounts will not be banned.

I currently also have my bots farming trades.
As far as trades go is it counting the old school manual trades as in through the steam trade window also? (I assume it does, it would be silly if it didn't) Also how can I see the how many trades one of my accounts has done, is it listed somewhere on steam? I'm guessing it is and it's somewhere really obvious that I've blindly overlooked.

@doktokto Please let us know when you've finished!
If this does fix the captcha issue, I'd be happy to publish some code that trades emotes or something between accounts.

Spent some time getting from 6700 combined (trades + market) to 7000. Still have captcha.
Maybe it's based on more than just these two figures.

@bluejellybean
Yet scored only one thousand, then the bot broke, had to restart, noticed recently, will have to wait even longer and monitor because it can break again.

Hoping it wasn't a one time whitelist for all accounts that had over X transactions at the date they added the captcha.

@bobjane It's probably around 7500 combined
@jwigley Indeed...
My bot is almost ready, I had some trouble getting a bot account without Steam Guard restriction yesterday.

5200, still captcha, still making trades.
Another bots have 10000+ trades.

Someone reported on the bp.tf forum, that he has captcha with 22k trades. sigh

@scholtzm Any clue why he still has captchas?

No idea frankly.
btw. Regarding the trade count. Inventory history shows some dummy number while profile showcase shows accurate number. You have to go couple pages back from your max inventory history page.
Example: http://i.imgur.com/0v2ge3U.png
9265 - actual trades
9596 - bugged number
Also notice the page number on the right side. 11 empty pages follow after page 309.

Solution.

render captcha image with PhantomJS
show the image with captcha for user
auto-input text-field of reCaptcha
retrieve token and add &captcha= param for tradeoffer.

I would like to point out a strange behavior for whitelisted accounts in case you happen to have the same situation. It seems that often sending a trade offer actually results in a 'error no captcha given' which won't occur on the second retry. At least this is what I noticed happening to my bot.

@Maxpain177  Did u make this work already?

@filsmick He says he had the captcha sporadically. Apparently only twice. Maybe the whitelist is buggy? No idea.
I hope this doesn't turn into another ghost chase. Still waiting for someone to confirm you can even become whitelisted just by trading.

Epic win. 5762 trades. No captcha anymore.

so it's not a ghost chase then :3

I dont understand why doesnt it take so long to farm ur trades up?
Sending trades consecutively doesnt work?

@doktokto share? :dancers:
@Maxpain177 To send the image to the person who sent the trade offer requires you to be friends with them though, I don't think this is a valid solution

Bot 1 and bot 2.
Bot 1 invite bot 2 to trade.
When trade init bot 2 add item.
When bot 1 see that item added he add item too.
When bot 2 see that item added he mark that he is ready.
Bot 1 mark too.
Bot 2 accept.
Bot 1 too.
Add to bot 1 OnTradeClose() sending invite again.
Each trade takes 11-16 seconds.
One of bot sometimes crash (randomly), need to add try{} catch {} in that places where was error.
When I started it one of bot (random) crashed after 100 trades. After try catch adding bot's crashed each 1000 trades. Maybe you will make it better.

@vudung45. Yes

@srabouin . on my site this image shows for user

@Maxpain177 Btw how do u make the google recaptcha thing appear captcha instead of ticking the option?

@vudung45 just text

@Maxpain177 What do you mean, "just text"?

Because without clicking the button, it wont show the captcha

Maybe it's in a noscript tag...?

We have to wait for an answer form Maxpain then :-? How to make it present the captcha image :(

@vudung45 PhantomJS. (page.render('captcha_231321.png'))

@Maxpain177 I have used PhantomJS myself. Here you're rendering the page as an image and saving it as a file named "captcha_231321.png". Are you 'clicking' the checkbox with the headless browser?

@filsmick yes.
page.evaluate(function() {
	a = document.getElementById('recaptcha-anchor');
	var e = document.createEvent('MouseEvents');
	e.initMouseEvent('click', true, true, window, 0, 0, 0, 0, 0, false, false, false, false, 0, null);
	a.dispatchEvent(e);
});

@Maxpain177 Okay, thanks.

So the token that google returns gonna be new captcha param right?

@doktokto I tried your method to mass farm trade counts, but I cannot get past 1 trade accept, after I get the item, and ask another trade, it goes to Trade Decline from Bot.cs:600 , My bot did not respond with a decline, so I must be missing something when I accepted the first trade. Did you encounter this issue? Thanks.

@ippo20 You cannot do two trades at once. Is that what you were trying to do?

You are possibly sending the next trade request too soon while the trade is not finished.

@scholtzm I did Sleep 10s just to make sure, plus the trade was accepted on both bots, an item was removed and an item was added on the other. Then I called Bot.OpenTrade on the second bot for the next trade loop and Bot.OpenTrade return was True, Sadly it says Trade Decline in console.

I just got done farming 5,504 trades and no more captcha appears!
http://steamcommunity.com/id/trollvin

Perhaps the accounts age is also taken into consideration. Consider also that any variables that go into an account being whitelisted might not be weighted linearly...

Good news guys, Some of my bots got out of the CAPTCHA now. I'll list which ones can trade and how many trades they have below!
=== NO CAPTCHA ===
(Also my two oldest bots...)
BillySheen: 5208 | Steam: http://steamcommunity.com/profiles/76561198147778631/
TicketTony: 5001 | Steam: http://steamcommunity.com/profiles/76561198151823380/
=== STILL HAS CAPTCHA ===
FabriPhile: 4139 | Steam: http://steamcommunity.com/profiles/76561198154320195/
ShinySheen: 4095 | Steam: http://steamcommunity.com/profiles/76561198154506673/

Got it, error in my program and it does sometimes fail, i got to loop at around 20+ then fails between 15-25, i just close all bots then open it with bat files loop every 1 min whether it fails or not.

In the spirit of open source, one of you guys should share your bot so that the entire trading community can do that easily :D

@srabouin Actually, I just followed @doktokto instuctions. It was simple when you follow it to the letter

@ippo20 I know it's simple, but a lot of people don't know how to write their own bot and would probably enjoy removing the captcha.

@srabouin I am ok to share, but it wont matter because if they cant write that simple code, they could use the TradeOffer feature anyway. Plus, I hate to share my code that is a copy of @doktokto I think he should share since his was more better than mine

@ippo20 can you please confirm if the 5000+ trades per bot worked for you?

I plan on releasing the code I used to farm trades soon. If anyone would like to help test it before I publish it feel free to add me on Steam

I would share mine but it's linked with my bots abstracted item routing class for moving around items internally between all my bots.

@alvinl thanks! my team already contacted you.
@dsisco11 can you atleast share the logic behind your code?

@darylcharm pls, send me yours contact (email or other) - foozzione[dog]gmail.com

Ok, I don't have much time because of exams. I have contacted Steam support obviously to no avail... I have tried to email them too. Again nothing, I think farming trades must be the next best thing. @alvinl I have added you on Steam. I don't see why they don't have an easy turn off bit in the settings.
I just want to get on that whitelist.

Because they don't want regular users to turn it off and cry about losing
items I think.
13 sty 2015 11:55 "Alexander C E Lewis"<EMAIL_ADDRESS>napisał(a):

Ok, I don't have much time because of exams. I have contacted Steam
support obviously to no avail... I have tried to email them too. Again
nothing, I think farming trades must be the next best thing. @alvinl
https://github.com/alvinl I have added you on Steam. I don't see why
they don't have an easy turn off bit in the settings.
I just want to get on that whitelist.
—
Reply to this email directly or view it on GitHub
https://github.com/Alex7Kom/node-steam-tradeoffers/issues/67#issuecomment-69727476
.

@Kuba77, I just hope all of the Steam Stealer's bypass this captcha stuff quickly. Although it would be nice to have a system that would stop these, captchas is not one of them.

@AceLewis SteamStealer is actually really terribly written and mostly made out of different snippets found on Russian forums. Even if there was a way to bypass it which I don't think is a case I doubt they'll be able to find it. Exploit they used to trade is quite easy to find so there's really no comparision.

Edit: "Did you know? We currently work after the Captcha Update that Valve implemented (even the NoCaptcha Recaptcha!). We have also just opened public sales once again!"

@Kuba77 : I dont even get that =)). So they have found a way to bypass that :-?

Yes,  they did

The farming trades thing seems to be the best answer for now cause it's simple but yet not confirmed working on all accounts yet.
I would like to know how the bot could render the captcha then ask the bot-user to solve the captcha

@vudung45 PhantomJS.

@filsmick : I'm doin this on a different form ( not node, but jessy c# steambot), i would like to see the captcha thing showing up on my browser first in order to do my code.

@filsmick : Have u done this before, what div does the code have to do mouseevent on:-?

Don't have captcha myself, but I think you can just get link to it and
download.

@vudung45 I have used PhantomJS, but not for this. See the code posted by @Maxpain177, and experiment yourself with Chrome and the Web Inspector to see what elements to operate on.
Also, to all: I got the captchas unlocked at ~5200 trades.

I've published the code I used to farm my trades here - https://github.com/alvinl/auto-trader. It still has a few issues but I plan on knocking them out later today. Also, it's pretty quick

Thanks for saving the day Alvin! :)
On Wednesday, January 14, 2015, Alvin Lazaro<EMAIL_ADDRESS>wrote:

I've published the code I used to farm my trades here -
https://github.com/alvinl/auto-trader. It still has a few issues but I
plan on knocking them out later today. Also, it's pretty quick
[image: lusnhei7lv]
https://cloud.githubusercontent.com/assets/730112/5724044/c6b5b6da-9b16-11e4-82b1-2b980a1db2a4.png
—
Reply to this email directly or view it on GitHub
https://github.com/Alex7Kom/node-steam-tradeoffers/issues/67#issuecomment-69771952
.

Confirmed this is a work-around, 5001 trades and i got no captcha.

What is the delayed time between the two trades that u guys r using?

Looks like Valve have replaced the captcha with a trade email verfication step.

The issue is no longer relevant, so I close this.
Issue ticket about Trade Confirmation: https://github.com/Alex7Kom/node-steam-tradeoffers/issues/68


New-SPOTenantSite documentation not up to date

https://github.com/OfficeDev/PnP-PowerShell/blob/master/Documentation/NewSPOTenantSite.md missing required parameter description for the TimeZone parameter

Thanks Vesa. Fixed in PR #43 . Will close the issue.

Was not that urgent... just noticed that it was there... but no harm being fast.


[BUG] Add-PnPApp fails with a ResourceNotFound exception.

Add-PnPApp fails with a ResourceNotFound exception.
After creating a new team site and creating the site app catalog, it's not possible to add an app package. When checking the site in Sharepoint, the app catalog does exists. Some other checks:

Get-PnPApp -Scope Site : same error
Get-PnPList -Identity AppCatalog : no problem
Get-PnPSiteCollectionAppCatalogs : Returns ErrorMessage = "Access denied." for the created site.

Reporting an Issue or Missing Feature
Issue
Expected behavior
App should be uploaded into the site app catalog
Actual behavior

Steps to reproduce behavior
Connect-PnPOnline -Url $adminUrl -Interactive
$newSiteUrl = New-PnPSite -Type TeamSite -Title $siteAlias -Alias $siteAlias
Add-PnPSiteCollectionAppCatalog -Site $newSiteUrl
Connect-PnPOnline -Url $newSiteUrl -Interactive
Add-PnPApp -Path xyz.sppkg -Scope Site
What is the version of the Cmdlet module you are running?
1.10.0
Which operating system/environment are you running PnP PowerShell on?

[ X ] Windows
[ ] Linux
[ ] MacOS
[ ] Azure Cloud Shell
[ ] Azure Functions
[ ] Other : please specify

@RobinDijkstra - did you create a site collection app catalog ?
You can use Add-PnPSiteCollectionAppCatalog to first create the app catalog and then upload your apps
https://pnp.github.io/powershell/cmdlets/Add-PnPSiteCollectionAppCatalog.html

@gautamdsheth - Yes, I did. See step 3 in the steps to reproduce. I've also checked that the site collection app catalog is actually present.

Can you maybe try this in another site collection and check or maybe in a different tenant ? Could be some site collection or tenant specific issue ? Not able to reproduce it.

I've tried the steps to reproduce several times on our two tenants and also on my personal develop tenant. On all tenants the behavior is the same.
The problem is only with new sites, adding an app to older, already existing sites works just fine.
The steps to reproduce are a simplified version of an extended set of scripts to create and configure new sites. These scripts have always worked and they don't work anymore. So something must have changed.

Could this be race conditions?
Depending on what kind of site you create (especially TeamSites that are more complex) the site may not be ready yet to receive your app.
Please for the sake of testing try and add a Start-Sleep 120 and see if the behaviour changes
hth

After a few hours I've tried the Add-PnpApp cmdlet again, it made no difference. I've also tried it on a site I've created a few days ago and that also didn't work.

I've found the problem. For those of you who run into the same issue (and those who are just interested):
According to MS documentation, the account that is used to create the site collection app catalog must be Site Collection Administrators on both the tenant-level App Catalog and the target Site Collection. Normally I can ignore these kinds of remarks because I'm a Global Administrator or a Sharepoint Administrator. In this case however it's really necessary to be a Site Collection Administrator on both sites.
So I've added myself as a Site Collection Administrator to the tenant-level App Catalog site. In the steps to reproduce I've added a step to be Site Collection Administrator in the newly created site, before creating the site collection app catalog:
Connect-PnPOnline -Url $adminUrl -Interactive
$newSiteUrl = New-PnPSite -Type TeamSite -Title $siteAlias -Alias $siteAlias
Set-PnPtenantSite -Identity $newSiteUrl -Owners $loginName
Add-PnPSiteCollectionAppCatalog -Site $newSiteUrl
Connect-PnPOnline -Url $newSiteUrl -Interactive
Add-PnPApp -Path xyz.sppkg -Scope Site
This is the solution for all my three tenants. Still, I don't know what has changed because it used to work. I suspect it has something to do with the change that MS has rolled out on the App Catalog Site (which now is called the Apps site).
@gautamdsheth, @AndersRask: thanks for your input!


relative path issue for CSM

Hi, when I called CSM using relative path on Linux, for example, ../dscsm048 A UFCI0301.SWX, the program fail and return "Configuration file not found". When I use the absolute path: /home/fa/run/dscsm048 A UFCI0301.SWX, the program did not come across with the problem.
I found that when I use the relative path, the program wrongly searching the file DSSATPRO.L48 with the path "/DSSAT48/DSSATPRO.L48". I found out that replacing the number 8 with 12 will solve the problem.

Hi @Eugllena
Yes, the program will not run because of the DSSATPRO.L48. You have two choices, changing the DSSATPRO for the specific path or using the DSSAT_HOME env variable.
Thank you.
Fabio Oliveira


tracker-miners: 3.5.2 -> 3.5.3

Description of changes
https://gitlab.gnome.org/GNOME/tracker-miners/-/blob/3.5.3/NEWS?ref_type=tags
Related to CVE-2023-43641, see https://github.blog/2023-10-09-coordinated-disclosure-1-click-rce-on-gnome-cve-2023-43641/

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/libexec/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

aarch64 build works, not able to confirm the sandbox change mitigates the exploit in libcue

Working on the backport


Delete table database versioning (revised PR with unit tests)

This takes the changes to versioning.py from https://github.com/ministryofjustice/data-platform/pull/2465 and adds unit tests.
We decided this morning that the delete table endpoint should return early if a schema doesn't exist, even if there may be data hanging around, so some of the changes in that PR won't be needed any more. We don't want to delete data if there is no metadata, and in future we will ensure that the schemas being generated is a prerequisite of loading data into a table.
So the expected behaviour when deleting a table is

If the schema is not found, do nothing
If the glue table is there, delete it from both the current and new version
Delete data files from all historical versions
Create a new version of the database with the current version number

Part 2 is weird - dropping a table from a historical table will catastrophically break any consumer code that depends on it, so this breaks our versioning guarantees. Even if we do want to nuke everything, there's still a gotcha that we only drop the prior version of the table.

Decision after speaking to @PriyaBasker23
For now, we will just delete the table & data from the new version. This deviates from our original plan of deleting from all historical versions "delete means delete", but we had some doubts about implementing contradictory behaviour (i.e. creating a new major version while at the same time making breaking changes to existing versions doesn't make sense).
I'll stick another ticket in to revisit the delete means delete behaviour. For now the exact logic doesn't matter as long as the endpoint behaves consistently.

Need to rebase this monday due to conflicts.
Remaining stuff

[ ] Bump the version
[ ] Make sure glue errors are handled in the delete table lambda
[ ] Test the lambda


Switch default branch back to Python 2

I recently switched this image to Python 3 by default... and while 95% of the stuff I do in Ansible works great on Python 3 (and 100% of it on Ubuntu or CentOS 8 at this point), apparently the yum module does not support Python 3 at all.
So builds of certain projects started failing :(
See, for example: https://github.com/ansible/ansible/issues/67699

Fixed via https://github.com/geerlingguy/docker-centos7-ansible/commit/8ce1ac250a1c5815d0ca0526e4830824f0cf9fc0
Sigh.


Real-time best bid/ask data

I've created the following script to get the best bid/ask data in real-time:
import * as GTT from 'gdax-trading-toolkit';
import { GDAX_WS_FEED, GDAXFeed, GDAXFeedConfig } from 'gdax-trading-toolkit/build/src/exchanges';
import { GDAX_API_URL } from 'gdax-trading-toolkit/build/src/exchanges/gdax/GDAXExchangeAPI';
import { TickerMessage } from 'gdax-trading-toolkit/build/src/core';

const product = 'LTC-EUR';
const logger = GTT.utils.ConsoleLoggerFactory();

const options: GDAXFeedConfig = {
    logger: logger,
    auth: { key: null, secret: null, passphrase: null }, // use public feed
    channels: ['ticker'],
    wsUrl: GDAX_WS_FEED,
    apiUrl: GDAX_API_URL
};

GTT.Factories.GDAX.getSubscribedFeeds(options, [product]).then((feed: GDAXFeed) => {
    GTT.Core.createTickerTrigger(feed, product)
        .setAction((ticker: TickerMessage) => {
            console.log(`Best Bid: ${ticker.bid}`);
            console.log(`Best Ask: ${ticker.ask}`);
        });
});

I was expecting to get the best bid/ask data instantly, as soon as there's a new best bid/ask but that's not happening:

Any idea on what could be the issue here?

I think this is happening because the ticker channel only provides real-time price updates when a match/trade/tick happens and not when there's a new available best bid/ask. Correct me if I'm wrong.
The REST API loadTicker method seems to retrieve the latest best bid/ask, which is what I want, but still is there anyway to achieve this using the websocket feed?

I think you need a LiveOrderbook to get real-time updates on the spread. Once you have a LiveOrderbook you can access the spread inside the LiveOrderbook.update event:
const bestBid = liveOrderbook.book.highestBid.price;
const bestAsk = liveOrderbook.book.lowestAsk.price;

Yup, @nunoarruda and @zhm are correct. The GDAX WS feed only sends ticker updates when a trade is made.
The LiveOrderbook is the way to go if you want spread updates in-between ticks.

Thanks @zhm and @CjS77, that was helpful. I've managed to code exactly what I wanted:
import * as GTT from 'gdax-trading-toolkit';
import { GDAXFeed } from 'gdax-trading-toolkit/build/src/exchanges';
import { LiveBookConfig, LiveOrderbook, LevelMessage } from 'gdax-trading-toolkit/build/src/core';

const product = 'LTC-EUR';
const logger = GTT.utils.ConsoleLoggerFactory({ level: 'debug' });
const spread = {
    bestBid: '',
    bestAsk: ''
};

GTT.Factories.GDAX.FeedFactory(logger, [product]).then((feed: GDAXFeed) => {
    const config: LiveBookConfig = {
        product: product,
        logger: logger
    };

    const liveorderbook = new LiveOrderbook(config);

    liveorderbook.on('LiveOrderbook.update', (msg: LevelMessage) => {
        const highestBid = liveorderbook.book.highestBid.price.toFixed(2);
        const lowestAsk = liveorderbook.book.lowestAsk.price.toFixed(2);

        if (highestBid.valueOf() !== spread.bestBid.valueOf()) {
            spread.bestBid = highestBid;
            console.log(`New best bid: ${spread.bestBid}`);
        }

        if (lowestAsk.valueOf() !== spread.bestAsk.valueOf()) {
            spread.bestAsk = lowestAsk;
            console.log(`New best ask: ${spread.bestAsk}`);
        }
    });

    feed.pipe(liveorderbook);
});

@nunoarruda Glad to help.
If you think spread is a useful metric, how about submitting a PR to add it to the LiveOrderbook class?

@CjS77 not sure what functionality I would add.


[BUG] - Input required does not clear when using controlled component

NextUI Version
2.3.6
Describe the bug
I'm using a controlled Input component with isRequired and I have functionality to reset the form back to some previous state. If a user clears out the input field, triggers the validation by clicking away from the input (validation will show that the field is required), then clicks a button to reset the input value, the required validation continues to display even though a valid value is set in the input. Additionally, focusing the input (and un-focusing) does not cause the validation to clear either.
Example sandbox here: https://codesandbox.io/p/devbox/exciting-margulis-6y6h37
Your Example Website or App
https://codesandbox.io/p/devbox/exciting-margulis-6y6h37
Steps to Reproduce the Bug or Issue

Clear the input
Click off input to trigger validation
Click Update button
Notice email field is populated but validation still shows
Only changing input then change focus causes validation to clear

Expected behavior
After resetting the value I would expect the validation error to clear.
Screenshots or Videos
No response
Operating System Version
Windows
Browser
Chrome

@wingkwong can i work on this one. You have assigned it to yourself that's why i am asking?

@abhisektomar1 Sure. I'm a fully occupied these days. Feel free to ping me if you have any questions. Thanks.

I don't reproduce this issue.
code and dev mode are different.

@jijiseong Thanks for checking. in fact we've changed the validation behaviour in v2.4.0 so this issue should be no longer reproducible in latest version.


net/arp: optimize the arp command display result

Summary
netlib_getarptab.c: improved the display without any arp entry
nsh_netcmds.c: handle the ENETUNREACH error number
Impact
Testing
sim:local with KASAN

[Experimental Bot, please feedback here]
No, this PR does not fully meet the NuttX requirements.
Missing Information:

Summary:

A more detailed explanation of why these changes are necessary is needed. Are these bug fixes? Feature enhancements?
What is the exact functionality of netlib_getarptab.c and nsh_netcmds.c?
Provide specific examples of how the display is improved and how the ENETUNREACH error is handled.
Include links to related NuttX Issues or Pull Requests if applicable.

Impact:

Address all impact points. Even if there is no impact, state "NO" explicitly for each point.
For example:

Impact on user: NO
Impact on build: NO
...

Testing:

Build Host(s):  Specify the operating system, CPU architecture, and compiler used for building.
Before/After Logs: The provided logs are placeholders. Include actual testing logs demonstrating the issue before the change and the improvement after the change.

Recommendations:

Expand the Summary: Provide a clear and concise explanation of the changes, their purpose, and their effects.
Address all Impact Points: Even if there's no impact, state it clearly for each item.
Provide Detailed Testing Information: Include specific details about your build environment and provide real testing logs showcasing the changes in action.


Brackets and Dark Mode

Feedback from someone who used VZCode in VizHub to give a live coding workshop:

went well!  people loved it. some fun technical issues not only to do with Vizhub. the room was so bright i had to switch to light mode. but that made the brackets disappear too? anyway such a minor thing.

I think this may be to do with the rainbow brackets.
I feel like @michaelhelper you may have mentioned this earlier but I don't recall.
The first thing to do is to test the light mode / dark mode, then see if we can get the syntax highlighting to work well in both modes, including rainbow brackets.
One potential short term "fix" would be to disable rainbow brackets until we can get this resolved. Could do by putting the entire rainbow brackets feature behind a feature flag.

TODO drop screenshots of light mode vs. dark mode