Add yarn lockfiles to integration tests to speed up tests.

Summary
No reason not to use yarn properly for these. cc @dmitriiabramov
Test plan
jest command should be faster on the Jest repo. It saves about 13 seconds.

i feel like that might introduce some issues in future :(

It's no different from any other dependencies in the project and we manage them all with Yarn. The only other solution I recommend is to hoist the deps to the top level like we do for examples and not run yarn at all within the integration test folders. What do you think?

is there a way to setup recurring task of updating dependencies? we can do it for packages, examples and integration tests together

(Dmitrii said it was fine to merge this)

Codecov Report

Merging #3397 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #3397   +/-   ##
=======================================
  Coverage   64.27%   64.27%
=======================================
  Files         176      176
  Lines        6538     6538
  Branches        4        4
=======================================
  Hits         4202     4202
  Misses       2335     2335
  Partials        1        1

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e9b8d00...dc8b1af. Read the comment docs.


Change from cocoon to cocooned, which is maintained

Unfortunately this doesn't work yet, but I have no idea why. It's a mystery. So, I'm saving it for later.

Working now!


backend relay housekeeping

https://github.com/bithyve/internal-docs/issues/149

Cleaned up Dev but I don't have access to Staging. Also, I'm not sure whether we need to backup dev/staging clusters.

No need to backup dev and staging.

Closing this issue as discussed with dev.


Disable social password reset

Is there any way to accomplish this easily other than modifying the passwordreset view?
Any setting perhaps on either the allauth or dj-rest-auth side?
Otherwise I think this would be a great enhancement as resets make no sense for social accounts, right?

Actually I noticed that no email is being sent, however the JSON returned indicates it was successful.

This is my solution, would appreciate feedback :)
class CustomPasswordResetSerializer(serializers.Serializer):
    email = serializers.EmailField()
    password_reset_form_class = PasswordResetForm

    def validate_email(self, value):
        self.reset_form = self.password_reset_form_class(data=self.initial_data)
        if not self.reset_form.is_valid():
            raise serializers.ValidationError(_('Error'))

        ###### FILTER YOUR USER MODEL ######
        if not User.objects.filter(email=value).exists():

            raise serializers.ValidationError(_('Invalid e-mail address'))

        random_user = User.objects.get(email=value)
        if SocialAccount.objects.filter(user=random_user).exists():
            raise serializers.ValidationError(_('User has a social account'))

        return value


"nanoc check" exception when some issues do not have a subject

If a "nanoc check" results in multiple issues, and some issues have a subject and others do not, then the command will fail with a "ArgumentError: comparison of String with nil failed".
The add_issue(desc,subject:nil)method takes and optional subject that will become nil if not filled out. When the subject is used on some issues but not others an exception can occur because the command runner tries to sort the issues by subject and will fail when trying to compare nil with a string.
A simple solution would be to make issues without a subject default to an empty string, or to a string such as "(no subject)" here. Alternatively, the sort method in the runner can be changed to account for this.

Fixed in #811.


possible issue with data splitting in all notebook examples

Hi, I have a question regarding how data splitting has been done in the example notebooks. It is pretty obvious so I don't know whether it is a problem or is intentional. Take the galaxies.ipynb as example (I think this snippet is reused everywhere though):
# Run prediction-powered inference and classical inference for many values of n
results = []
for i in tqdm(range(ns.shape[0])):
    for j in range(num_trials):
        # Prediction-Powered Inference
        n = ns[i]
        rand_idx = np.random.permutation(n_total)
        _Yhat = Yhat_total[rand_idx[:n]]
        _Y = Y_total[rand_idx[:n]]
        #  _Yhat_unlabeled = Yhat_total[rand_idx[n:]] --> possible correction
        _Yhat_unlabeled = Yhat_total[n:] # --> this is not using the permuted indices?

        ppi_ci = ppi_mean_ci(_Y, _Yhat, _Yhat_unlabeled, alpha=alpha)

It seems that the _Yhat_unlabeled vector is not constructed through the randomly permuted indices for each trial, while the gold standard _Yhat does. Essentially no matter how many repetitions are made, the former vector stays the same.

You're totally right.
My guess as to why this didn't cause an error is that $N$ is very large, so
Yhat_total[n:].mean() $\approx$ Yhat_total[rand_idx[n:]].mean()
This is a great find. Would you be willing to open a PR that

Changes this in all notebooks
Re-runs all notebooks, from top to bottom, with the new code

Thank you so much for your help @listar2000 !

Thanks for the quick response @aangelopoulos  and I'm very willing to do that. Will send a PR soon.

Addressed by #19 ; Thank you @listar2000 !


Shadows not displaying for certain GLTF models

Describe the bug
When rendering a model (in tandem with AR core for surface detection and positioning), certain GLTF models
(loaded with the AssetLoader class from gltfio) doesn't cast a shadow on the planes that are rendered.
Here are the different models used:

Non-shadow casting model:
trex.zip

Shadow casting model:
scooter.zip

To Reproduce
Steps to reproduce the behavior:

Load the gltf asset using AssetLoader.
Render a flat plane below the asset that will be loaded (set receive shadows to true)
Set directional lighting and cast shadows to true.
Place the gltf model on top of the plane
Assert that the scooter model renders a shadow but the trex model doesnt.

Expected behavior
Both models should be rendering shadows
Screenshots

Smartphone (please complete the following information):

Device: Pixel 4XL
OS: Android 12

Additional context

The t-rex model itself is receiving shadows from the direct lighting, it just doesn't seem to want to cast shadows.

Is the directional light oriented the same way in both cases?

Is the directional light oriented the same way in both cases?

Yes, they both use the same parameters for the directional light:
intensity = 200_000f
dir = (0.0f, -5f, 0.0f)

Could be related to issue #5088

I have reverted #4939, which doesn't fix the problem -- so my guess was wrong.

I had a look at the recent changes. I was wondering if the issue could be in getters.vs :
mulBoneNormal(vec3 n, uint i) seems to be using bonesUniforms.bones[i].transform instead of cof for recomputing cof[2].z but I'm not sure ?

I have reverted everything related to #4999 and that fixes the lighting issues (not sure about the shadows). I'm not sure where things went wrong yet.

So maybe that could be a hint (or not) : I tried 1.17.0, and it looked like my tube test mesh wasn't using the normal map it was supposed to use. It seemed to use a normal map from another object in the scene whereas I'm pretty sure I used a default normal map.
This is getting weirder, now I'm wondering if the lighting problem I was seeing was also it using another (less obvious) map, like a white or black one.
(I also have some warnings in the log but they were here before : [entity=1765, primitive @ 0] missing required attributes<PHONE_NUMBER>00001F), declared=000000000000000F; maybe I should fix those first).

@daemyung I investigated this a bit and it seems like skinned normals are broken since #4999 (I tried to revert everything but that change). Note that this is skinning, not morphing that is broken. It is strange because #4999 doesn't seem to change the code path for skinning.
I'm not sure yet what's going on with shadows, but normals are definitely broken on the dinosaure model above.
It would be great if you could have a look, as it's getting late here. Thank you.

@pixelflinger Thanks I will look into.

@daemyung thank you!
More data points:
@teub, I think the shadowing issue might be a problem with the asset, its bounding box is completely wrong, and shadowing is using it to compute the light frustum.
@daemyung, the issue with the normals happens only in OpenGLES, it works fine in both Metal and Vulkan.

@pixelflinger Do you have some tips to render bawanglong is too big.

@pixelflinger Do you have some tips to render bawanglong well? It is too big.

Yes, use the -s option, it makes things better.

@daemyung it is definitely a problem with the 2nd SamplerInterfaceBlock.
If I comment out Material.cpp:385:
//        addSamplerGroup(pb, BindingPoints::PER_RENDERABLE_MORPHING,
//                SibGenerator::getPerRenderPrimitiveMorphingSib(variant), mSamplerBindings);

The this model works.
This must be an existing bug that we never ran into before.

@pixelflinger Hm.. strange... I can't see this issue. Are you testing on Android?

No actually, I'm testing on a MacMini M1.
If you don't see the problem on Android, then it totally sounds like an opengl binding issue, because Marcos still uses GL4.1.
Thank you for investigating, I think I can fix this tomorrow. It's definitely an existing GL (not ES) bug.

@pixelflinger Upper screenshot captured on MacbookPro Intel. I can't reproduce this issue.
FEngine resolved backend: OpenGL
[ATI Technologies Inc.], [AMD Radeon Pro 5500M OpenGL Engine], [4.1 ATI-4.7.101], [4.10]
Active workarounds:
GL_MAX_DRAW_BUFFERS = 8
GL_MAX_RENDERBUFFER_SIZE = 16384
GL_MAX_SAMPLES = 8
GL_MAX_TEXTURE_MAX_ANISOTROPY_EXT = 16.000000
GL_MAX_UNIFORM_BLOCK_SIZE = 65536
GL_UNIFORM_BUFFER_OFFSET_ALIGNMENT = 256
OS version: 0

I just tested on an Intel MacBook Pro and I definitely see the lighting/normal problem:

I figured out the lighting problem, it was indeed an existing bug in OpenGLProgram, I'll fix it tomorrow.
The bug would happen when a SamplerInterfaceBlock was defined on a program but not used/set.
Thank you all for all the help!

@teub I think the shadow problem is because the bounding box of this model is wrong. Can you try after adjusting the bounding box?

I think the shadow problem is because the bounding box of this model is wrong. Can you try after adjusting the bounding box?

Is it possible to post this scaled down model?

@daemyung Is it possible to post this scaled down model?

No problem. Test scaled.glb.zip.


angular_analyzer_plugin doesn't working correctly - NoSuchMethodError: The method 'send' was called on null.

Dart SDK Version (dart --version)
2.7.0
AngularDart Version (i.e. 4.0.0, 5.0.0-alpha+5, MASTER)
angular: ^6.0.0-alpha+1
Whether you are using Windows, macOS, or Linux (if applicable)
Windows 10
Whether you are using Chrome, Safari, Firefox, Edge (if applicable)
Chrome
Any other packages or constraints we should know about
Intellij IDEA 2019.3.3

angular_analyzer_plugin doesn't working correctly.
According to Troubleshooting, I did the following test:

If you run bin/plugin.dart from .plugin_manager/HASH/analyzer_plugin, do you get any import errors? (Note: this is expected to crash when launched in this way, but without import-related errors)

And it shown this error:

C:\Users\samue\AppData\Local.dartServer.plugin_manager\7b2b624ba25bf337c9e3e051bc8ef6c2\analyzer_plugin\bin>dart plugin.dart
Unhandled exception:
NoSuchMethodError: The method 'send' was called on null.
Receiver: null
Tried calling: send(SendPort)
#0      Object.noSuchMethod (dart:core-patch/object_patch.dart:53:5)
#1      new PluginIsolateChannel (package:analyzer_plugin/src/channel/isolate_channel.dart:109:15)
#2      Driver.start (package:analyzer_plugin/src/driver.dart:30:40)
#3      start (package:angular_analyzer_plugin/starter.dart:9:8)
#4      main (file:///C:/Users/samue/AppData/Local/.dartServer/.plugin_manager/7b2b624ba25bf337c9e3e051bc8ef6c2/analyzer_plugin/bin/plugin.dart:7:3)
#5      _startIsolate. (dart:isolate-patch/isolate_patch.dart:303:32)
#6      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:174:12)

The angular_analyzer is working again after last updates in Dart plugin/IntelliJ and packages.


Fix path to hash_req.py.

fixes #1081
@timabbott please review.

Automated message from Dropbox CLA bot
@umairwaheed, it looks like you've already signed the Dropbox CLA. Thanks!

Nice find! I thought provision.py was only supposed to be run from Zulip's root directory. That's why I used a path relative to that. But on looking closely I saw that line 214 suggests otherwise.

Merged, thanks @umairwaheed!  As a sidenote, in general, it's worth making all of our scripts work from any directory, since it's usually not hard, and it's a lot nicer to use scripts that aren't fragile that way :)


Mesos Agents are not reusable

When a Mesos Agent is auto-created and runs a Jenkins Job, it is immediately placed into a suspended state after the job completes. This leads to hundreds of agents being created in a busy build environment and build delays waiting for new agents to be created by Mesos.
I'm using Mesos 1.9.0, but the problem seems to be a hard-coded configuration variable in the Jenkins Framework plugin i.e.
org.jenkinsci.plugins.mesos.MesosAgentSpecTemplate.reusable = false
When this is set to true, the agents work as expected, being reused when new matching jobs are submitted and expiring after an idle timeout.
The parameter is commented with a Jira tag: DCOS_OSS-5048
But this Jira has now been resolved.
I'm not sure whether it can just be enabled (it works for Mesos 1.9.0, but this may not be true for other versions)
Ideally, I guess this should be exposed as a configurable variable within Jenkins Cloud config.

I'm testing a fix for this just now. If all goes well, I'll push it in the next few days.


make easier to customize logo sizes

tweaking logo height/width is probably necessary in most collections, currently surprisingly fiddly to do.

[x] add id to home banner logo img
[x] add id to footer logo img
[ ] add method to quickly tweak pixel dimensions (is this in _custom.scss or theme.yml or next to the logo image src in config.yml??)

I think it might be worth changing _config.yml options to have:
organization-logo-home
organization-logo-banner
organization-logo-nav
organization-logo-footer
it is a lot, but would make make simpler to customize, since it is a bit ambiguous now (i.e. -banner currently is on non-home page banner, -nav is in footer, homepage, and mobile nav--> but these decisions don't necessarily work if you change the theme at all, and if your logo is differently shaped they aren't all interchangeable in that way)

actually, maybe remove the "organization" part, just "logo-", since we don't need to assume it is an org (thinking of context of CB-GH, where currently we have a different banner and footer without org logo and links). It could just be a collection logo or not used at all, too.

current bits of CSS notes:
// in _base.scss
#mobile-nav-image {
    max-height: 1.5em;
}

#footer-logo {
    height: 40px;
}

// in collection-banner.html
#home-banner-logo {
    height: 40px;
}

stuff in collection-banner.html added to be able to use Liquid to set values without complicated set up to pass values through the main scss.


Fix another find related bug

Start with sample data
hide patients n/fdsfr
find n/david
find r/covid
all data disappears

Codecov Report

Merging #189 (0ca480e) into master (40da736) will decrease coverage by 0.02%.
The diff coverage is 75.00%.

@@             Coverage Diff              @@
##             master     #189      +/-   ##
============================================
- Coverage     70.58%   70.55%   -0.03%
+ Complexity      791      790       -1
============================================
  Files           111      111
  Lines          2682     2680       -2
  Branches        351      350       -1
============================================
- Hits           1893     1891       -2
  Misses          678      678
  Partials        111      111

Impacted Files
Coverage Δ

.../address/logic/commands/UnhidePatientsCommand.java
0.00% <0.00%> (ø)

...java/seedu/address/logic/commands/FindCommand.java
96.87% <100.00%> (-0.19%)
:arrow_down:

...erson/predicates/CombinedAppointmentPredicate.java
91.30% <100.00%> (ø)

...del/person/predicates/CombinedPersonPredicate.java
100.00% <100.00%> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


tf-nightly2.4  run error

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
OS Platform and Distribution (e.g., Linux Ubuntu 16.04):Ubuntu18.04
TensorFlow version (use command below):tf-nightly2.4
Python version:3.7.4
CUDA/cuDNN version:CUDA11.1
GPU model and memory:RTX3090

Describe the current behavior
In my current environment, I use tf-nightly2.4 to run the bert model. After I run an epoch, I reported an error.
value error: layer model expects 3 input(s), but it received 4 input tensors.
It seems that the output layer of the model is also regarded as the input layer.
But there is no problem with my code, it can run completely on CUDA10.1, so what is going on?

I think the tf-nightly2.4 can't support RTX 3090 very well. we need to wait new version. I have the same question about that.

@wwww666
Please refer to this comment and let us know, also[lease refer to this issue with respect to the configuration mentioned :#43718 #43588 #44251


Dockerfile: increase choom-wrap-packages adjustment to 300

When this starts to have an effect we should begin to see 500s resulting from subprocess calls being killed (perhaps with exit code 137).

Oh yes, I guess when I say 500s I mean unhandled exceptions in general.
Come to think.. I haven't noticed any sentry errors from celery workers in general - do we actually have it set up properly to catch them?


Add ANU hub config

This adds the config for the ANU hub. Reference https://github.com/2i2c-org/infrastructure/issues/991

I've just deployed this manually :rocket: I was able to login, checked out the admin list which seemed ok and also started RStudio successfully (haven't run anything there though).
So, I'll go ahead and merge this now so that it can be tested more thoroughly!

@GeorgianaElena I pushed a commit to make the comment a little bit more explicit - but otherwise this is great! Let me know what you think!

That's perfect! Thanks @sgibson91! I just propagated your suggestion to all of the comments


Revert "Added axisOrientation property to Scrollbar"

Reverts flutter/flutter#75497
Causing analysis failures at top-of-tree.

LGTM - ok to land on red.

/cc @chinmoy12c

The pre-submit results on https://github.com/flutter/flutter/pull/75497 were 4 days old when this landed AFAICT. It's possible we need a better system for invalidating stale presubmit results.


[Rails5.2] Fix MoveAwxCredentialsToAuthentications

Fixes spec failure do to a change in Rails that wraps all migrations errors in StandardError.
We still try to check for the nested error by reading the msg (via the Regexp in the second argument)

This has been cherry-picked into https://github.com/ManageIQ/manageiq-schema/pull/472 so closing.


Refactor ToggleButtons

ToggleButtons use RawMaterialButton which is scheduled to be deprecated and removed (eventually).
We need to refactor them to use the new button universe, either TextButton, OutlinedButton or ElevatedButton.
cc @TahaTesser and @HansMuller
Completing this will likely resolve https://github.com/flutter/flutter/issues/97302 as well.

Assigning to @TahaTesser per our chat today

Re-opening as the change is being reverted


Paniukhin HW07

You have a bunch of really good properties, but could you add let-expressions to the generator and the interpreter?

Added Let-expressions to evalDirect and prop_directEvaluationConsistency test

Ok.


How to evaluate the results on test set with ground truths

Hi,
can someone guide me how can we evaluate the model on test set using the predictions and ground truths.
Thanks,
Amardeep

Hi @amardeepjaiman,
Actually in this project we do not evaluate the model regarding the test set (its done only with training and validation sets during training process). However if you get a ground-truth version of an image and a semantic segmentation prediction (let say, as two .png files), you can substitute pixel triplets by corresponding label IDs (see some functions in generator.py that deal with this process) and compute metrics of your choice (ex accuracy, IoU) by comparing both structures in a a posteriori logic.
Contributions are welcome ! :)

As for #136, there is no news here, I close this issue.


Bypass alignment related copy whenever possible

The memory returned by msg->getData() is most likely already be aligned. add class AlignedArray to bypass alignment related copy whenever possible.

Hmm this is a nice cleanup.
Instead of a class, can it be a function that returns a kj::heapArraycapnp::word?

In fact, I'm reading this pattern. Could it return a "cereal::Event::Reader" and that would save even more lines?

we still need a class to free allocated memory in destructor.currently, I can't figure out how to return cereal::Event::Reader from a single function.

Hmm, then maybe a class is okay, and we put the cereal::Event::Reader functionality into it.

how about this?exactly the same lines - what a coincidence lol.

The FlatArrayMessageReader is the owner of the message content,and only can be initialized from constructor.
if  we put cereal::Event::Reader into class, we have to create FlatArrayMessageReader in heap that may make things more complicated.
I've modified the previous code to reduce lines.

still not good.need to figure out a better pattern.


cmd/pprof: add HTTPS support with client certificates

Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (go version)?
go version go1.8.3 darwin/amd64
What operating system and processor architecture are you using (go env)?
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/smo/go"
GORACE=""
GOROOT="/usr/local/Cellar/go/1.8.3/libexec"
GOTOOLDIR="/usr/local/Cellar/go/1.8.3/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/dh/gykpvks552v4h7yfms86x3r80000gn/T/go-build191348200=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"
PKG_CONFIG="pkg-config"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
What did you do?
Tried to connect to a HTTPS service in order to pull out pprof data
What did you expect to see?
The pprof data
What did you see instead?
http fetch https://<IP_ADDRESS>:32473/debug/pprof/profile?seconds=5: Get https://<IP_ADDRESS>:32473/debug/pprof/profile?seconds=5: remote error: tls: bad certificate
I think it would sort of nice to be able to connect via pprof to a server that runs SSL/TLS with client certificates. There
is a work-around this of cause, just put up and proxy that does SSL termination, but that might not always be feasible.
I assume the command would look something like.
go tool pprof -seconds 5 -caert cert.pem -key key.pem -ca.cert ca.pem https://<IP_ADDRESS>:32473/debug/pprof/profile

Why not download the profile using a program that likely has these features already, like wget or curl, then use the file in go tool pprof instead?

It is already supported:
$ go tool pprof -seconds 5 https+insecure://<IP_ADDRESS>:32473/debug/pprof/profile

@OneOfOne do you know if this is documented? If not, could we document this and then we can close the issue, as you've already provided the fix.

@OneOfOne not sure that works if you have mutual TLS enabled. But @mvdan had a point, I can download it using curl and then load it with proof.
We can mark this is resolved IMO, just me not thinking 🙄

It's documented in the go1.8 release notes but I think it should be added to the tool's documentation.
https://tip.golang.org/doc/go1.8

The pprof tool can now profile TLS servers and skip certificate validation by using the “https+insecure” URL scheme.

@OneOfOne wanna send a documentation CL? /cc @bradfitz

Keep in mind that pprof is vendored from an upstream repo, so you can't change some parts of it in Go's repo directly. If you did, it'd just get deleted the next time we imported from the upstream pprof repo.
But I'm not sure which docs you're proposing editing.
https://golang.org/cmd/pprof/ doesn't have much as-is.

Ahh I see, thanks for the clarification @bradfitz. I've godoc'd https://github.com/google/pprof and also grep'd for insecure and found that 'https+insecure' is in an internal package for the default fetcher so I no longer believe that documenting this trivial.

Hi,
I am trying to profile swarm using "go tool pprof"
Below are error messages with different options that I have tried:
# go tool pprof http://<IP_ADDRESS>:2376/debug/pprof/trace
Fetching profile over HTTP from http://<IP_ADDRESS>:2376/debug/pprof/trace
http://<IP_ADDRESS>:2376/debug/pprof/trace: Get http://<IP_ADDRESS>:2376/debug/pprof/trace: net/http: HTTP/1.x transport connection broken: malformed HTTP response "\x15\x03\x01\x00\x02\x02"
failed to fetch any profiles

# go tool pprof https+insecure://<IP_ADDRESS>:2376/debug/pprof/trace
Fetching profile over HTTP from https+insecure://<IP_ADDRESS>:2376/debug/pprof/trace
https+insecure://<IP_ADDRESS>:2376/debug/pprof/trace: Get https://<IP_ADDRESS>:2376/debug/pprof/trace: remote error: tls: bad certificate
failed to fetch any profiles

Below are the config details:
# go version
go version go1.9 linux/amd64

# /var/vcap/packages/swarm/bin/swarm --version
swarm version 1.2.8 (HEAD)

Swarm is running with debug option enabled, however, the curl ouput at "info" endpoint has the debug set to false:
./bin/swarm --debug=true --log-level=info manage --cluster-driver=swarm --host unix:///var/vcap/sys/run/swarm_manager/swarm_manager.sock --engine-failure-retry 10 --host=<IP_ADDRESS>:2376 --strategy=spread --filter=health --filter=port --filter=dependency --filter=affinity --filter=constraint --replication=false --replication-ttl=30s --advertise=xx.xx.xx.xx2376 --heartbeat=20s --api-enable-cors=false --cluster-opt=swarm.overcommit=0.5 --tls=true --tlscacert=/var/vcap/jobs/swarm_manager/config/docker.cacert --tlscert=/var/vcap/jobs/swarm_manager/config/docker.cert --tlskey=/var/vcap/jobs/swarm_manager/config/docker.key --tlsverify=true nodes://yy.yy.yy.yy:4243
#curl -k --key /var/vcap/jobs/swarm_manager/config/docker.key --cert /var/vcap/jobs/swarm_manager/config/docker.cert https://<IP_ADDRESS>:2376/info
The output of above cmd has "Debug":false in its output.
Is there anything that I missing here?

I've pushed a PR to pprof to add this functionality. https://github.com/google/pprof/pull/261

This needs to be fixed upstream, so closing the Go issue.


Authorization For Each Link

Adding a new way to check if a link can be shown to the user or not using the native function ->can().

Is this repo still active?

Sorry, I have been busy. Great idea adding permissions!


Display price with $ sign in the UI

Right night it displays price under other details as a number like 4.00.
To make it more obvious we're talking about price here, add a $ sign in front.
Future feature could be to allow chaning the $ sign into their local currency, once a userconfig system has been added (user accounts/settings).

Pull Request Test Coverage Report for Build 73

2 of 2 (100.0%) changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.004%) to 92.321%

Totals

Change from base Build 72:
0.004%

Covered Lines:
1587

Relevant Lines:
1719

💛  - Coveralls


FileNotFoundError: [Errno 2] No such file or directory:  'C:\[...].jpg'

Hey! I'm writing a game engine that needs to be capable of loading obj and mtl files.
However, the wavefront/obj format isn't completely standardized, so different programs handle
things in different ways. I found that often mtl files will use absolute paths that the original
creator used while creating the mesh, rather than relative paths. The problem with this is that
the end user may not have the full original path the creator used, but they should have a relative
path containing the exact filename. I propose that this library could allow users to create
some sort of callback so they can handle paths used in files themselves and return the actual
path that should be returned. So I could have a function that says "def handleFilePath(path, type): return newPath" and I also think it would be good to have an option in the load function to just do a subdir search.
To clarify, here was the problem I faced:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\DocumentsandSettings\AGH\MyDocuments\3dsMax\sceneassets\images\FlammableBarrelTexture.jpg'
And here is the relavent mtl "code":
map_Ka C:\Documents and Settings\AGH\My Documents\3dsMax\sceneassets\images\Flammable Barrel Texture.jpg
map_Kd C:\Documents and Settings\AGH\My Documents\3dsMax\sceneassets\images\Flammable Barrel Texture.jpg

This is clearly not the proper filepath, since I don't have that filepath, but the problem is
that in certain programs like 3D Viewer, it's still textured, which is why this should be solved
in the code and not in the file.

I suggest adding a find method in Texture: https://github.com/pywavefront/PyWavefront/blob/master/pywavefront/texture.py
It would check if path actually exists and fall back to using glob in pathlib to do a quick subdirectory scan.

I agree completely, but I also think it may be important to allow developers to handle this in there own way as well.

Agreed. It could just be there as an optional tool. Right now the library just present the user with the raw path and name of the texture and are left to their own devices anyway regarding how this information should be used.

PR: https://github.com/pywavefront/PyWavefront/pull/102


Pytivo addition

This is really old software... Does it still actually work? Personally, I'm somewhat uncomfortable with unmaintained software and docker images...
Don't forget to run "ansible-lint nas.yml" on your branch before submitting the PR - you've got trailing whitespace issues.

Thanks for the wonderful work on ansible-nas, much appreciated. My additions don't seem to be of interest to the project so I'll just use my fork and not bother you.

@bcurran3 You can't even answer a question about whether the software still works? Enjoy your rage quit I guess.

I will enjoy my "rage quit."
I've been trying to be active in this community for months. Except for the Samba additions I've received nothing but pushback on EVERYTHING. There's obviously no value to my suggestions. I've completed 4 of my own enhancement requests and 2 other people's. I want to contribute, but I'm beating my head against the wall going nowhere.
"Does it still actually work?" Is a completely ignorant question as #1 see https://github.com/davestephens/ansible-nas/issues/149 #2 why the hell would I want to submit it to the rest of the community if it doesn't work? I'm happily using it. I'm probably only 1/640 that will use it currently. I'm probably one of ten who might ever use it.
You want to talk CODE_OF_CONDUCT.md? How about a single modicum of respect for volunteers such as me trying to contribute to the project? I'm not seeing it. Some serious management of the project to consider: #1 encourage people who want to contribute #2 respect volunteers time and effort #3 when they say their going to take on an issue, assign it to them and mark it in progress. This creates a feeling of ownership and responsibility which turns into dedication. #4 READ what they say and attempt to understand it instead of skimming it, misunderstanding it, and giving knee jerk defensive responses that aren't necessary. #5 Try to promote positivism instead of negativity.
So yes.
I got what I needed out of ansible-nas. I've modified it to suit my needs. I'm good with what I've got and I have the capability to continue adding on to any future wants and needs. ansible-nas is cookie cutter cut and paste, easy to understand in an hour or two. I wanted to contribute back and make it better. Thank you, really, for giving me an excellent starting base. You've done a good job and I see it getting better.

I think "does software abandoned by the dev in 2013 with no updates for over 2 years still work" is a very good question to ask. You yourself admit that pretty much nobody else will want to use it too. I've tried to help you multiple times and either been ignored or rebuffed. You seem to expect everybody to have an idetic memory for anything you've ever posted and have rejected or ignored all the help and advice provided.  Enjoy your fork, I hope it does everything you need it to.

I think "does software abandoned by the dev in 2013 with no updates for over 2 years still work" is a very good question to ask. You yourself admit that pretty much nobody else will want to use it too. I've tried to help you multiple times and either been ignored or rebuffed. You seem to expect everybody to have an idetic memory for anything you've ever posted and have rejected or ignored all the help and advice provided.  Enjoy your fork, I hope it does everything you need it to.

1> I call BULLSHIT. You offered help. When I tried to take you up on your offer, you dismissed it. BTW: All I needed to know was click on the master branch button on GH and enter in a name for a new branch before entering a commit. It was that simple of a hurdle to overcome.
2> "this project will be better without you" Now who's the asshole? I'm passionate in what I do and I'm a guy who, by nature, likes to help others and fulfill their requests.  You have no clue of what I might have done.  Refer back to my management suggestions. You've obviously are not management material and should be labeled "does not play well with others." If you had the slightest potential for decency you would have simply said "Sorry, let's reboot and start this again and have a productive relationship instead of adversarial one." Nope, guess it's an alien concept to you. For the record I've not rejected, ignored, or rebuffed any advice. You too must have a reading compression issue as there is no way you can find proof of what you say, only the contrary.

@davestephens - sorry to be toxic. Not being disrespectful to the project, just doing an "exit interview" and trying to provide constructive criticism before I was attacked.

A few words, if I may.

Name-calling, personal attacks and bad language are not acceptable, and will not be tolerated on this project, end of. We're on GitHub here folks, a public platform that the world has access to. This is not Fidonet in 1996.

There are contribution guidelines and a Code of Conduct. They are expected to be followed, by everyone.

Ansible-NAS (and other open source projects) are not the place to be spoon-fed every piece of knowledge you don't yet have. Doing your own research is very much expected of contributors. People will help get you up to speed, but they're not going to do your work for you. This is not the first time I've highlighted this to you @bcurran3 (first time in October 2019) so I'm slightly disappointed we seem to be having this same conversation again three months later.

If someone offers help to get you from a to b, accept it.

On this note, you've been offered assistance by me personally multiple times, and @animeai probably equally as many, if not more times. I don't know why you haven't taken up these offers.

Patience when dealing with open source projects is very much something you need. As I said to you yesterday, I run Ansible-NAS in my free time, because I enjoy it. I don't get paid (running Ansible-NAS has made me < £100 over the years), it is not a business, and I have a real life away from the computer.

As I've said previously, I'm truly grateful for yours (and everyone else's) contributions but you need to put the effort in getting up to speed with the tooling required to contribute. Me rejecting your PRs because you've not done this is not "pushback", it's calling "running a project with standards". If I allowed everything to be merged that was raised in a PR with absolutely no standards or quality being upheld, then the project wouldn't be of the same quality that it is now. The project doesn't owe you anything, and it's not a piece of software you pay for, so effort absolutely is required on your part too.
I'd still be happy for you to contribute to the project, but you need to:

listen to advice you've been given multiple times over the last few months
be nice to people
put effort into learning the tooling
realise that "your way" is not necessarily "the right way", and that people here that will disagree with you from time to time.

If this doesn't sit well with you, then I would suggest that Ansible-NAS is maybe not the project for you. I wish you well in your future endeavours should you choose to move on.


Improve error message for exc StorageNoResponseLeft

https://github.com/packit/ogr/pull/538#issuecomment-840793867
For us, the maintainers of requre and packit, it's somewhat clear what the error message says. But for external contributors it can be tricky to decode it, how about we reworded the message to:
f'Unable to find response file {path} for test case {test_case_name}.
You are in read-only more now, that's why you are seeing this error message.
You should record the test case again.'

Would this be helpful?

Is l hard to find: {test_case_name} because cassette does not know anything about who called me.
Theoretically I can climb trough stack and found the latest stack info to path what is relative to pwd but it may lead to misleading output as well and could be ery noisy, because stack it too big.
I can improve it little but, but this information could be better to handle inside BaseClass in ogr. in tearDown, and check for this exception inside basic testCase where we can cath information about current testcase

https://github.com/packit/requre/pull/196

Issue fixed by PR.


yt-dlp --live-from-start feature

Checklist

[X] I'm asking a question and not reporting a bug/feature request
[X] I've looked through the README
[X] I've read the guidelines for opening an issue
[X] I've searched the bugtracker for similar questions including closed ones

Question
Hello, thanks for your work!
Have a question - yt-dlp --live-from-start feature in Linux uses too much RAM, maybe caching all parts somehow, but on Windows it doesn't take RAM.
Is this a feature or not?
Both systems using latest 2<IP_ADDRESS> version from pip install.
Verbose log
No response

It shouldn't take much more RAM than normal downloads

It shouldn't take much more RAM than normal downloads

Should I submit it as bug? 'Cause for me it takes more than 16 gb of ram

yes, but I'm not sure the normal logs would help here.
@Lesmiscore, can u reproduce? or have any idea how we can debug this

a sort of, probably
https://discord.com/channels/807245652072857610/807245652072857613/952195812438450236

-N 32 could freeze the system :pain:

https://discord.com/channels/807245652072857610/807245652072857613/952206614088282133 and the next message

my VPS with 4 GB memory
with 8GB memory laptop and file server, it usually won't be a problem

https://discord.com/channels/807245652072857610/807245652072857613/952207707639787530

~7MiB isn't probly
but the process is killed by OOM Killer

context: I was trying to reproduce #3043.
Here's yt-dlp -v of the VPS in the quotes:
$ yt-dlp -v
[debug] Command-line config: ['-v']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, err utf-8, pref UTF-8
[debug] yt-dlp version 2<IP_ADDRESS> [e491d06d3]
[debug] Python version 3.9.10 (CPython 64bit) - Linux-5.13.0-30-generic-x86_64-with-glibc2.34
[debug] exe versions: ffmpeg 4.4 (setts), ffprobe 4.4
[debug] Optional libraries: Crypto, sqlite
[debug] Proxy map: {}
Usage: yt-dlp [OPTIONS] URL [URL...]

yt-dlp: error: You must provide at least one URL.
Type yt-dlp --help to see a list of all options.

Since --live-from-start for YT itself shouldn't take memory so much, I suspect there could be problem(s?) on parallel fragment downloads.
Anyway it looks like I have to try to reproduce it too.

a sort of, probably
https://discord.com/channels/807245652072857610/807245652072857613/952195812438450236

-N 32 could freeze the system :pain:

https://discord.com/channels/807245652072857610/807245652072857613/952206614088282133 and the next message

my VPS with 4 GB memory
with 8GB memory laptop and file server, it usually won't be a problem

https://discord.com/channels/807245652072857610/807245652072857613/952207707639787530

~7MiB isn't probly
but the process is killed by OOM Killer

context: I was trying to reproduce #3043.
Here's yt-dlp -v of the VPS in the quotes:
$ yt-dlp -v
[debug] Command-line config: ['-v']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, err utf-8, pref UTF-8
[debug] yt-dlp version 2<IP_ADDRESS> [e491d06d3]
[debug] Python version 3.9.10 (CPython 64bit) - Linux-5.13.0-30-generic-x86_64-with-glibc2.34
[debug] exe versions: ffmpeg 4.4 (setts), ffprobe 4.4
[debug] Optional libraries: Crypto, sqlite
[debug] Proxy map: {}
Usage: yt-dlp [OPTIONS] URL [URL...]

yt-dlp: error: You must provide at least one URL.
Type yt-dlp --help to see a list of all options.

Since --live-from-start for YT itself shouldn't take memory so much, I suspect there could be problem(s?) on parallel fragment downloads.
Anyway it looks like I have to try to reproduce it too.

@random29 Did you use -N option at that time? If yes, what value did you set?

Yes, I use it, N = 16. Tried without it - no noticable RAM drain. Guess that's there the problem.
But man, without concurrent fragments stream loads quite slow, and this is not a problem on windows

Yes, I use it, N = 16. Tried without it - no noticable RAM drain

In that case, I believe we have found the issue and the fix 👍
@Lesmiscore You can directly commit the fix (that we discussed on discord) if your tests are giving good results. In future, it may be useful to cache a few fragments instead of reading everything back from disk, but that is low priority and we can address it later

@pukkandan Can I change it not to read fragments at download time completely?
diff --git a/yt_dlp/downloader/fragment.py b/yt_dlp/downloader/fragment.py
index 95fb2f9e7..9358ea6f7 100644
--- a/yt_dlp/downloader/fragment.py
+++ b/yt_dlp/downloader/fragment.py
@@ -133,19 +133,19 @@ class FragmentFD(FileDownloader):
         }
         success = ctx['dl'].download(fragment_filename, fragment_info_dict)
         if not success:
-            return False, None
+            return False
         if fragment_info_dict.get('filetime'):
             ctx['fragment_filetime'] = fragment_info_dict.get('filetime')
         ctx['fragment_filename_sanitized'] = fragment_filename
+        return True
+
+    def _read_fragment(self, ctx):
         try:
-            return True, self._read_fragment(ctx)
+            down, frag_sanitized = self.sanitize_open(ctx['fragment_filename_sanitized'], 'rb')
         except FileNotFoundError:
-            if not info_dict.get('is_live'):
+            if not ctx.get('live'):
                 raise
-            return False, None
-
-    def _read_fragment(self, ctx):
-        down, frag_sanitized = self.sanitize_open(ctx['fragment_filename_sanitized'], 'rb')
+            return False
         ctx['fragment_filename_sanitized'] = frag_sanitized
         frag_content = down.read()
         down.close()
@@ -467,11 +467,10 @@ class FragmentFD(FileDownloader):
                 headers['Range'] = 'bytes=%d-%d' % (byte_range['start'], byte_range['end'] - 1)

             # Never skip the first fragment
-            fatal = is_fatal(fragment.get('index') or (frag_index - 1))
-            count, frag_content = 0, None
+            fatal, count = is_fatal(fragment.get('index') or (frag_index - 1)), 0
             while count <= fragment_retries:
                 try:
-                    success, frag_content = self._download_fragment(ctx, fragment['url'], info_dict, headers)
+                    success = self._download_fragment(ctx, fragment['url'], info_dict, headers)
                     if not success:
                         return False, frag_index
                     break
@@ -497,7 +496,7 @@ class FragmentFD(FileDownloader):
                 ctx['dest_stream'].close()
                 self.report_error('Giving up after %s fragment retries' % fragment_retries)
                 return False, frag_index
-            return frag_content, frag_index
+            return True, frag_index

         def append_fragment(frag_content, frag_index, ctx):
             if not frag_content:
@@ -520,23 +519,23 @@ class FragmentFD(FileDownloader):

             def _download_fragment(fragment):
                 ctx_copy = ctx.copy()
-                frag_content, frag_index = download_fragment(fragment, ctx_copy)
-                return fragment, frag_content, frag_index, ctx_copy.get('fragment_filename_sanitized')
+                _, frag_index = download_fragment(fragment, ctx_copy)
+                return fragment, frag_index, ctx_copy.get('fragment_filename_sanitized')

             self.report_warning('The download speed shown is only of one thread. This is a known issue and patches are welcome')
             with tpe or concurrent.futures.ThreadPoolExecutor(max_workers) as pool:
-                for fragment, frag_content, frag_index, frag_filename in pool.map(_download_fragment, fragments):
+                for fragment, frag_index, frag_filename in pool.map(_download_fragment, fragments):
                     ctx['fragment_filename_sanitized'] = frag_filename
                     ctx['fragment_index'] = frag_index
-                    result = append_fragment(decrypt_fragment(fragment, frag_content), frag_index, ctx)
+                    result = append_fragment(decrypt_fragment(fragment, self._read_fragment(ctx)), frag_index, ctx)
                     if not result:
                         return False
         else:
             for fragment in fragments:
                 if not interrupt_trigger[0]:
                     break
-                frag_content, frag_index = download_fragment(fragment, ctx)
-                result = append_fragment(decrypt_fragment(fragment, frag_content), frag_index, ctx)
+                _, frag_index = download_fragment(fragment, ctx)
+                result = append_fragment(decrypt_fragment(fragment, self._read_fragment(ctx)), frag_index, ctx)
                 if not result:
                     return False

That is a bigger change since mhtml and ism also use the _download_fragment function. Can u make a PR?

Expected answer :sweat_smile:
No problem


Install --arch flag silently fails

When trying to install packages of different architectures, only -a 32bit/64bit works, contrary to the documentation provided by scoop help install, which states that --arch should work (I even tried --architecture, as mentioned in #235).
The --arch flag is not rejected like the --architecture flag is, but the file installed is the default for your system, rather than the architecture specified after the flag. This is certainly a minor issue with a workaround, but it's slightly confusing due to the silent failure.

Sorry yeah that was a mistake when I mentioned --architecture. I think the info in scoop help install is correct.
I do get an error when I run
$ scoop install test --architecture 32bits
scoop install: option --architecture not recognized

And
$ scoop install test -architecture 64bit
scoop install: option -a requires an argument

So I'm not sure what's going on there...

Sorry, I should also mention that --arch did work for me.

Do you have getopt installed on your path? It could be overriding scoop's implementation.

Hmm. Did it install the correct version when you used --arch? I'm not receiving any errors, but this is my output for various commands:
Using --arch (incorrect):
$ scoop install --arch 32bit mysql
installing mysql (5.7.11)
loading https://dev.mysql.com/get/Downloads/MySQL-5.7/mysql-5.7.11-winx64.zip from cache...

Using -a (correct):
$ scoop install -a 32bit mysql
installing mysql (5.7.11)
loading https://dev.mysql.com/get/Downloads/MySQL-5.7/mysql-5.7.11-win32.zip from cache...

So it's not so much an error, as it just doesn't seem to actually obey --arch, and goes with the system default.
And, at least according to get-command getopt, I don't have getopt installed on my path.

Also, I can confirm that using the architecture flag after the app name (as in, scoop install mysql --arch 32bit) has the same effect.

Should be fixed now. Thanks for patiently reporting this!

Not a problem. Glad I could help!


Updates

Right, so, there hasn't been much heard from me in the past long while concerning where I want things to go with this. I promise you I'm still working on this. Slowly, but I still am.
My main concern is getting everything fully localized. Followed next will be finishing planned additions/features. Afterwards will involve a private alpha where we security audit the hell out of the engine. Once this has concluded, and any and all bugs are squished, we'll be able to finally put this into a live instance!
I've been committed to keeping security a priority, and I will not cheap out on this. Thank you, and have a good evening folks.

excitied to see how this works, I've played around with a lot of engine but its good to see someone that hasn't abandoned one 5 years ago.


Fix for upstream changes

r? @alexcrichton
(rust_highfive has picked a reviewer for you, use r? to override)


perf: Remove the quadratic behavior in haskellPackages.ghcWithPackages

Description of changes
This code was using lib.closePropagation which internally do a recursion on the dependencies of ghcWithPackages and returns all the haskell dependencies.
lib.closeProgation is implemented in pure nix and uses an unique function for list which is quadratic and does "true" equality, which needs deep set comparison.
Things done
Instead, we use the builtins.genericClosure which is implemented as a builtin and uses a more efficient sorting feature.
Note that genericClosure needs a key to discriminate the values, we used the outPath which is unique and orderable.
On benchmarks, it performs up to 15x time faster.

Built on platform(s)

[X] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

Testing / Benchmark
I've tested this implementation on a large haskell codebase where I'm using nix as a build system. I have something like 250 calls to ghcWithPackages. When building everything, nix build --dry-run was tacking up to 20s, now it runs in 1.5s.
In order to compare, I've created the following benchmark, drop this file (name it fiz.nix) at the root of nixpkgs
let
  pkgs = import ./. { };

  haskellPackages = p:
    with p; [
      ad
      aeson
      AesonBson
      aeson-pretty
      algebraic-graphs
      async
      attoparsec
      barbies
      base
      base16
      base64-bytestring
      base-noprelude
      bazel-runfiles
      binary
      blaze-html
      bson
      bytestring
      case-insensitive
      casing
      cassava
      cereal
      cereal-text
      cereal-unordered-containers
      cereal-uuid
      cereal-vector
      clock
      conduit
      conduit-extra
      containers
      criterion
      cryptohash-sha1
      data-default
      deepseq
      directory
      erf
      exceptions
      extra
      filepath
      foldl
      formatting
      generic-data
      generics-sop
      ghc-prim
      githash
      Glob
      gloss
      hashable
      hedgehog
      hedis
      heredoc
      hmatrix
      hmatrix-nlopt
      hspec
      hspec-core
      hspec-wai
      http-client
      http-conduit
      http-media
      http-types
      hvega
      hxt
      integration
      JuicyPixels
      katip
      KdTree
      lens
      lexer-applicative
      linear
      megaparsec
      mime-types
      mmorph
      monad-classes
      monad-control
      mono-traversable
      mtl
      mwc-random
      openapi3
      optparse-applicative
      optparse-generic
      ordered-containers
      pandoc
      parallel
      parsec
      parser-combinators
      pqueue
      pretty
      pretty-show
      pretty-simple
      primitive
      process
      profunctors
      PyF
      QuickCheck
      quickcheck-instances
      quickcheck-text
      random
      random-fu
      reflection
      regex
      regex-applicative
      regex-compat
      resourcet
      retry
      safe
      safe-exceptions
      scientific
      selda
      semigroups
      servant
      servant-blaze
      servant-client
      servant-client-core
      servant-conduit
      servant-openapi3
      servant-server
      servant-swagger-ui
      singletons
      singletons-th
      split
      splitmix
      srcloc
      statistics
      stm
      storable-tuple
      streaming
      streaming-bytestring
      strict
      string-conv
      tagged
      tardis
      tasty
      tasty-expected-failure
      tasty-golden
      tasty-hedgehog
      tasty-hunit
      tasty-quickcheck
      tasty-wai
      template-haskell
      temporary
      text
      time
      tls
      transformers
      transformers-base
      trie-simple
      tuples-homogenous-h98
      typed-process
      uglymemo
      unicode-transforms
      unix
      unliftio
      unliftio-core
      unordered-containers
      url
      utf8-string
      uuid
      uuid-types
      vector
      vector-algorithms
      versions
      vinyl
      wai
      wai-extra
      warp
      yaml
      zip
      zlib
    ];
in builtins.genList (n:
  pkgs.haskellPackages.ghcWithPackages
  (p: pkgs.lib.lists.take n (haskellPackages p))) 167

Then I'm benchmarking with time nix build -f fiz.nix --dry-run
Without this change: 18.6s
With this change: 1.2s
Note that nix will still announce that it will fetch the same numbers of paths, but before that patch, it generates 94 ghc-xxx-with-packages.drv and with this patch it generates 147 distinct derivations. I don't understand why. I suppose that the ordering of the results is changing. I was however able to confirm that it behaves the same (by building my 250 packages and running their test suite).
This change should not (as far as I'm aware) trigger a mass rebuild because it only impacts shell / dynamic environments.

@ofborg build tests.haskell.shellFor

Damned, thank you @sternenseemann, it fails on shellFor, let's try to understand why and fix that.

shellFor has some extra code to merge multiple ghcWithPackages environment, maybe that gets tripped up somehow.

Could this function be put in lib so we can convert other users as well?

@FRidh It could yes. In which lib subfile do you think it will be at home?

If the behaviour is the same it could replace the original function. Or is there any reason for not doing that?

Will debugging, the new function is missing some values:
I'm trying to understand. That's weird because it is able to build my packages, so there may be a weird corner case I did not anticpiated.
@FRidh I'm not sure the new function is as general as the old one.  Simple mistake when copy pasting (I've refactored the item.propagatedNativeBuildInputs or [ ]  and missed the item.val).

@ofborg build tests.haskell.shellFor

The tests should now pass (at least they pass locally).
As suggested by @FRidh, I replaced the lib.closePropagation by the new implementation (with a conditional based on the existence of builtins.genericClosure).
Looks like the behavior should be highly similar, except for the ordering of the results. There are not much usage of this function:
nixos/modules/services/audio/mopidy.nix:    paths = closePropagation cfg.extensionPackages;
pkgs/applications/editors/eclipse/default.nix:            filter (x: x ? isEclipsePlugin) (closePropagation plugins);
pkgs/development/compilers/yosys/default.nix:      paths = lib.closePropagation plugins;
pkgs/development/haskell-modules/hoogle.nix:  docPackages = lib.closePropagation
pkgs/development/haskell-modules/with-packages-wrapper.nix:  paths         = lib.filter (x: x ? isHaskellLibrary) (lib.closePropagation packages);
pkgs/development/idris-modules/with-packages.nix:let paths = lib.closePropagation packages;
pkgs/development/tools/misc/hydra/unstable.nix:    paths = with perlPackages; lib.closePropagation
pkgs/top-level/perl-packages.nix:  makeFullPerlPath = deps: makePerlPath (lib.misc.closePropagation deps);

A quick look makes me think that we may be safe to do this change on the complete nixpkgs. However it may trigger a massive rebuild, so perhaps it should be merged to another branch. What do you think?

Yeah, as-is this now needs to target staging due to the mass rebuilds

@lovesegfault rebased on staging, thank you.

@guibou Please add a changelog entry for this to the release notes for 21.11, since it is a breaking change (which is probably not too worrysome, since probably very few downstream users will be using it without having an outPath attribute). It should mention:

Improved performance of lib.closePropagation for e.g. ghcWithPackages.
The breakage, i.e. that every set needs an outPath now.

@sternenseemann documentation added (note, I changed the notes for 22.11 instead of 21.11 as you suggested. Please tell me if you were really meaning 21.11). Thank you @teto who also told me to regenerate the db thing.

@sternenseemann done.

So, this breaks my dev shell, and I'm not sure how to fix it.
{ pkgs ? import <nixpkgs> {},
  ghc ? pkgs.haskell.packages.ghc924,
llvm ? pkgs.llvmPackages_13.llvm,
}:

with pkgs;

let
  ghc' = ghc.ghcWithPackages(ps: with ps; [
haskell-language-server cabal-install
profiterole profiteur
ghc
  ]);
in
pkgs.stdenv.mkDerivation {
  name = "cosmothought";
  nativeBuildInputs = [pkg-config pcre];
  buildInputs = [
    zlib icu ncurses.dev llvm icu racket pcre haskellPackages.alex haskellPackages.happy
    bzip2 ghc'
    ];
  shellHook = "eval $(egrep ^export ${ghc'}/bin/ghc)";
}

The last few lines of the trace are:
❯ nix-shell --show-trace
error: attribute 'outPath' missing

       at /nix/store/r2v7sig14g93wwf6ljr0jb0q1v2d1ngh-nixpkgs/nixpkgs/lib/deprecated.nix:171:15:

          170|       startSet = builtins.map (x: {
          171|         key = x.outPath;
             |               ^
          172|         val = x;

       … while evaluating 'closePropagationFast'

       at /nix/store/r2v7sig14g93wwf6ljr0jb0q1v2d1ngh-nixpkgs/nixpkgs/lib/deprecated.nix:168:26:

          167|   # See https://github.com/NixOS/nixpkgs/pull/194391 for details.
          168|   closePropagationFast = list:
             |                          ^
          169|     builtins.map (x: x.val) (builtins.genericClosure {

       … from call site

       at /nix/store/r2v7sig14g93wwf6ljr0jb0q1v2d1ngh-nixpkgs/nixpkgs/pkgs/development/haskell-modules/with-packages-wrapper.nix:57:57:

           56|   packageCfgDir = "${libDir}/package.conf.d";
           57|   paths         = lib.filter (x: x ? isHaskellLibrary) (lib.closePropagation packages);
             |                                                         ^
           58|   hasLibraries  = lib.any (x: x.isHaskellLibrary) paths;

       … while evaluating anonymous lambda

       at /nix/store/r2v7sig14g93wwf6ljr0jb0q1v2d1ngh-nixpkgs/nixpkgs/pkgs/development/haskell-modules/with-packages-wrapper.nix:17:1:

           16| #   (hpkgs: [ hpkgs.mtl hpkgs.lens ])
           17| selectPackages:
             | ^
           18|

       … from call site

       at /home/spacekitteh/code/cosmothought/shell.nix:11:10:

           10| let
           11|   ghc' = ghc.ghcWithPackages(ps: with ps; [
             |          ^

@spacekitteh

ghc ? pkgs.haskell.packages.ghc924 introduces the ghc name into the scope statically…
…which takes precedence over dynamic resolution via with. Specifically with ps; [ /* … */ ghc ] will then evaluate to effectively [ haskell.packages.ghc94 ] which of course lacks an outPath attribute.

@sternenseemann Ok, removing the ghc fixed it, but why does it not have an outPath attribute?

haskell.packages.ghc94 is a package set of derivations, not a single derivation. ps.ghc is a derivation, but shadowed due to Nix's scoping rules.

Ah. So what would it have been bringing in before this patch?

Nothing, as it would have been filtered out by this line in with-packages-wrapper.nix:
  paths         = lib.filter (x: x ? isHaskellLibrary) (lib.closePropagation packages);


Missing error and error_description when acquiring the token interactively

I am using the InteractiveRequestParameters to acquiring the token Interactively, which is working properly when there are no problems.
But when there are problems, like the authorization code isn´t provided to the redirect url, then the only message from the browser is:
Authentication failed. You can return to the application. Feel free to close this browser tab.

Error details: error {0} error_description: {1}

I captured the request, and the error and error_description are present - so the MSAL gets this information, but did´t return it in their response to the browser.
This is the captured request (I truncated it a bit)
POST http://localhost:12345/ HTTP/1.1
Content-Type: application/x-www-form-urlencoded

error=invalid_request&error_description=AADB2C90146%3A+The+scope+%27openid+profile+offline_access...

My Questions:

Is this a bug or do I miss something.
How can I get the error and error_description programmatically? I already activated DEBUG Level logging but I only get:
"Exception in thread "main" java.util.concurrent.CompletionException: com.microsoft.aad.msal4j.MsalClientException: No Authorization code was returned from the server" instead of the real reason, which is statet in 'error_description'

Thank you in advance :)

Hello @deBFM : That error message with Authentication failed. You can return to... is just a default error message that our library originally only provided if there was no error message in the HTML response, but it looks like a couple years ago it was changed to also be the default if systemBrowserOptions is null.
I'm still looking into why it was changed to check systemBrowserOptions, but it does seem like it's either a bug/deficiency in that logging or there's something else that should be providing a better error message but isn't. Either way it definitely seems like something that needs a fix, and will be in our next release. I don't have an ETA now, but I'll update this thread once we have more info.

I am currently working with MSAL4J to replace basic authentication in an application and noticed that the error and the description are missing. The default message looks like it it expects the error and the error_description to be passed into MessageFormat.format.
It seems like the information is there but it is not passed into the error message (see AuthorizationResponseHandler#L75).
I am now creating a custom error page and in future I would like to be able to display the error there.
Ideally, I would like to pass a  Function which gets the result and returns the html string, so I can use custom translations for the error messages.

The expected behavior is for MSAL to throw a service exception here, and for the error and error_description to be encapsulated in that exception. The 2 strings should also be included in the exception message.

can this be fixed? i made an error and it took a lot of effort to just to realize where this issue is coming from because the error message doesn't say anything.
i had to debug inside msal to figure this out


Fix UG

Codecov Report
Base: 72.15% // Head: 72.15% // No change to project coverage :thumbsup:

Coverage data is based on head (d8ae719) compared to base (550926b).
Patch has no changes to coverable lines.

Additional details and impacted files
@@            Coverage Diff            @@
##             master      #45   +/-   ##
=========================================
  Coverage     72.15%   72.15%
  Complexity      399      399
=========================================
  Files            70       70
  Lines          1232     1232
  Branches        125      125
=========================================
  Hits            889      889
  Misses          311      311
  Partials         32       32

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


thumbnail as per width specific not working

tried with a thumbnail - not able to fix width and height,
$image = Vips\Image::thumbnail($filename, 200, ['height' => 200]);
$image = Vips\Image::thumbnail($filename, 100,['width' => 100]);
not able to with specific proportion to width wise, as this is working height wise even i mention option width and after filename param always considered as min-height
How can I achieve following with above params pass to method?
need thumbnail w=100 and proportion as height
thumbnail - 100wx100h  etc.

Hello, do you want to fix the width at 100 and let the height float? Just give the width and don't specify a height.
If you want to resize to an exact size and break the aspect ratio, give a height as well and set size to force.

Okay! It works like this: $image = Vips\Image::thumbnail($filename, 200,['height' => 200,'width' => 100, 'size' => 'force']);
otherwise take care of width or height propotion as per more size side in image.

The first 200 sets width, so you don't need to set it again. Try:
$image = Vips\Image::thumbnail($filename, 100,['height' => 200, 'size' => 'force']);

Otherwise ... looks good!

Hello, try:
vipsthumbnail *.jpg --size 200x200<

Hi,
/test $ vipsthumbnail *.jpeg --size 200x200<
bash: syntax error near unexpected token `newline'

Yes, for bash you need to put quotes around the argument or it'll think you're doing IO redirection.
vipsthumbnail *.jpg --size "200x200<"

Hi,
vipsthumbnail *.jpg --size "200x200<"
in case of false condition thumbnail generated of actual images size with prefix = "tn_"
seems there is bug, as it should not generate if condition false, ex: width not less than 200px

It always generates the output file -- the < just stops the resize stage.


Gh 1294 add reactive sc loadbalancer support

Fixes gh-1294.

Applied changes from review. Merging.


Very confusing for the Defender for Endpoint section

[Enter feedback here]
The following needs clarification or rewording the way it is written it leads customers to believe that Microsoft Defender for Endpoint in Azure Defender is for Windows only.
Integrated license for Microsoft Defender for Endpoint (Windows only) - Azure Defender for servers includes Microsoft Defender for Endpoint. Together, they provide comprehensive endpoint detection and response (EDR) capabilities. For more information, see Protect your endpoints.
When Defender for Endpoint detects a threat, it triggers an alert. The alert is shown in Security Center. From Security Center, you can also pivot to the Defender for Endpoint console, and perform a detailed investigation to uncover the scope of the attack. Learn more about Microsoft Defender for Endpoint.
Important
The Microsoft Defender for Endpoint sensor is automatically enabled on Windows servers that use Security Center.
I suggest moving the Windows only out and adding a sub header right before or as part of this statement and adding Windows  #Only.
Important  (Windows Only)
The Microsoft Defender for Endpoint sensor is automatically enabled on Windows servers that use Security Center.
Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 8432d9d8-e414-acf4-9287-ebbafab23c7e
Version Independent ID: 9eb184bd-e66d-6b1b-6b86-509ebe70bd3e
Content: Azure Defender for servers - the benefits and features
Content Source: articles/security-center/defender-for-servers-introduction.md
Service: security-center
GitHub Login: @memildin
Microsoft Alias: memildin

Thanks for the feedback! We are currently investigating and will update you shortly.

Hi @stscholz914, thanks for the feedback and for taking the time to help us improve the docs.
This documentation page is going to get refreshed in the coming weeks, and we'll incorporate the information discussed here.
I'm going to close this issue, but feel free to comment again here or open a new one if you have additional requests.
#please-close


Package Manager Support

Hi, would it be possible to add Unity's package manager support for easy updating straight from the git repository? Thanks for a great asset btw!

I will be adding this to UPM! I just want get a few more major feature's done before adding it.

Okeydokey, thanks man! 😃


A question about  the L2Norm.py code

Hello, I don't understand why you calculate out = weight*x before return out and not return the x straightly.Could you tell me the reason?
thx~~   :）
def forward(self, x):
norm = x.pow(2).sum(1).sqrt()+self.eps
x/=norm.expand_as(x)
out = self.weight.unsqueeze(0).unsqueeze(2).unsqueeze(3).expand_as(x) * x          <= here
return out

Hi! The reasoning behind this stems from the fact that convolution 4_3 learns to recognize features at a different scale so the norm at each location of the feature map is initially scaled to 20 and then during backprop the scale is learned by means of the weight matrix that you're referring to in our code.  This is explained in the Experiment Results section of the SSD paper here.

@fanser In L2Norm function, what scale is used for? i dont get the meaning of it used in this function :/


Pass plugin config via stdin

What does this change
Reads optional configuration from the porter configuration file and passes it to the plugin on stdin as json.
It is not sent as part of the plugin interface/contract because each plugin may or may not have config, it may look completely different or be loaded from other places, and I'd like to have the plugin contract mimic the cnab contracts. For example the interface for claims is the CrudStore.
default-instance-store = "porterci"

[[instance-store]]
  name = "porterci"
  plugin = "azure.blob"

  [instance-store.config]
    env = "PORTERCI_AZURE_STORAGE_CONNECTION_STRING"

What issue does it fix
N/A
Notes for the reviewer
This relies on a patch: https://github.com/hashicorp/go-plugin/pull/127
This is the first step in getting a general framework in place for plugins. Next steps after this will be to make the pluginProvider generic so it can load a plugin and config for any interface. Right now I'm just feeling out what works.

/azp run porter
/azp run porter-integration

I was going to hold off until someone can install a plugin without having to compile it themselves.


Update the source information

Type: Missing
Summary:
Update the source information
Details:
With the rise in supply chain attacks and OSS dependencies being used as a attack vector, Microsoft is working with our ecosystem partners, such as the Linux Foundation's OpenSSF, to enable OSS consumers to track packages back to their public sources.
Resolution:
This PR was created to add this value, ensuring future releases will include this provenance information.
Published NPM packages with repository information:
* minipass-collect
Affected definitions:

minipass-collect 1.0.2

Same as #13370


Improve the recommended way to use Celery

Currently docs are recommending to call run() method after creating a Flask context. It seems that calling super().__call__() instead would be a better approach, because calling run() directly skips Celery's internal threading stack management:
https://github.com/celery/celery/blob/master/celery/app/task.py#L382
    def __call__(self, *args, **kwargs):
        _task_stack.push(self)
        self.push_request(args=args, kwargs=kwargs)
        try:
            return self.run(*args, **kwargs)
        finally:
            self.pop_request()
            _task_stack.pop()

I've applied this change in our project and can confirm that it still works without any issues.

We already went through this in reverse in #2055. After extensive testing and digging through the code, I came to the conclusion that what is currently shown is correct. You'll need an authoritative reason that using __call__ instead of run is actually correct in order for me to change this again. I'm not willing to investigate this myself again.

While the Celery code you've linked appears to indicate that your change is correct, from what I remember there's actually code in a seemingly unrelated area that monkeypatches it in other situations, and that messes up things with your change.

Closing this for now. I think my previous conclusion is still valid. If you get a definitive answer, comment here and we can revisit it.


Add a --strict-properties option

Fixes #53.

Codecov Report

Merging #56 into master will decrease coverage by 1.44%.
The diff coverage is 95.65%.

@@            Coverage Diff            @@
##           master     #56      +/-   ##
=========================================
- Coverage   88.95%   87.5%   -1.45%
=========================================
  Files          29      29
  Lines        2626    2786     +160
=========================================
+ Hits         2336    2438     +102
- Misses        290     348      +58

Impacted Files
Coverage Δ

stix2validator/scripts/stix2_validator.py
0% <0%> (ø)
:arrow_up:

stix2validator/musts.py
93.61% <100%> (+1.08%)
:arrow_up:

stix2validator/test/observed_data_tests.py
100% <100%> (ø)
:arrow_up:

stix2validator/test/indicator_tests.py
100% <100%> (ø)
:arrow_up:

stix2validator/test/custom_obj_tests.py
100% <100%> (ø)
:arrow_up:

stix2validator/util.py
82.27% <50%> (+4.5%)
:arrow_up:

stix2validator/output.py
43.82% <0%> (-2.09%)
:arrow_down:

stix2validator/test/bundle_tests.py
100% <0%> (ø)
:arrow_up:

stix2validator/__init__.py
100% <0%> (ø)
:arrow_up:

... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 95fa281...593356e. Read the comment docs.


Update the preview GIF

The preview in the readme is outdated and should be updated.

Close via #50.


ERROR CODE: 10013

My rat wont connect, its showing the below error message.
"An unexpected socket error occurred: An attempt was made to access a socket in a way forbidden by its access permissions."
Kindly assist

You might have another program listening on the requested socket. use the command prompt with the command "netstat -ano | findstr "YourPortHere"" and replace YourPortHere with the port you are trying to use. If you get results back, you can kill them by "taskkill /f /pid number" and replace number with the number at the end of the result from netstat.


No module named 'decorator' on webapp up

This is autogenerated. Please review and update as needed.
Describe the bug
Brand new install of Fedora 36. Installed azure-cli got this error. Upgraded using az upgrade, same error.
I saw a closed issue with a similar problem for an older version, but it appears that this is not fixed or has reared its head again.
Command Name
az webapp up
Errors:
The command failed with an unexpected error. Here is the traceback:
No module named 'decorator'
Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/fabric/connection.py", line 5, in <module>
    from invoke.vendor.six import StringIO
ModuleNotFoundError: No module named 'invoke.vendor'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/knack/cli.py", line 231, in invoke
    cmd_result = self.invocation.execute(args)
  File "/home/adam/.local/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 556, in execute
    self.commands_loader.load_arguments(command)
  File "/home/adam/.local/lib/python3.10/site-packages/azure/cli/core/__init__.py", line 515, in load_arguments
    self.command_table[command].load_arguments()  # this loads the arguments via reflection
  File "/home/adam/.local/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 313, in load_arguments
    super(AzCliCommand, self).load_arguments()
  File "/usr/lib/python3.10/site-packages/knack/commands.py", line 104, in load_arguments
    cmd_args = self.arguments_loader()
  File "/home/adam/.local/lib/python3.10/site-packages/azure/cli/core/commands/command_operation.py", line 125, in arguments_loader
    op = self.get_op_handler(self.op_path)
  File "/home/adam/.local/lib/python3.10/site-packages/azure/cli/core/commands/command_operation.py", line 59, in get_op_handler
    handler = import_module(mod_to_import)
  File "/usr/lib64/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/adam/.local/lib/python3.10/site-packages/azure/cli/command_modules/appservice/custom.py", line 25, in <module>
    from fabric import Connection
  File "/usr/lib/python3.10/site-packages/fabric/__init__.py", line 3, in <module>
    from .connection import Config, Connection
  File "/usr/lib/python3.10/site-packages/fabric/connection.py", line 10, in <module>
    from decorator import decorator
ModuleNotFoundError: No module named 'decorator'

To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

Put any pre-requisite steps here...
az webapp up --os-type {}

Expected Behavior
Environment Summary
Linux-5.17.6-300.fc36.x86_64-x86_64-with-glibc2.35, Fedora Linux 36 (Workstation Edition)
Python 3.10.4
Installer: PIP

azure-cli 2.36.0

Additional Context

route to CXP team

@adamringeisen Can you please look this issue and try the action plan provided there? https://github.com/Azure/azure-cli/issues/13343#issuecomment-635811624

Well the action described there is to upgrade to a newer version of the cli and I am using the newest version so no I can’t try the action described there.

@adamringeisen Apologies, i mean this comment https://github.com/Azure/azure-cli/issues/13343#issuecomment-624367680. I am still looking into it, in the meantime give it a try.

The command didn't work unless I removed the --system argument.
Then it did work but I got a warning about using sudo with pip potentially causing errors with system package manager.
file:///home/adam/Pictures/Screenshots/Screenshot%20from%202022-05-15%2005-56-42.png

this seems like a problem with CLI install package - please try with a brand new full  re-install of CLI.

Hello, already followed all the steps and is still not working.
Any solution?


Expressions, scopes and return values

This is pretty much the same as Rust.
In a scope, the result of evaluating the last expression is the value assigned to the scope. If this scope is the body of a function, that value is returned, or you can explicitly return using the return keyword.
These are equivalent:
fn foo() : i32 {
    4
}

fn bar() : i32 {
    return 4;
}

rust doesn't have a semicolon if it's an implicit return tho

Maybe we could replace the implicit return with
fn foo(): i32 = 4;

But if somebody wants to put logic in before an implicit return, that wouldn't work.

But if you're going to do that you might as well just use the explicit return statement

More typing

less confusion

Let me change my example (again...)

when we get around to lambda or something like linq or the map func(rust) not haveing the return keyword is priceless, we can make it good practice to use the full return keyword when it makes sense but it should still be optional


Incorrect statement on number of cluster agents

Summary:
Documentation at https://docs.ranchermanager.rancher.io/reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters#2-cluster-controllers-and-cluster-agents incorrectly states a cluster has one cluster agent.
Details:
The text is:

There is one cluster controller and one cluster agent for each downstream cluster.

However, in a cluster, seeing two:
$ kubectl get pods -n cattle-system | grep cattle-cluster-agent
cattle-cluster-agent-85fd974484-m8gpk        1/1     Running   0          7d7h
cattle-cluster-agent-85fd974484-gmbsb        1/1     Running   0          7d7h

Should check with engineering. In some cases there is only 1.

Hi, I would like to work on this issue. Could you assign it to me?

@mantosh-mariner Thank you for volunteering to help with this. I've assigned this issue to you as requested.


Navigation performance api

Hi All,
I am trying to monitor the load times of our application which uses turbolinks. However, when I am using the navigation performance API, the requestStart seems to be always the same.
In our application we also embed tableau dashboards, I am trying to measure how much time the dashboard is taking to load, tableau provides a event called onFirstInteractive which is fired when a embedded dashboard is loaded.
The page that has a embedded dashboard can be opened by clicking a link or directly copy / pasting a link in the browser.
I am trying to access the time when a link was clicked to open a dashboard by accessing window.performance.timing.requestStart, however it always is the same.
Can you please guide me in accessing the actual start time of when a request was started using navigation performance API?

Hi @aksharj, timing information for Turbolinks loads is available in the event.data.timing event object on turbolinks:load. e.g.
document.addEventListener('turbolinks:load', function (event) {
  // event.data.timing === { requestEnd: …, requestStart: …, visitEnd: … visitStart: … }
})

Hope that helps.


How to make child as root and and maintain hierarchy

Hi,
Is it possible to make child as root (it should be at same position i.e current place of the node should not change) and sub child of the children should maintain hierarchy
?
Currently I am doing like this and it is working fine. Is there any other good approaches
**

var position=this.getFlatNodeIndexPosition(node); // flat list position

        this.deleteNode(this.items,node.data);

        node.data.parentId=0;

      this.items.splice(position,0,node.data);

**
And also while dragging, how to move children along with parent? currently, parent is moving and children are moving in background.

Hi,

Doesn't look like an issue for this library - I suggest you open a stackoverflow item.
Please notice that you need to call update() on the treeModel if you mutate the tree structure.
currently there's no plan to support custom drag image. You can implement it using a custom template.


Hide Survey Popup on Fandom

What filter does this issue target?
Fandom Cleanup https://letsblock.it/filters/fandom-cleanup
What changes should be made?

Survey popup in the bottom left.
Did you write a uBlock filter solving this?
fandom.com##.notifications-placeholder
Additional notes
This fixes it for me on a bunch of random pages I tested it on, but I'm not familiar with ublock filters. Figure I'd mention it though.

Hello @hianick, thanks for your contributions, I'm opening https://github.com/letsblockit/letsblockit/issues/509 to implement it.


Setup CI and continuous build

Most immediate candidate is Travis CI, of course.

Presubmit and continuous builds are now setup with the internal CI system.
Presubmits are also auto-triggered for org members and repo collaborators on pull requests.
Otherwise, to manually trigger a build, add kokoro:run label to a pull request.
A couple things left to do:

update tools/ci-* scripts to actually build and test useful targets
possibly speed up builds using bazel's remote caching
make build/test results public. they're currently corp-only. will be working on this next week.

Is the result purely static site? Maybe use https://www.netlify.com/


Fix broken links

during the migration some links are broken now, make sure to keep the backticks so the links will be created correctly.

Fixed in https://github.com/microsoft/playwright/commit/2a5acfac97b0b453369e8c663906da3eee4044f0


Fix alert's error message

error is not an object so error.message will return 'undefined'

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. In order for us to review and merge your code, please sign up at https://code.facebook.com/cla - and if you have received this in error or have any questions, please drop us a line at<EMAIL_ADDRESS>Thanks!

By analyzing the blame information on this pull request, we identified @spicyj and @javache to be potential reviewers.

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Facebook open source project. Thanks!

@facebook-github-bot shipit

Thanks for importing.If you are an FB employee go to Phabricator to review internal test results.


Add speaker name - Frank Wiles

It was presented on YouTube video recording,
from 0:00 to 0:11 (first 11 seconds of the recording).

Thank you @kwadrat

Thank you!


[SPARK-36263][SQL][PYTHON] Add Dataframe.observation to PySpark

What changes were proposed in this pull request?
With SPARK-34806 we can now easily add an equivalent for Dataset.observe(Observation, Column, Column*) to PySpark's DataFrame API.
Why are the changes needed?
This further aligns the Python DataFrame API with Scala Dataset API.
Does this PR introduce any user-facing change?
Yes, it adds the Observation class and the DataFrame.observe method.
How was this patch tested?
Adds test test_observe to pyspark.sql.test.test_dataframe.

@cloud-fan here is a follow-up to #33422.

Can one of the admins verify this patch?

ok to test

Test build #141536 has started for PR 33484 at commit d192861.

Test build #141536 has finished for PR 33484 at commit d192861.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141536/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46054/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46054/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46054/

Test build #141545 has started for PR 33484 at commit fbde8d2.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46063/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46063/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46063/

Test build #141545 has finished for PR 33484 at commit fbde8d2.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141545/

* This patch **fails Spark unit tests**.

Unrelated org.apache.spark.sql.kafka010.KafkaRelationSuiteV2.explicit offsets failed with timeout.

Test build #141553 has started for PR 33484 at commit 7fc332d.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46071/

Test build #141559 has started for PR 33484 at commit 49e9b34.

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46071/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46071/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46077/

Test build #141553 has finished for PR 33484 at commit 7fc332d.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46077/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141553/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46077/

Can one of the admins verify this patch?

Test build #141559 has finished for PR 33484 at commit 49e9b34.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141559/

Test build #141565 has started for PR 33484 at commit 6d0a418.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46083/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46083/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46083/

Test build #141565 has finished for PR 33484 at commit 6d0a418.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141565/

Test build #141585 has started for PR 33484 at commit 92d785b.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46102/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46102/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46102/

Test build #141585 has finished for PR 33484 at commit 92d785b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141585/

@HyukjinKwon thanks for the input and all the pointer, everything looks green now.

Can one of the admins verify this patch?

ok to test

Test build #141603 has started for PR 33484 at commit 92d785b.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46119/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46119/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46119/

Test build #141603 has finished for PR 33484 at commit 92d785b.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141603/

@HyukjinKwon Thanks for the thorough comments above. Waiting for outcome on "using Scala signature" comment and "returning Map" comment before pushing above fixes.

retest this please

@EnricoMi seems like you forgot to push the changes you made 😂

Test build #141610 has started for PR 33484 at commit 92d785b.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46127/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46127/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46127/

Test build #141610 has finished for PR 33484 at commit 92d785b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141610/

@EnricoMi seems like you forgot to push the changes you made joy

@HyukjinKwon seems like you haven't seen my comment on that :smile:. No worries, you have answered my last two questions. Will push now.

Test build #141615 has started for PR 33484 at commit fa8c802.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46132/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46132/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46132/

Test build #141615 has finished for PR 33484 at commit fa8c802.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141615/

retest this please

Test build #141634 has started for PR 33484 at commit fa8c802.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46150/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46150/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46150/

Test build #141634 has finished for PR 33484 at commit fa8c802.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141634/

* This patch **fails Spark unit tests**.

@HyukjinKwon These test failures seem to be transient and unrelated. This time org.apache.spark.sql.kafka010.KafkaRelationSuiteV2.SPARK-30656: minPartitions failed, last time it was org.apache.spark.sql.execution.ui.AllExecutionsPageSuite.SPARK-27019: correctly display SQL page when event reordering happens.

Can you rerun Github actions job? https://github.com/G-Research/spark/actions/runs/1066412255

retest this please

@HyukjinKwon I have rebased my branch to latest master, there are so many red jobs and every rerun they change.

Test build #141704 has started for PR 33484 at commit 90675b0.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46217/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46217/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46217/

Test build #141704 has finished for PR 33484 at commit 90675b0.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141704/

Yay, all green! Quick, before something reruns :smile:.

thanks, merging to master!


Refactor OpenCypher specific code into Neo4j and Neptune formats

Originally, openCypher was chosen for the first csv format to add to Gaffer as it is used by Neo4j and Amazon Neptune. However, the actual formats used by both differ very slightly in that the headers are different. This causes awkward checks to see which is used in the import and export code.
There is extra confusion because the import will detect the format but the export can't and so the format must be selected by the user in a flag. As well as this, not many people know that openCypher is the csv format used by Neo4j and Neptune.
Therefore, there should be two formats, one for Neo4j and one for Neptune, and this format should be shared by the import and export code.

One of the key parts of the headers for these openCypher formats is that properties are tagged with types. Currently this works differently across the import and export code:
Import
https://github.com/gchq/Gaffer/blob/f4e0b940ae056da246e2252f49e402178e8b8cdf/core/data/src/main/java/uk/gov/gchq/gaffer/data/generator/OpenCypherCsvElementGenerator.java#L103-L106
Export
https://github.com/gchq/Gaffer/blob/371a6d6e9cd7ceb55caa51350bc7f0f93708c4ec/core/store/src/main/java/uk/gov/gchq/gaffer/store/operation/handler/output/ToOpenCypherCsvHandler.java#L72-L74
This should also be refactored into one class that can map String to java type, and vice versa.

Eventually it would be great to add more types so that the road-traffic example can use the import csv functionality, rather than a custom generator.

Closed in favour of https://github.com/gchq/Gaffer/issues/2726