nginx:access-logs permission denied

clean install on a new virtual machine and things are running fine but the first time I ran ssh -t dokku@dokku nginx:access-logs myapp -t I got permission denied.
Further investigation shows the /var/log/nginx folder with drwxr-x--- www-data adm permissions, preventing the dokku user from accessing the folder.
I'm not sure what the "best" solution is here. If I just allow +x on the folder then inotify doesn't work with tail (it needs to be able to read the directory). So the folder probably needs to be changed to the dokku group or something else.
This is using ubuntu 14.04 LTS, dokku v0.4.2.

I was pretty sure we did that on install. I'll have to look when I'm back at a computer.

I can't find a reference to anything that looks like it'd touch /var/log/nginx anywhere in the debian preinst/postinst scripts.

I think it's in an install trigger. Maybe under the nginx-vhosts plugin.

Ok well I still can't find it if it's there and my filesystem says otherwise!

Here it is
https://github.com/progrium/dokku/blob/master/plugins/20_events/install#L12-L13

@michaelshobbs that only applies to /var/log/dokku, not /var/log/nginx.

Ah! Fair point.  I'm not sure I see your point there.

if [[ $(id -un) != "dokku" && $1 != plugin:install && $1 != "plugin:update" && $1 != nginx:*-logs ]]; then
sudo -u dokku -E -H $0 "$@"
exit
fi

I don't understand that check. That's checking if the user is **not** dokku then to run things as root if the command is **NOT** any of those? I'd expect something closer to (assuming I knew bash well enough):

if [[ $(id -un) != "root" && ($1 == plugin:install || $1 == "plugin:update" || $1 != nginx:*-logs) ]]; then
sudo -u dokku -E -H $0 "$@"
exit
fi

Wait, nevermind... the check is trying to explicitly run things as dokku. Fine. This means that the /var/log/nginx must be accessible by the dokku user which as I've stated previously is not true.

Right and to my point that I'm not sure why your /var/log/nginx perms differ. That's all. Did you already have nginx installed before installing dokku perhaps?

It was a new VM where the first thing I did was run the bootstrap script.

I must have misread you previous response. Your directory permissions are correct and match mine. The point I was actually attempting to make is that dokku nginx:access-logs myapp -t should be executed as root not dokku.
However, I was not testing over ssh and I think that is the issue. Can you attempt to run dokku nginx:access-logs myapp -t  locally on the instance to verify?

Ok so this is definitely an issue over SSH only. I'll look into how best to patch this.

@mmerickel mind commenting here with the issue you are seeing and your personal recommendations?

AFAICT the fix in #1585 is not actually a fix because nginx's logrotate script switches new log files back to www-data:adm so unless that script is switched to www-data:dokku or dokku is added to the adm group this is still a problem.
Came back to work on Monday after logrotate and my access-log tail was busted again with permission denied.
/var/log/nginx/*.log {
        weekly
        missingok
        rotate 52
        compress
        delaycompress
        notifempty
        create 0640 www-data adm
        sharedscripts
        prerotate
                if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
                        run-parts /etc/logrotate.d/httpd-prerotate; \
                fi \
        endscript
        postrotate
                service nginx rotate >/dev/null 2>&1
        endscript
}

I'd recommend updating the work done to s/invoke-rc.d/service to also s/www-data adm/www-data dokku if you are going to continue down this route. The simpler fix is to revert #1585 and just add dokku to the adm group but I remember @josegonzalez pushing back on that in irc due to security concerns despite adm being used for only logs.

Are you sure you're still seeing this issue? In my installation, the log files are world-readable. can you post the output from ls -l /var/log/nginx after a logrotate has completed?

$ ls -l /var/log | grep nginx
drwxr-x--x 2 www-data  dokku   4096 Nov  6 06:45 nginx
$ ls -l /var/log/nginx
-rw-r----- 1 www-data adm         0 Nov  6 06:45 myapp-access.log

I'm not sure what the confusion is but the logrotate conf file I pasted above clearly shows the 0640 permissions after rotate.

Ah yea. I missed that. Sorry. This should be simple enough to patch.


CVE URLs update: www sub-subdomain no longer valid

Thanks @webmaven!

You're welcome. Was wondering if adding the new CVE (http://cve.mitre.org/cgi-bin/cvename.cgi?name=2018-18074) would also be an appropriate documentation change at this point, but would need to know next release's version number to include it.

@webmaven, thanks for the offer! We’re likely going to be doing a release sometime next week, and I think we’ll get all of that info bundled during the release process.

@nateprewitt and @webmaven Do you have any insight as to whether or not this change could be applied to versions 2.6.0 and 2.7.0 and, if so, would doing so address the security concern outlined in the CVE? We're distributing those versions in a public repo and are considering just applying this change instead of forcing users through an upgrade quite far forward. Any thoughts would be much appreciated. :)

Hi @cachedout, I think you could apply the patch in #4718 (or a derivative) to the head of 2.6 or 2.7. We don’t have any intention to maintain that in Requests though since both of those release are approaching 4 years since release and are 13 versions behind.
If you choose to go down that path for Saltstack, we probably want to make it clear that it’s a forked version of Requests at that point. If you’re already vendoring copies though, that may not be a problem.

@nateprewitt Totally understood and thanks for the quick reply. That gives me what I need. Thanks!


Only report missing expected failures for request types that ran

Filter out expected failures for request types that weren't actually run
to reduce confusion when reading through the wrapper output.

@swift-ci please test


[REVIEW] Inner & Left Join API cleanup

Single column inner and left joins have been replaced with multi column data type agnostic function calls i.e. gdf_(inner/left)join(generic/i8/i16..) are replaced with gdf_(inner/left)_join.
Google Test and Python Tests have been changed to reflect the new API.

@nsakharnykh @mt-jones we have the bindings PR ready to go here: gpuopenanalytics/pygdf#203 so feel free to merge this when you're ready.


Allow manually send event for update element width (close #1666)

This is workaround to fix placeholder width=10px if ui-select was not visible when created. It is possible now fix this by sending special event: $broadcast('updateUISelectSize').

Pull request for #1666

Hi @indvd00m ,
I don't really agree on this approach. This is a very specific fix that will help but not resolve the problem.
Can you have a look at the real problem? If so i'm happy to review. But this PR i cannot approve.


how to test model on new images

how can i modify code to input a image and get the semantics as output

Hi!
We don't provide a specific Semantic Segmentation code to obtain the semantics, we just provide the semantic segmentations extracted by PSPNet used to obtain the results from the paper.
You can use any Semantic Segmentation CNN of your choice. You just have to save the top3 labels and top3 scores leveraging the 3 channels of an RGB image.
Alex.


Highlight text with links in markdown notation

I´m trying to highlight a text that containts a markdown link in it.
For example ==Google is [quietly ramping upits AI efforts](https://www.wired.com/story/meet-bard-googles-answer-to-chatgpt/) in response to==
But what really renders is just the string itself:

is this plugin expected to work with links as well?

Thank you for report the issue. I will take care.

Hi @felipebayres,
remark-flexible-markers traces over the text nodes, and looks for == signs.
Main problem is this: The markdown link at the example breaks the connection between the leading and trailing signs (==)
==Google is [here](https://www.google.com) and always available==

Above markdown phrase consists 3 nodes of texts:
1st text --> ==Google is
2nd text (the link has a text as well) --> here
3rd text -->  and always available==
As you see none of the text nodes does not satisfy the marker format, ==text==
In order to make connection between the leading and trailing signs (==), due to the fact that there is link that cracks the connection between them, it requires too much effort for the implementation. I need community support for that.
But for now, you can try to put == signs at both sides of each text phrase like this:
==Google is ==[==here==](https://www.google.com)== and always available==

I am waiting your response.

Fixed with version 1.2.0. See https://github.com/ipikuka/remark-flexible-markers/releases/tag/v1.2.0


feat: update ITaikoL1.sol to use Taiko protocol v1.9.0

Updated ITaikoL1.sol to align with Taiko’s recent protocol release (v1.9.0). This release introduces support for proposing multiple blocks in a single L1 transaction using ITaiko.proposeBlocksV2. Note that this feature is currently not utilized by IPreconfTaskManager (pending further evaluation on whether it should be).

I realized Jeff has a PR that actually covered this change. I'll close this PR.


Is roctracer_get_timestamp is device specific?

Having multiple devices, does roctracer_get_timestamp provides a device specific timestamp? I.e., does this depends on hipSetDevice?
Additionally, is it allowed to call this from inside the roctracer_properties_t::buffer_callback_fun pool callback?

BTW, I'm not allowed to call roctracer_get_timestamp early on. Not sure, what I need to ensure that I'm allowed to call it, but its not roctrace_load. But after hipGetDeviceProperties its ok.
The error I get is:
Error Calling hsa_iterate_agents: HSA_STATUS_ERROR_NOT_INITIALIZED: An API other than hsa_init has been invoked while the reference count of the HSA runtime is zero.
Aborted (core dumped)

Its actually no surprise, as HsaRsrcFactory::Instance() does not initialize HSA, if needed. See

https://github.com/ROCm-Developer-Tools/roctracer/blob/f32619b8f109d48281bc1d684f93951e0589bd2e/src/util/hsa_rsrc_factory.h#L287

Have a work around. Close for now.

roctracer_get_timestamp returns a system monotonic clock. All timestamps returned by the tracer are in the same clock domain. You can call roctracer_get_timestamp from API callbacks or memory pool callbacks.


Using Image export on Torque maps gives an error

related https://github.com/CartoDB/cartodb/pull/2199#issuecomment-74838769

@andrewxhill: can you paste or send me the link to the vizjson, please?

http://team.cartodb.com/api/v2/viz/13062538-b129-11e4-a9f5-0e9d821ea90d/viz.json

Ok, that's related with this ticket: https://github.com/CartoDB/cartodb.js/issues/354 and another bigger issue we are talking in the email.

great thanks!
On Wed, Feb 18, 2015 at 11:50 AM, Javier Arce<EMAIL_ADDRESS>wrote:

Ok, that's related with this ticket: CartoDB/cartodb.js#354
https://github.com/CartoDB/cartodb.js/issues/354 and another bigger
issue we are talking in the email.
—
Reply to this email directly or view it on GitHub
https://github.com/CartoDB/cartodb/issues/2320#issuecomment-74899616.

--
andrewxhill http://twitter.com/andrewxhill
http://cartodb.com

I didn't notice this was a torque_heat. @rochoa, @fdansv: you were working on a fix for this, right? Is there an issue for that already?

This will be sorted here: https://github.com/CartoDB/torque/issues/131

Once we release https://github.com/CartoDB/Windshaft/issues/271 (today or tomorrow) at least it won't fail (although the heatmap won't be displayed).

This took longer than expected but we finally solved the issue and improved torque server side by using a buffer for the queries :-)

\o/

wowza
On Mon, Mar 9, 2015 at 7:20 AM, Javier Arce<EMAIL_ADDRESS>wrote:

\o/
—
Reply to this email directly or view it on GitHub
https://github.com/CartoDB/cartodb/issues/2320#issuecomment-77861449.

--
andrewxhill http://twitter.com/andrewxhill
http://cartodb.com


Active error handler is ignored

Thanks for this great utility. I hoped I could use it to fix the issue that Application.Run ignores the active error handler.
However, calling a function with this method suffers from the same problem. Please see below for a minimal reproducible example.
Any ideas if this could be made to work?
(Tested with MS Access 2019 16.0.10397.20021 64-bit)
Public Sub Test()

On Error GoTo ErrorHandler

Dim exampleModuleAccessor As Object
Set exampleModuleAccessor = GetStandardModuleAccessor("SomeModule", Application.VBE.VBProjects("SomeProject"))

' Brings up the Debug Dialog; expected behavior is to handle the error with the error handler
' (as happens if "TestCallback" is method of a real class module).
CallByName exampleModuleAccessor, "TestCallback", VbMethod

ErrorHandler:
If Err.Number <> 0 Then
	Debug.Print "Error handled."
	Err.clear
End If

End Sub

Public Function TestCallback()

Debug.Print "TestCallback called."

' Throw error
Debug.Print 1 / 0

End Function

Known limitation I'm afraid. I imagine Application.Run uses this exact same mechanism under the hood. I believe the IDispatch::Invoke of the VBComponent creates an independent debugger session. So would need to manually disable it and that would likely require some assembly instruction hacking. I think IIRC Rubberduck itself struggles in this area - if you have an anhandled error in a unit test RD cannot suppress it.
Workaround is of course to add error handling inside the TestCallback function, similar to how RD inserts automatic error handling code in the default test method template.

Sidenote:
Dim exampleModuleAccessor As Object
Set exampleModuleAccessor = GetStandardModuleAccessor("SomeModule", Application.VBE.VBProjects("SomeProject"))

' Brings up the Debug Dialog; expected behavior is to handle the error with the error handler
' (as happens if "TestCallback" is method of a real class module).
CallByName exampleModuleAccessor, "TestCallback", VbMethod

Could be
Dim exampleModuleAccessor As Object
Set exampleModuleAccessor = GetExtendedModuleAccessor("SomeModule", Application.VBE.VBProjects("SomeProject"))

' Brings up the Debug Dialog; expected behavior is to handle the error with the error handler
' (as happens if "TestCallback" is method of a real class module).
exampleModuleAccessor.TestCallback

although that won't help with the error trapping, it is just a neater syntax

Thanks for the prompt response and the suggestions! It seems that this is indeed a VBA language limitation.

I investigated the error handling issue a bit further:

Error handling doesn't work even with CallByName(obj, "fcn", ...) exactly as with calling obj.fcn: the returned Err.Description is always "Application-defined or object-defined error" (see also https://stackoverflow.com/questions/18241906/vba-callbyname-and-invokehook and http://web.archive.org/web/20090104123310/http://support.microsoft.com/kb/194418)
Error handling works (with above limitation) if standard module function is called by replacing the function address in vtable (see https://stackoverflow.com/questions/65370160/overload-com-class-methods-with-methods-from-a-standard-module-using-vtable-hack/65603584#65603584). Unfortunately this not very useful in practice because the function must have the helper class as first argument.

Thanks for the research!
Does InvokeHook enable error handling/trapping? Or is it the same as CallByName - untrappable errors but at least the correct error message.
The vtable trick is actually my post:) Thinking aloud - I don't believe just because the method is called from a class, that is what is enabling the error handling, I think vba's AddressOf operator points to a wrapper function that contains the debugger context because otherwise you couldn't put breakpoints in window subclassing procs, which IIRC you can.
Does dispcallfunc AddressOf Foo with the error raising .bas method trap errors?

I didn't test InvokeHook because it requires distributing/installing TLBINF32.DLL, which isn't possible in our case. However, I assume it works as it's the official workaround provided by Microsoft.
Error trapping from caller doesn't work with dispcallfunc (tested).
Yet another undocumented detail about CallByName: If the error number raised by the called method is in range 0-65535, it's returned correctly as is. But if error number is < 0 (as it often should be, see <IP_ADDRESS>.1.2 in MS-VBAL), it's replace by generic error number 440.

InvokeHook most likely calls ITypeInfo::Invoke under the hood - this is possible to do without TLBINF32.dll (I left a comment on that qu you linked pointing to the VBA version of this repo on codereview. In this repo's original VBA implementation, I use QueryInterface and various vtable calls to navigate the typelibs manually to avoid a dependency on TLBINF32).
However I understood from that linked microsoft article that it only fixed the Err.Number, and didn't make mention of whether it fixed the problem with Error handlers being ignored., which is what you care about?
So worth checking if TLBINF32.InvokeHook on the StandardModuleAccessor's ITypeInfo allows invocation with error handling, or even gets the error numbers right (unlike callbyname). I would be surprised if it did help since CreateStdDispatch makes IDispatch::Invoke forward calls to ITypeInfo::Invoke so I don't see these as behaving differently. But perhaps VBA uses a custom IDispatch implementation or something...

Another test would be if CallByName exampleModuleAccessor, "TestCallback", VbMethod also raises error 440 every time, indicating it is using the exact same mechanism for invocation as CallByName


Upperbound ray to 2.8

[ ] Closes #xxxx (Replace xxxx with the GitHub issue number)
[ ] Tests added and passed if fixing a bug or adding a new feature
[ ] All code checks passed.
[ ] Added type annotations to new arguments/methods/functions.
[ ] Added an entry in the latest docs/release_notes/index.md file if fixing a bug or adding a new feature.
[ ] If the changes are patches for a version, I have added the on-merge: backport to x.x.x label.

Looks like Ray 2.8.0 that was released 5 days ago is causing the autotune tests to hang (https://github.com/scverse/scvi-tools/actions/runs/6783079708). Upperbound for now until we can find the source of this.


Is there are some APIs or some methods to get the opend files.

I have noticed that the Close All  function was implemented and when open the vscode , it can restore last time's opened files. I wonder if there are some APIs or some methods where i can get the opened files in vscode .  I found the similar question in https://github.com/microsoft/vscode/issues/15178 , but an API may be more reasonable and elegant. Is there any other reasons that can't give such an API?  Hope your answer . Thanks a lot.

Closing as duplicate of that issue.
You can continue the discussino there. Thanks

You can continue the discussino there.
I 've finish reading the disscusion but i wonder if there are some progress for this problem. Thanks a lot .


remove fast_pbkdf2 dependency

This PR remove the usage of fast_pbldf2 by using latest crypto API in Erlang. If needed i will add some legacy support for fast_pbkdf2. Or should it be an option?
I did remove this library because any libray based on openssl is hard to to deploy in multiple environnement, and I don't think  the performance s lower with this change though I didn't measure it precisely first.

Amazing idea, I wanted to get to try this for a while, thank you so much for the contribution!
Nevertheless, I'll get picky with performance: it is indeed pretty much in all cases 1.85x slower.
I tried benchee with these inputs:
Benchee.run(
 %{
   "fast" => fn {hash, pass, salt, it} -> :fast_pbkdf2.pbkdf2(hash, pass, salt, it) end,
   "otp" => fn {hash, pass, salt, it} ->
                %{ size: len } = :crypto.hash_info(hash)
                :crypto.pbkdf2_hmac(hash, pass, salt, it, len)
            end
 },
 inputs: %{
   "sha256-12password-1024it" => {:sha256,
                :base64.encode(:crypto.strong_rand_bytes(8)),
                :base64.encode(:crypto.strong_rand_bytes(8)),
                1024},
   "sha256-12password-10000it" => {:sha256,
                :base64.encode(:crypto.strong_rand_bytes(8)),
                :base64.encode(:crypto.strong_rand_bytes(8)),
                10000},
   "sha256-12password-500000it" => {:sha256,
                :base64.encode(:crypto.strong_rand_bytes(8)),
                :base64.encode(:crypto.strong_rand_bytes(8)),
                500000},

   "sha256-32password-1024it" => {:sha256,
                :base64.encode(:crypto.strong_rand_bytes(22)),
                :base64.encode(:crypto.strong_rand_bytes(8)),
                1024},
   "sha256-32password-10000it" => {:sha256,
                :base64.encode(:crypto.strong_rand_bytes(22)),
                :base64.encode(:crypto.strong_rand_bytes(8)),
                10000},
   "sha256-32password-500000it" => {:sha256,
                :base64.encode(:crypto.strong_rand_bytes(22)),
                :base64.encode(:crypto.strong_rand_bytes(8)),
                500000}
 },
 memory_time: 5
)

All of them memory-wise take 72B for fast_pbkdf2 and 328B for libcrypto, so memory being a constant I don't mind at all not even half of a kilobyte. But speed-wise it's an interesting difference. To me that 1.85x is quite close to 2x, with makes me suspicious of libcrypto's implentation of PBKDF. Usually the complexity of the algorithm grows linearly with the iteration count, so complexity is O(n) but under a naive implementation the complexity on the number of compression functions is 4n, which in a proper implementation is 2 + 2i. So while both are O(n), the 2x constant makes a difference.
My implementation surely does the proper (that's the whole point why I did it :D), so that 1.85x diff makes me think libcrypto got something unoptimised there 😅
Even more interesting, when setting parallelism to 12 (the number of cores in my machine), libcrypto's one is then 1.35x slower, so good for us. But more interestingly, when setting parallelism to 60 (5x number of cores), libcrypto's one then varies a lot for the iteration count: 1024 is as before 1.85x slower, 10000 is 1.30x slower for the small password and 2.3x slower for the big password, and 500000 (0.5M) is 4.3x slower for both password sizes. So it makes me think it's using dirty schedulers and blocking operations, my hypothesis.
Still, I agree simplicity of dependencies is a great win, so I wouldn't say no even for a 2x factor. But at such factor I'd like to have it possible to choose. After all, half a million iterations is not . Do you know think we can make this configurable? 🙂

@benoitc any news on this contribution?
I also saw that CI fails for older OTP versions but I'm totally in favour of pushing for newest ones anyway.
Let me know if I can help you with this 🙂

Closed in favour of #13


Limit task count to 1 for ECS provenance cluster

Research options to limit task count to max 1 for ECS provenance cluster. The requirement is to not kick off another task until the existing task completes.
If a task runs for too long (goes over a certain time period), it should through an error and terminate.
Investigate possible solutions :

Capacity provider for ECS cluster using AutoScaling groups (will need SA support)
Update provenance script to cap # of documents per task

Sagar reports: focusing on this right now

@tloubrieu-jpl - I do not have access to create an ASG so I've opened a ticket with SA team.

@tloubrieu-jpl @jordanpadams as discussed today, we might no longer need this ticket (or we can re-purpose it) since we don't need the ASG and instead we're going to use the lock-file approach with Lambda that SA team is working on using a separate ticket.
https://itsd-jira.jpl.nasa.gov/servicedesk/customer/portal/16/DSIO-4247

@tloubrieu-jpl  is this where we're tracking lambda creation for ECS tasks? Or should that be tracked in #30 ?

I don't know @sjoshi-jpl , maybe the lambda would make more sense in this ticket since it is orchestrating the provenance processes and guarantee that only one is running on each domain at once and especially that not a new porcess is launched if one already runs.
The monitoring is more an after the fact, metrics measurements.

Per latest update from Susan, they're working on creating the lambda as well the registry-sweeper cluster.

#60 New ticket created for tracking this task. @tloubrieu-jpl we can close this one.


Update BaseHandler.php

Problem:
The isPdf function uses a string comparison of the file extension but it is not case insensitive so will fail when the extension is not lowercase:
var_dump(isPdf('pdf')); // TRUE
var_dump(isPdf('PDF')); // FALSE
var_dump(isPdf('pDf')); // FALSE
var_dump(isPdf('pdF')); // FALSE
var_dump(isPdf('PdF')); // FALSE

Proposed solution:
Using the strcasecmp function to compare the file extensions in a case insensitive mode:
var_dump(isPdf('pdf')); // TRUE
var_dump(isPdf('PDF')); // TRUE
var_dump(isPdf('pDf')); // TRUE
var_dump(isPdf('pdF')); // TRUE
var_dump(isPdf('PdF')); // TRUE

Thank you!

Hi Jerry,No need to thank me. This is a great library and I’ve been using it for quite some time.Please let me know if you need help with some of the work. I don’t have much time but I’d set some aside for this as I’d love to contribute more.Kind regards,AlexOn 31 Jul 2024, at 17:07, Jerry Chen @.***> wrote:
Thank you!
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>


Automation of the Bluesky account(s)

Converted from https://github.com/nodejs/admin/issues/932 - Bluesky has a free API and an Node.js SDK which I think we can explore, it would be nice for the community to curate content and submit them here via GitHub PR, reviewed here, then get published to the official Bluesky account via automation.
I asked a while back on Bluesky about what automations other projects use, it seems the most popular one is https://github.com/myConsciousness/bluesky-post - however on a closer look it seems only the post function would be useful for our use case, but the Node.js SDK of AT protocol provides handy functionalities like repost/quote post/reply/like etc. and it only takes a couple of lines to do it. For example I have some toy snippets here where you can see how simple it is to do things https://github.com/joyeecheung/bluesky-playground/ - so I wonder if we should just maintain our own GitHub actions here to automate actions on the official Bluesky account.
We could also consider publish our actions to the GitHub marketplace when we have something that works well, and share it with other projects that want to automate their official accounts similarly. I think it would be lovely if the Node.js project can contribute some stuff back to the Bluesky ecosystem (which runs on Node.js, I love how meta it is).

why not having a posts dir in this repo with md file that will automatically make a post. It reduces the foundation's workload.
It could also allow contributors to pass on announcements more easily, such as new content in the learn section. Or a post to discover a new feature (post separate from the release).

@AugustinMauroy I think that's included in the OP already :)

Some ideas besides post/repost/quote post/reply:

I see people have started at-mentioning nodejs.org. We should ideally have some automation to pipe the notifications to some slack channel so that those don't go unnoticed (maybe pipe DMs to a private channel, too?)
When new collaborators join we can have some automation to add them to a starter pack.

I created an app password for the nodejs.org account and saved it to the TSC 1Password. Will add it to this repo later for future GitHub actions when I figure out the right place to save it.
Meanwhile, I worked out some snippets to perform post, repost, quote post and reply. I haven't done anything for proper embedding (thumbnails, images, etc.) yet. But I think they are probably fine to be sent over to this repository now. Will do that later.
https://github.com/joyeecheung/bluesky-playground/tree/main

Managed to develop something: see https://bsky.app/profile/pixel-voyager.bsky.social which is sending post/reply/quote post/reposts from PRs merged in https://github.com/joyeecheung/bluesky-playground/pulls?q=is%3Apr+is%3Aclosed - there are currently no docs, no tests, but I can add some as I PR them in here..

I see people have started at-mentioning nodejs.org. We should ideally have some automation to pipe the notifications to some slack channel so that those don't go unnoticed

Maybe, but I'm afraid this will be extremely noisy. Many people at-mention the account instead of writing "Node.js", not to really to get our attention

maybe pipe DMs to a private channel, too?

Currently, nobody can DM the account

Many people at-mention the account instead of writing "Node.js", not to really to get our attention

We might want to document how we prefer people engaging with the Node.js account somewhere. Not sure if people would necessarily see it but it might help.

I haven't read through https://github.com/bluesky-social/feed-generator yet but my hunch is that we can just implement a simple algorithm to generate a feed that include all the at-mentions of nodejs.org, and whoever interested in adding that feed to their bluesky tabs can just do so.

I managed to get https://github.com/joyeecheung/bluesky-playground to a working shape - for example, I triggered https://bsky.app/profile/pixel-voyager.bsky.social/post/3lbnkx3si652h with https://github.com/joyeecheung/bluesky-playground/pull/8 - there's a GitHub action that can validate the JSON before the PR is merged, and another GitHub action that performs the requested action on the Bluesky account once it's merged.
Currently, this supports posting, reposting, quote-posting and replying (you can find the JSON examples in https://github.com/joyeecheung/bluesky-playground/tree/main/records/new).
Obviously the code is a bit messy and it has almost no tests (just one integration test that I run locally and see if https://bsky.app/profile/pixel-voyager.bsky.social does what I want it to do). But for what we currently need for the nodejs.org account I think it's adequate for now (or I can say, trust me bro/sis, it works). I propose that we add the scripts and workflows in here (with license etc. changed) and start using it. And we can work on better linting/testing later.
(My longer-term vision is to publish that action on the GitHub actions market so that other people who want to collaboratively control a Bluesky account can share it and maybe contribute back as well. But baby steps, let's just start using it now before it becomes painful to have to manually logging into the account to do everything, and think about the rest later).


Update create application script

This script has succumed to a bit of bit-rot, and no longer works. Now the API clears the database on each restart locally, it’s useful to have a script to generate an in-progress application. This uses the application data in the fixtures, updates the release date to be a date in the future and adds some questions about the tier to make a valid application.
The script used to create an assessment too, but I feel that generating an in-progress application is enough for us to do local testing.

Yeah, good plan. I'll do that 👍


JIT graph building

Here's a motivating example for what I'm referring to:
import dask.array as da
import time

zero = da.zeros((10000,5000,5000,5), dtype='float32', chunks=(1000,500,25,1))
data = da.overlap.overlap(zero, depth={0:10, 1:2, 2:2}, boundary=-1)

# Compiling the graph takes ~43 seconds for ~4.3 million tasks
start = time.time()
tasks = len(data.dask)
print(f'{time.time()-start:.2f} seconds to compile graph')
print(f'Graph contains {tasks:,} tasks')

# Computing the first block takes ~3 seconds since the graph is now compiled
start = time.time()
data.blocks[0,0,0,0].compute()
print(f'{time.time()-start:.2f} seconds to compute the first block\n')

# Create a fresh data graph
data = da.overlap.overlap(zero, depth={0:10, 1:2, 2:2}, boundary=-1)

# Computing the first block immediately takes ~43 seconds (as it will
# interally compile the whole graph, thus taking around the same time)
start = time.time()
data.blocks[0,0,0,0].compute()
print(f'{time.time()-start:.2f} seconds to compute the first block')

# Computing any other blocks is quick, now that the graph is compiled
start = time.time()
data.blocks[1,0,0,0].compute()
print(f'{time.time()-start:.2f} seconds to compute a different block')

# Graph is already compiled, so checking total tasks is now immediate
start = time.time()
tasks = len(data.dask)
print(f'{time.time()-start:.2f} seconds to compile graph')
print(f'Graph contains {tasks:,} tasks')

In the actual data I'm working with, the graph takes a significant amount of time to create before the first block can even start work.
In this case it's due to the rather large amount of operations that need to take place in dask.layers.ArrayOverlapLayer, but the general idea I'm discussing is: how feasible would it be to allow just-in-time graph compilation, in order to remove startup time dependency on the size of the data / overall number of tasks? Is it possible to allow compiling only the exact task dependency graph for a given object when it's computed, rather than having to build all tasks at the outset?

Thanks for your report.
Before I get into this, what you call "compilation", i.e. the process of building the "low level graph" / initializing all tasks, is what we typically refer to as "materialization".
With this out of the way, let's look at your data. Calling data.dask shows the representation of your graph up to this point, layer by layer (roughly every expression that is applied to your collection is represented as a layer).
You'll notice the attribute is_materialized which indicates whether the graph was already fully built since this is actually done layer by layer. You'll also notice the white vs grey circles.

There are a couple of layer types in here that are already materialized. Looking through the code base it appears that for those operations (primarily concatenate and overlap) the "non-materialized" representation of the graph is simply not implemented yet.
If this is implemented properly, we would indeed just compute whatever is necessary to calculate the desired output key as you are describing.

Apart from the materialization issues, your example is generating a graph of about 4MM tasks. This is larger than what we are typically dealing with and especially if you are adding more layers to this graph, this could quickly grow out of hand. If there is any opportunity to rechunk / use larger chunks, this could help significantly. IIUC your input dataset has chunks of about 50MiB. You may be able to go a little higher here
zero = da.zeros((10000,5000,5000,5), dtype='float32', chunks=(1000, 500, 25, 1))
# If the input data is the way it is, you can still rechunk later
# Mind that the new chunks are a multiple of the input, this way the operation is trivial
zero = zero.rechunk((2000, 1000, 25, 1))
# Adjust the overlap as necessary
data = da.overlap.overlap(zero, depth={0:5, 1:2, 2:2}, boundary=-1)

Thanks for the reply!
So to reiterate what you're saying, the materialization does in fact already happen just-in-time, it's only because the non-materialization representation isn't implemented for concatenate/overlap that the current behavior requires those layers to be fully materialized in order to compute any output keys. If/when a non-materialization representation is available for those layers, it should be the case that in the example above, the first block can be computed without actually materializing the other (unnecessary for that block) parts of the low level graph. Did I understand that correctly?
For example, this specific area is where that non-materializing representation would be implemented? And this pull request is actually addressing that, except that it hasn't been merged yet. I can bump that pull request, since it seems like it may have slipped through the cracks.
So I guess that answers my main question, thanks! One thing from your reply that I'm unsure of, though: having something like 4MM (or even 400MM) tasks in a graph should only be a problem for the scheduler if you're actually trying to compute that entire graph, right?
If instead, we're only doing a small segment of the graph at a time (like in the example above, computing one block at a time) - and assuming the graph has a full non-materialization representation - there really shouldn't be any bottleneck on how large the overall graph can be. The bottleneck is purely at the number of tasks specifically required for the requested output key. Is that correct?

layers, it should be the case that in the example above, the first block can be computed without actually materializing the other (unnecessary for that block) parts of the low level graph. Did I understand that correctly?

Yes, we call this culling. We'll basically take the output key you requested and walk our way backwards building the graph. This way, you'll only build and compute what you actually require for your result.

For example, this specific area is where that non-materializing representation would be implemented? And https://github.com/dask/dask/pull/8476 is actually addressing that, except that it hasn't been merged yet. I can bump that pull request, since it seems like it may have slipped through the cracks.

I'm not familiar with this yet but from a brief glance, yes.
For concatenate, I believe it is simply not implemented, yet. One would need to implement a ConcatenateLayer (It looks like this is old code that is generating the low level graph eagerly. I don't know how difficult it is to move this into a layer.)

For the sake of transparency, this HighLevelGraph / Layer is an abstraction we're not entirely satisfied with. For DataFrames, there is a rewrite/experiment happening over in https://github.com/dask-contrib/dask-expr. This does not support arrays yet but it is our intention to add this to it. This will not help you now, I'm just mentioning this in case you are interested.

Unfortunately my application needs to work with arrays rather than DataFrames, but thanks for mentioning it - I'll definitely keep my eye on the progress.
With that said, I am somewhat invested in getting a faster subgraph computation like in the example above, and so I'm trying to dig into exactly why this example needs to take so long to compute the first block. In the reference you provided above to culling, one of the first lines of that function is to call get_all_external_keys, which is what triggers the materialization of the overlap layer.
The problem with that is, when there are millions of external keys for a layer (which seems to commonly be the case for an overlap layer), this takes significantly more time and memory to create and work with than any single key dependency path actually requires (regardless of if the layer is actually materialized, if I understand things correctly).
I'm having some difficulty understanding exactly why an individual layer needs the entire graph's external keys in order to cull its own dependencies, however. When I trace back the usage of the full external key set, it appears as though each layer is just performing an intersection between its own dependencies and the full graph keys (here).
I've made some modifications to dask that allow the above example to completely run in < 1 second (essentially removing the time dependency on the size of the graph/data). This requires removing the calculation of all external keys that I mentioned above, however, which does work for this example - and I think theoretically should never be required for any other situation - but I'm not familiar enough with dask internals to say for sure. Do you (or anyone else) know why the calculation of all external keys is necessary?

@rjzamora maybe you got some insights here?

In the reference you provided above to culling, one of the first lines of that function is to call get_all_external_keys, which is what triggers the materialization of the overlap layer...  I'm having some difficulty understanding exactly why an individual layer needs the entire graph's external keys in order to cull its own dependencies

You have a good eye @BrandonSmithJ - The call to get_all_external_keys was meant to be a "temporary" thing when we first started shipping HighLevelGraph objects from client to scheduler. Before that, we were always culling a fully-materialized graph anyway, so the optimization was written in a way that assumed you could always check if a particular string or tuple was also a key in the graph.  The "real" problem came when this assumption began to leak into critical areas of the HLG serialization code.  For example, while "stringifying" both input and output keys during the serialization of a specific Layer, we would often use the set of all known external keys to tell us which tuple objects are actually keys.
You are absolutely correct that there is no fundamental reason that we need get_all_external_keys. In fact, now that the graph is usually not fully materialized when culling is performed, it is quite silly that we are still doing this. After materialization, the scheduler does need a full dependency graph for all keys in the optimized graph, but this certainly shouldn't require us to know all possible keys before optimization.
Both myself and others have proposed changes to this in the past (e.g. https://github.com/dask/dask/issues/9216). However, slightly more-visible problems, like HighLevelGraph serialization and key "stringification", have always tended to block/delay its prioritization.  The good news is that these other priorities are mostly out of the way now, so it is probably a good time to discuss a path forward (even if HighLevelGraph design is ultimately replaced with an expression system).

Thanks @rjzamora! The historical usage of it makes sense, and I think that does solidify my understanding on things.
I cleaned up my patch a bit and created a pull request (#10534) that addresses this issue, and seems to pass all tests (locally, at least). So it appears this might offer a generalized significant improvement in speed when a user is only computing parts of the overall graph.
Happy to discuss this further - whether it's any other small modifications that should be made, or if the change can't be incorporated for whatever reason.


Handling querypb.BindVariable better for lists.

Handling list bind variables type *querypb.BindVariable with Type TUPLE, in vtgate and vttablet.

@sougou I am not sure the vttablet side is correct, there is a FIXME comment in there, input appreciated. Thanks!

@sougou PTAL I think this is what needs to be done, thanks for the feedback!

LGTM


Fixed a JSDoc-related crash when obtaining type of a type literal type argument

fixes the crash reported here https://github.com/microsoft/TypeScript/issues/60355#issuecomment-2440205019

Thank you!


Fetching posts via a custom key

Hi,
I assume the answer is no, but is it possible to fetch a post in a collection via something other than the slug?
For example each post in my collection has a field “foo_id” which is a custom field type. The custom field type is pulling data from an external system so we can link posts together. I want to then fetch the posts that match a given “foo_id”.
If it’s not possible would you be open to a PR to allow this functionality?
Thanks

Hey @joshhornby, sorry for the late reply. You are correct, this isn't currently possible.
Custom filtering has been on the roadmap for a little while now, but in anticipation for Statamic v3, I've stopped dev as there is a high chance that the addon will need to be rebuilt anyway.
If you require the feature anyway, feel free to submit a PR -- if the code is easy to follow and I can test that there are no breaking changes then I may be able to merge your work. Worst case, you'll have to use your own forked version which is probably fine too!


Updated italian README.md file

[x ] I have read freeCodeCamp's contribution guidelines.
[ x] My pull request has a descriptive title (not a vague title like Update index.md)
[ x] My pull request targets the master branch of freeCodeCamp.
[ x] None of my changes are plagiarized from another source without proper attribution.
[ x] My article does not contain shortened URLs or affiliate links.

Fixes #18312

language- Italian

Closed and Reopened this PR to attempt to resolve a specific Travis build failure.


pointer support

currently, pointer field is ignored

always set zero value is not good? (e.g. '*string -> &("")

This is correct behavior isn't it?

currently, pointer field is ignored

if tag is existed, handling even pointer field

spec

if tag is not found,  unexported field, skipped
if tag is found,  unexported field, not skipped
if tag is not found,  pointer field, skipped
if tag is found, pointer field, not skipped

with json
and with json, override by flag tag.
type S struct {
  NotSkipped *string `json:"not-skipped"`
  Skipped *string `json:"skipped" flag:"-"`
}

// b.FlagnameTags = append(b.FlagnameTags, "json")


Cannot delete a job

At least one job has gotten into a state where it cannot be deleted. I've tried restarting the beanstalkd server but it doesn't seem to free up. Also, notice how the releases count is 0 for the job.
Server stats:
stats
OK 972
---
current-jobs-urgent: 0
current-jobs-ready: 2
current-jobs-reserved: 1
current-jobs-delayed: 6690
current-jobs-buried: 1480463
cmd-put: 500348
cmd-peek: 1
cmd-peek-ready: 0
cmd-peek-delayed: 0
cmd-peek-buried: 0
cmd-reserve: 0
cmd-reserve-with-timeout: 546523
cmd-delete: 500378
cmd-release: 74
cmd-use: 821
cmd-watch: 393
cmd-ignore: 393
cmd-bury: 3
cmd-kick: 0
cmd-touch: 0
cmd-stats: 30
cmd-stats-job: 462
cmd-stats-tube: 73
cmd-list-tubes: 1
cmd-list-tube-used: 0
cmd-list-tubes-watched: 12
cmd-pause-tube: 0
job-timeouts: 4
total-jobs: 500348
max-job-size: 65535
current-tubes: 17
current-connections: 425
current-producers: 71
current-workers: 197
current-waiting: 142
total-connections: 1101
pid: 11708
version: 1.9
rusage-utime: 36.818301
rusage-stime: 143.336958
uptime: 721
binlog-oldest-index: 433964
binlog-current-index: 434066
binlog-records-migrated: 711683
binlog-records-written: 1712481
binlog-max-size: 10485760
id: cf281b94ce75771f
hostname: queue3

Job stats:
stats-job<PHONE_NUMBER>
OK 196
---
id:<PHONE_NUMBER>
tube: raw_item_indexer
state: reserved
pri:<PHONE_NUMBER>
age: 3138203
delay: 0
ttr: 120
time-left: 103
file: 434013
reserves: 26135
timeouts: 26100
releases: 0
buries: 0
kicks: 0

Job data:
peek<PHONE_NUMBER>
FOUND<PHONE_NUMBER> 113
{"item_id": 291725218, "occurrences": 7, "event": "repeat_item", "new_occurrences": 8, "raw_item_id":<PHONE_NUMBER>}

Attempt to delete the job:
delete<PHONE_NUMBER>
NOT_FOUND

I've found that the job needs to be in ready state to be deletable.
kick 10
delete $jobId

I've not found anything else that reliably deletes the job.

Personally I gave up on using Beanstalkd.  Not enough documentation, and too hard to troubleshoot (case and point: this issue).  RabbitMQ, Resque, and Gearman all seem like better alternatives.

@kr Any chance of addressing this?
I think @coryvirok's NOT_FOUND may be explained by protocol.txt. It states that the response to a delete command will be "NOT_FOUND\r\n" if the job does not exist or is not either reserved by the client, ready, or buried. This could happen if the job timed out before the client sent the delete command. In @coryvirok's case it appears that he hasn't reserved the job with the client (and it's still reserved).

@jtreminio: That makes sense if the client you're issuing the delete from hasn't reserved the job. protocol.txt says:

A client can delete jobs that it has reserved, ready jobs, delayed jobs, and jobs that are buried.

I've experienced with the same problem. The reason is the ttr, as @contentfree said. Take a example, worker A reserved a job X with ttr 120s. The job X will cost A for 300 seconds, but at 120s, X was expired and reserved by another worker B. Then when A released the job X, it will throw a NOT FOUND exception for A don't acquire X anymore.

error message:No job by the id of None was found
hey guys, resolved it?

Not a bug, but could IMHO maybe give a nicer error message if possible.

We have just experienced the same problem, and the reason was the number of jobs in the queue: 65 536.
Jobs were generated with TTR 3600, but when these jobs were over the limit 65 535, their TTR was lowered to 60! ( visible via https://github.com/schickling/dockerfiles/tree/master/beanstalkd-console ).
We had to lower number of generated jobs.

@MartinMikita Would you mind creating a separate issue for this? Also, would it be possible for you to write a small script to recreate this?

Is there any reason you cannot delete a delayed job? You can't even seem to be able to update it once delayed either. IE you can't reduce the delayed time once saved.

@m4tthumphrey Please file a new issue. Your comment is not related to this issue.

I could not replicate anything mentioned in the comments. Also I see that beanstalkd 1.9 was used by @charukiewicz.
Please comment if you can reproduce the problem.


Adding cipher standalone app

Signed-off-by: pegartillo95<EMAIL_ADDRESS>Signed-off-by: kubasiemion<EMAIL_ADDRESS>Signed-off-by: gbarcomu<EMAIL_ADDRESS>

Could you please divide this into smaller commits and add description in the commit?


serializers defined in views inspite of having separate serailizers.py files

Describe the bug
In link_views.py and list_views.py, serializers _LinkCreatedResponse and _CreateResponse have been defined respectively. These two files can be refactored in such a way that the serializers are defined in their appropriate serializer file
Additional context
Would like to work on this issue. Can CC team approve and assign this issue to me?

@aldenstpage

Those serializers are response stubs – they model responses for the documentation, which are generated by views. The decision to include them in the views was a conscious one.


[WFLY-16841] Use org.glassfish:jakarta.json explicitely on wildfly-me…

…ssaging-activemq-subsystem-jakarta to avoid getting it transitively from wildfly-cli
Jira issue: https://issues.redhat.com/browse/WFLY-16841

Thanks @yersan


No documentation.

Add documentation!

I 100% agree there's more we can do on the documentation front -- especially on how all the packages fit together -- but have you seen the documentation page for Tidier.jl and for the underlying base packages?

Tidier.jl: https://tidierorg.github.io/Tidier.jl/dev/

TidierData.jl: https://tidierorg.github.io/TidierData.jl/latest/

And so on...

I'm going to close this issue for now. We will continue to think of ways to make the documentation more explanatory/cohesive/centralized. If you have suggestions, feel free to reply to this thread.


enable allow any for outbound traffic demo profile

for demo profile only, per discussion in https://github.com/istio/istio/issues/11739

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: linsun
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

install/OWNERS [linsun]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@costinm since we are documenting all breaking changes from 1.0->1.1, should we introduce this change for the regular values.yaml too?

@linsun: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e_pilotv2_auth_sds.sh
641d9d41d0ca2780483421b494dba6317551dd90
link
/test istio_auth_sds_e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@linsun: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e_pilotv2_auth_sds.sh
641d9d41d0ca2780483421b494dba6317551dd90
link
/test istio_auth_sds_e2e

prow/istio-integ-k8s-tests.sh
641d9d41d0ca2780483421b494dba6317551dd90
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/lgtm

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: linsun, ymesika
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

install/OWNERS [linsun,ymesika]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

cc @wenchenglu


Mammoth Cyber

Do you want to add or update the vendor? Adjust the title accordingly!
How to use? Fill the yaml file below!

name: Use the vendors name
free_sso_providers: Do they support free SSO providers like Google, Apple, Github etc.? One per line.
scim_pricing: Use the vendors cheapest price that includes SCIM.
scim_starting_on_plan: When is SCIM available? lowest or highest
sources: Add a link or a note to where you got information like pricing from. One per line.
sso_on_free_plans: Is SSO available on free plans?
sso_on_all_paid_plans: Is SSO available on all paid plans? It must be true to appear on this list.
updated_at: yyyy-mm-dd of the day you filed this.
vendor_url: Use the vendors public website

---
name: Mammoth Cyber
free_sso_providers:
scim_pricing: 0 included
scim_starting_on_plan: all paid plans
sources:
  -https://www.sjultra.com/services/mammoth-cyber-zero-trust-network-access-and-enterprise-access-browser/

sso_on_free_plans: false
sso_on_all_paid_plans: true
updated_at: 2024-09-04
vendor_url: https://mammothcyber.com/

Hey @richardr35 , I was trying to find any pricing/sso/scim information on your linked source page but could not. Do you have details about it?

Their pricing is done though resellers and requires a quote based on size and use. However I know that working with them, SSO and SCIM are included in every plan. There is no extra charge for that. But I can't really get you pricing as it is pretty varied based on how each company will use it.


✨ Add player parameters for youtubeLite shortcode

https://github.com/paulirish/lite-youtube-embed?tab=readme-ov-file#custom-player-parameters

Hello @d4rklynk PR looks good, it's missing the documentaiton update. Can you please add to this PR?

Done, let me know what you think

@d4rklynk looks better - could you add the detailed list of parameters in the docs or surface them as individual parameters at the top level?

Do you mean explaining all the parameters described in the Google docs or just list them?

At least point people to where to find the parameters (sorry if it's already there and I missed it) if we keep the params var.

Indeed, I set it to the last paragraph.
I can rephrase though if you want.

Nothing need thanks. Sorry for missing it.


Universal listings: Add 'pills' to show active filters

Display the currently active filters, along with links to remove them individually.

Marking as draft for now, since we probably don't want to merge this until we have the 'pill' styling in place.

Merged in #11452


Os 130:  DB Sharding Registry module.

Indroducing shards:
Create a factory for DatabaseProvider. Based on provider constant the database provider instantiated.
Added models for holding the shards info as given in application yaml. Initialize the shards info components.
Added a default shard (as shard1) in application configutation.
Tested by postman

Closed PR- wrong raised


Response schema validation

Validating the first level of keys of the response according to this https://github.com/joanvila/raml-js-validator/issues/9

Coverage increased (+0.7%) to 95.181% when pulling 729db040f19ef464aae9815136a66be6cee63372 on response_schema_validation into 386f7ac47ed5ede94b4fe8ba0da8b1e6325824ea on master.


Enhancement: Ability to invoke generic methods with explicit type parameters when PowerShell cannot figure out  from the context

Please provide ability to invoke generic methods with explicit type parameters.
Steps to demonstrate
[Array]::Empty[string]()

or using a class:
class A {
    static [string] GetTypeName[T]() { return [T].Name }
}

[A]::GetTypeName[string]()

(Array.Empty was chosen as a simplest possible example. In scripts Iʼd use @())
Current behavior
At line:1 char:16
+ [Array]::Empty[string]()
+                ~
Array index expression is missing or not valid.
At line:1 char:16
+ [Array]::Empty[string]()
+                ~~~~~~~
Unexpected token 'string]' in expression or statement.
At line:1 char:24
+ [Array]::Empty[string]()
+                        ~
An expression was expected after '('.
    + CategoryInfo          : ParserError: (:) [], ParentContainsErrorRecordException
    + FullyQualifiedErrorId : MissingArrayIndexExpression

Ruminations
I understand that itʼs not important to have full support of generics in a scripting language or to have 100% feature parity with C#  (I might be wrong, — now we have classes in PowerShell after all :thinking:)
Here are some not-very-elegant workarounds from the Interwebs:

Workaround to call to a generic method OfType() in PowerShell, when PowerShell cannot figure out  from the context by @vors
PSGenericMethods module by @dlwyatt
Invoking Generic Methods on Non-Generic Classes in PowerShell by @LeeHolmes
How do I call a parameterless generic method from Powershell v3?

Here are my questions:

Is it possible to implement this at all? How hard it could be?
Anyone tried to implement this before?
Is it worthwhile?

Just posting this here for discussion. Didnʼt find any mentions of this problem in other issues.

At first glance, this is not feasible because the main asset of PowerShell is to hide complex details - Powershell automatically calls the correct methods.
As for implementation it seems is a lot of work but not so difficult.

@iSazonov Problem is that PowerShell can not automatically call correct methods when there is no way to deduce generic type parameter automatically (as seen in "Steps to demonstrate"). It must be specified explicitly. This is true for other CLR languages too.
We are forced to dig through all the complex details using reflection (see list of workarounds).

I mean that the implementation of the general case is likely to be redundant. On the other hand, if there are specific popular scenarios, maybe we could implement them natively in PowerShell.
We also have to take into account that PowerShell performs many implicit type conversions - we may encounter insurmountable hurdles.

This is totally a reasonable thing to add, and a perfect example of why I'm excited PowerShell is open source.
People do want this feature, but as you point out, it's not critical for a shell. With limited bandwidth, this feature was never quite important enough for a small team to implement, but that's not an issue anymore - there are plenty of motivated and capable people who might add this feature now.
For anyone willing to tackle this:

I think the proposed (and obvious) syntax should work, but you must be careful to not break array references, e.g.

$t = [System.Collections.Generic.Dictionary[string,int]]
$t.GenericTypeArguments[0]

When you first see the [, you can't know if you are expecting an expression or a type argument list.

We will need a new property in MemberAst. It might not be obvious why here and not InvokeMemberAst, but sometimes you refer to a method without arguments so you can call it later, e.g.

$op = "hello".PadLeft
$op.Invoke(15)

It's probably not important to support the equivalent for generic methods in the initial implementation, but it is important that the representation support it should we want that in the future.

This type probably needs a new property to hold the specified type arguments, which would get added from the Ast here.

Hopefully this is enough to get someone started, if more help is needed, just ask.

I'll take a look when I have some time.  It'll certainly be easier than writing PSGenericMethods was, since I can just use the type conversion / binding code / overload identification that's already there instead of having to reinvent it.

@lzybkr , how would you feel about this syntax for the method reference option:
$op = "Hello".SomeGenericMethod
$op.InvokeGeneric([someType], $arg1, $arg2)

This makes it easier to see what's happening at parse-time.  If we modify MemberExpressionAst instead (allowing for $op = "Hello".SomeGenericMethod[someType]), it's tricker to distinguish between a property followed by an array index versus a method with a generic type.  By looking for [type](args), it's not ambiguous (parser currently will complain about the open parenthesis).

Because a pipeline isn't allowed as the index expression, I don't think there is any ambiguity - one token of look ahead is sufficient - if could be a type argument list, it is, otherwise it must be an expression.
If a pipeline was allowed, a bare word is ambiguous - is it a type or an command invocation.
Note that one token look ahead doesn't mean there is only one token to look for, there is at least a bare word and an '[', e.g.
[Array]::Empty[string]()
[Array]::Empty[[string]]()

The extra square brace is there for a corner case where you want to specify the assembly name along with multiple type arguments, e.g.
[System.Collections.Generic.Dictionary[[string, mscorlib], string]]

Hmm... playing devil's advocate for a second, do we want to support variables in the type expression?
$something.MethodName[$typeInAVariable]()

If so, then this could be ambiguous:
$something.MethodName[$typeInAVariable] # Array index or method with generic type params?

That seemed useful in V2, but it's been that long since I've seriously considered it.
And honestly, if we wanted to support that, I'd look for a different syntax for explicit type parameters.

ok, cool.  I'll be on a plane for 4-5 hours tomorrow morning; if I'm not feeling too groggy (waking up at 4am, bleh), I'll plug away at it a bit.

I might be biting off more than I can chew on this one.  I've tweaked the parser and AST to support the syntax, but I'm having trouble wrapping my head around the compiler and binders to see where to make use of the new information from the AST.  Will sleep on it.

@lzybkr
Can I pick your brain a bit about this one?  Updating the parser was fairly straightforward, and when I look at the Adapter / DotNetAdapter / PSMethod classes, it's very similar to what I wrote in PSGenericMethods (easy to add some new overloads that accept a list of generic type arguments).
Between those, however, I'm getting lost in the Binders / DLR expression tree stuff, trying to figure out the best way to get the new information from the AST over to either DotNetAdapter or PSMethod.  Should there be new fields on binder class(es)?  Do we need to uniquely identify a binder based on the generic type arguments, or just use one binder instance per method / property name (the way it works now) and somehow pass on the generic type arguments so eventually PSMethod.Invoke can identify the right overload?
While looking into this, I came across another possiblity for syntax:
# Imaginary generic method that takes one T and two other arguments
$psmethod = [SomeClass]::SomeMethod
$psmethod.InvokeGeneric(@([string]), 'Argument 1', 'Argument 2')

(Names don't matter here, just saying that we'll probably be adding new input to PSMethod's Invoke* method(s) anyway, and may not need to support the $method = [SomeClass]::SomeMethod[string] syntax as a result.  $emptyStringArray = [array]::Empty[string]() could still be a convenient shortcut for calling it, though. )

Look at PSMethodInvocationConstraints - I think that's the right place - just be sure to update the hashing so you don't reuse the same binder in different dynamic sites.
And FYI - I'll be slow to respond over the next week.

Was there any change? Or maybe some better option rather than using "MakeGenericMethods"? Generics are widely used in .net assemblies that deal with data, like entity framework or ML.NET, where classes are used to defined data schema. It's extremely tedious to port such methods to PS. I think it would be great to have <> operator  in PS too.

Some allowances for casting operations would be nice as well. For example, if you're casting to [List[int]], the following does not currently work without an intermediate cast:
using namespace System.Collections.Generic

[List[int]]@(1..10)

Currently you must have an intermediary cast of [int[]] before that will go through. It would be nice if we could have the parser / conversion methods check if the target type is, e.g., IEnumerable<T> (where T is the same generic type parameter given to the target type we're casting to), and then automatically attempt to first convert anything that is [object[]] to [T[]] first to ensure the cast will succeed if it is able to.
In short, when given:
[List[int]] 1..10

The parser should infer that [List[int]] requires a cast to [int[]] first and automatically apply the intermediary conversion.
Given #11768, we may also want to special case certain types, since Memory<T> itself isn't IEnumerable<T> but it does contain Span<T> which is, and you can already cast T[] to Memory[T].

@vexx32, [System.Collections.Generic.List[int]] @(1..10) currently works fine, or are you talking about optimizations behind the scenes? Also note that (...) is preferable to @(...), as the latter would enumerate and reassemble an already strongly typed array as an [object[]] array.
Note that enclosure of the input array in () in something like  [System.Collections.Generic.List[int]] (1..10) is a syntactic necessity, because the cast operator has higher precedence than the range operator.

you can already cast T[] to Memory (I think?).

You can, but just to spell it out: only if it is the exact target type:
# OK - exact type match; omitting [int[]] or using [long[]] wouldn't work.
[Memory[int]] [int[]] (1..3)

Huh, I didn't recall casting directly to List working. Odd. Good to know that does work, though.
But yeah, the last one there is the one I'd look to improve specifically, then, perhaps.

To continue discussion on #12341, just realized same idea was suggested here. I had this issue in mind, but somehow thought it's about new syntax only.
So speaking of $obj.Method.Invoke - at a glance it's not obvious how it's implemented, but I guess it should use TypeInfo.GetMethod behind the scene. If so, shouldn't we be a step away form getting InvokeGeneric ( basically adding MethodInfo.MakeGenericMethod(types) in the chain)? @vexx32  you mentioned adding such method might need some change in .net core itself, does it mean that PSMethodInfo is using some different mechanism?

I was thinking the MethodInfo was coming from .NET Core directly, but it looks like we have a PSMethodInfo class I was forgetting about which we're already using. So adding something like your suggestion to it would probably be fairly easy, if you could figure out the necessary code to handle the generic method invocation directly. 🙂
It'd be a public API so at least probably require committee review, but beyond that I don't see anything too difficult about getting that added.
I would prefer personally just having generic methods work correctly in PS and be able to provide type args with a more c#-like syntax directly, but that's still a step in the right direction; when we eventually get to that point, we'll then have an available method to use to make it work directly that we can just reuse.

I never did figure out how to properly get the info from the AST through the binders / etc. The code updates I did for the parser are still in my fork though, if someone wants to pick up where I left off:  https://github.com/dlwyatt/PowerShell/commit/74809e0080106926907085d80adea59d6844f070

Binders are passed what they need and in general they don't need the AST. PSMethodInvocationConstraints encapsulates this extra information (beyond the types of arguments) needed to nudge overload resolution in the desired way.

Would it make sense to create a new PSGenericMethodInvocationConstraints to store the additional type arguments needed for a generic method as a separate value to the types of the method parameters?

@vexx32 - probably not - just add a new field to the existing type.

@lzybkr, @dlwyatt, I'm unclear on what the consensus ended up being regarding the syntax:
[Array]::Empty[string](...)

vs.
[Array]::Empty.InvokeGeneric([string], ...)

or both?
If it's either / or, the first seems much preferable; if we only support type literals, then confusion with numeric indexing shouldn't be a concern ([Array]::Empty[0]); also, even multiple overloads are reported as a single PSMethod`1 objects, so indexing isn't useful anyway.

I think that the latter is simpler to implement as it doesn't require parser changes. It could be implemented as a halfway measure, handling the actual logic of invoking the generic method first, and then we can tackle the parser challenge afterwards.
If I recall correctly, the perceived potential issue with a syntax like [type]::Method[type]($args) is that it may be somewhat complex to determine the difference between that, and something like [type]::DictionaryProperty[somekey]($methodArgs) -- which, while pretty rare and hard to use in itself, is (I think) currently valid-ish syntax.
I've gone back and forth on the syntax variations myself really, and I'm not really sure if there's a "good" answer to that myself, it may complicate handling properties that happen to be dictionaries or arrays at the tokenizing and/or parsing level.

@vexx32 - the sharing of logic between invoking methods on a PSMethod and invoking methods the normal way is less than ideal. That's not to say you shouldn't bother implementing the former, just don't get your hopes up that the problem is half solved if you go that route.

The first syntax is what's in the branch I linked earlier
On Fri, Apr 17, 2020 at 12:06 PM Michael Klement<EMAIL_ADDRESS>wrote:

@lzybkr https://github.com/lzybkr, @dlwyatt https://github.com/dlwyatt,
I'm unclear on what the consensus ended up being regarding the syntax:
vs.
Array::Empty.InvokeGeneric([string], ...)
or both?
If it's either / or, the first seems much preferable; if we only support
type literals, then confusion with numeric indexing shouldn't be a
concern (Array::Empty[0]); also, even multiple overloads are reported
as a single PSMethod`1 objects, so indexing isn't useful anyway.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/PowerShell/PowerShell/issues/5146#issuecomment-615330480,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABG67FGL2L3Y5C4BLHEF4CTRNB5ANANCNFSM4D7SUTNA
.

@dlwyatt thanks for that! After resolving a few merge conflicts I was able to get that code rebased on top of the current master branch, and with a few nudges from @SeeminglyScience here and there on some of the more confusing bits, I've got a working implementation.
Tidying it up and figuring out some tests to write, and then I'll throw up a PR.

@vexx32 Bump!

This is already sorted in #12412, it's pending a followup on the review. As far as I know, I've sorted all the outstanding comments out, unless I missed something.

Nice work man!

Wondering, what is the status of #12412 ? Is it dead?

I've rebased it & asked the other members of the Engine working group to give it a look over. 🙂


casting identifier to string since it is a varchar database field

I have had some strange performance issues with the Translatable Behavior after having arount 1 million rows in ext_translations. Queries were dropping to around 300 - 400ms. Checking the Queries I found the following:
This query, generated by the behavior, is slow ( ~300ms ):
SELECT * FROM ext_translations WHERE object_class = 'Demo\DemoBundle\Entity\Demo' AND locale = 'en' AND foreign_key = 111
This query is fast ( ~1ms ):
SELECT * FROM ext_translations WHERE object_class = 'Demo\DemoBundle\Entity\Demo' AND locale = 'en' AND foreign_key = '111'
There are two posibilities to "solve" this. Changing the the foreign_key column to integer did the same performance boost. But since ids can also be alphanumeric this is probably a bad idea to change globally. Casting the identifier to string seems to be no problem, since it is a varchar field anyways. Local tests only increased performance so far.
Thanks for discussing this change with me ( maybe there is a totally easy and obvious different solution to this issue :) ).
Regards,
Mario

Hi, another way would be to either use personal translations or extend translation superclass and override mapping to integer type. the default string was chosen, because it would fit all cases out of the box, though sacrificing performance, but in most cases if you have a mil of records you may need to customize anyway. The default behavior should fit only basic needs.
I see that this optimization will help a lot with the default behavior, thanks!

though, probably will still need to check metadata, whether it is integer or string..

I have seen your changes in commit 8bf241d which reverts the effect of my changes :)
Doctrine\DBAL\Types\StringType::convertToPHPValue() does nothing to the value, so the resulting query is slow again. Or did I miss anything?

maybe I've made a mistake here, will have a look later

@mariohiller I've made a workaround for preformance problems in 2.x versions of Translatable.
Could you look at #1512?


Section numbers for list items

When creating technical documents it would be nice to have the section numbering be extended to list items.
Example:

That's not the scope of the extension, it only affects the TOC.


Empty Clients

Hello there.
Thanks for this, it has been incredibly helpful.
However, the clients list (as specified in example 3) is empty, as I believe plex no longer has a '\clients' url. At least with both my PMP and mobile clients connected, that container is empty. I believe now this needs to be pulled from 'https://plex.tv/pms/resources'. Is this correct? If so, can the code be adjusted to that?
Thanks

I was mistaken. The firewall was blocking that information for some reason. Sorry

Oh my goodness...I'm so glad I found this...why in the world is a firewall blocking this? I've been stumped for hours.

lol, I think I'm hitting this as well now. Not sure whats going on as I have no firewall. What I am seeing sounds a bit like this thread: https://forums.plex.tv/discussion/144468/plex-companion-fling-option-missing-from-plex-web
Going to reopen this bug for now while I investigate.

OK, in my case.  For whatever reason GDM got disabled on my server. Simply re-enabling that got things working again: Settings > Server > Network > Enable local network discovery (GDM).


Rename the package to pygpu?

The name difference with the official conda package is causing some confusion amongst our users.
The problem is that many are doing conda install Theano pygpu and get Theano/python from conda-forge and pygpu/libgpuarray from official conda this way.  This often results in a non-functional install, especially on windows.
Maybe we could rename this package to pygpu to avoid the confusion issues.
I'm not sure how such things are handled in conda-forge which is why I made an issue to see if that was possible.

So IIUC in defaults there is a libgpuarray part that is just C and a pygpu part, which is a Python library that binds to libgpuarray, correct? Any thoughts on doing the same thing here?

Started the work in this direction with PR ( https://github.com/conda-forge/staged-recipes/pull/3142 ).

Once the next libgpuarray release is out, we will need to make some tweaks to this feedstock and the pygpu one. We can discuss more then when it happens.
In the interim, once pygpu is on conda-forge, we can add it as a dependency of theano. No further tweaks will be needed for theano (pertaining to this) after that switch is done.

I agree with splitting the package, so this looks like it's going to be fixed soon.

Yeah, this should help set things up for splitting this package.

With pygpu now in, am switching theano to require pygpu in PR ( https://github.com/conda-forge/theano-feedstock/pull/12 ). Please take a look.

Have added PR ( https://github.com/conda-forge/libgpuarray-feedstock/pull/5 ), which changes the libgpuarray recipe to a simple C library (dropping all Python portions). Also releases libgpuarray 0.6.7.

Also have PR ( https://github.com/conda-forge/pygpu-feedstock/pull/1 ) ready, which changes pygpu into a Python/NumPy package (instead of a metapackage). Also releases pygpu 0.6.7. This will only work after libgpuarray 0.6.7 is released.

Have merged the pygpu change. Should have binaries rebuilt with correct splitting of C and Python components shortly.


Send exploding messages

Reads the exploding mode of the current conversations to decide to send an ephemeralLifetime. r? @keybase/react-hackers for js

Also pipes through ephemeral data from the service to our message data


Compilation errors are reported but code stops executing

First of all, thank you for this wonderful piece of software. It's really great not having to use nodemon or tsc-watch anymore and to work with compiled .js files.. I really think and hope that TypeScript will be the future.
Now on to the issue. When working with ts-node-dev I encountered the same issues as #609. Errors are correctly reported in the console when adding the --type-check flag but when an error is encountered the code stops executing which is a bit annoying.:
// errors reported, script stopped
"dev": "ts-node-dev --type-check --respawn src/index.ts",

I've tried the suggestion of adding the flag --log-error but then no errors are reported in the console but the script does continue:
// no errors reported, script continues
"dev": "ts-node-dev --type-check --log-error. --respawn src/index.ts",
"dev": "ts-node-dev --type-check --transpile-only --respawn src/index.ts",

Ideally I would like ts-node-dev to report all TS errors in the console but don't stop the execution of the code on compilation errors. Would something like this be possible?
Versions:
"ts-node": "^8.10.2",
"ts-node-dev": "^1.0.0-pre.44",
"typescript": "^3.9.5"

Thank you for looking into this.

the code stops executing which is a bit annoying

It should stop and wait for the error to be fixed. No?

Ideally I would like ts-node-dev to report all TS errors in the console but don't stop the execution of the code on compilation errors. The code could still contain valid javascript code that woud work flawless. So no, it should not stop on errors, only report the errors to the console..

I think it is a bit strange approach, if you don't want to TS errors affect the runtime execution just use --transpile-only. All the errors are still usually highlighted in the editor, so you can see them anyway, I don't see a reason to add them to the terminal output considering they not really affect the flow.

True, that is a workaround. But then you can only see the errors in vs code. So you need to open every file to see the red lines. It would be nice to have an option to see the errors in the console but keep the program running. Maybe add another flag?
ts-node-dev --stopExecutionOnTSerror
ts-node-dev --showErrorsInConsole
With flags like this the user is free to select the behavior the wants.

Well first of all ts-node-dev (the same ts-node) is code executing tool, typechecking is optional if you want it to affect execution flow. To do what you ask would require to perform two types of compilation (in case of type error), which is not impossible, but seems redundant.
If you consider Type Errors to be important I would really recommend using typecheking and fix issues as you encounter them, stopping execution of code with errors only helps. Otherwise, if type errors are not important (if you prefer speed, and see typechecking as helping tool while development) just only use transpilation and make full type checking later on the relevant steps (like test, build, etc). It seems to me that what you are trying to achieve is a bit incorrect in terms of dev process logic to combine thigs that are better separate, this is my opinion on the issue.

Thank you for your feedback. Although I don't completely agree I do appreciate your point of view.
I'm new to development with TS and I'm trying to find a way of coding that works for me. To do this it seems to be nice to just be able to run npm run dev have it execute ts-node-dev and show compilation errors in the console so I can go fix them but still execute the program when possible. This will allow only one console to be visible with all the TS errors there are and not having to hunt down the IDE for red squiggle lines.
Thank you for reconsidering.

I just see this as a kind of a mess when you will see the errors in the console but execution continues. Ok, maybe reconsider later, if there will be more inquiries.

Thank you Alex, appreciate it. With the extra flags you can just tell users to use them as they desire. That way that can have what they want. You the functionality already in the code, it's just not modular enough to be used with a flag.
Anyhow, here's me hoping for some thumbs up on this request :)

@DarkLite1
--log-error actually should work as expected, I never used this ts-node option before. Here is the test for this https://github.com/whitecolor/ts-node-dev/blob/master/test/index.ts#L95

@whitecolor this does indeed fix my problem. Thank you for looking into this. Using that option now. Closing this one :)


nixos/seahorse: require gnome3.dconf

Fix #41886.
Motivation for this change
Seahorse previously would not function (at least, not without GNOME installed).
Things done
Adds pkgs.gnome3.dconf to environment.systemPackages. Tested with the gnome3.nix test.

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[X] NixOS
[ ] macOS
[ ] other Linux distributions

[X] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[X] Fits CONTRIBUTING.md.

I always though having a wrapper adding dconf as a module is enough.
GIO_EXTRA_MODULES='/nix/store/7vqsjsmkm0gi2xksa9jq1xy22540dx47-dconf-0.28.0-lib/lib/gio/modules

Now it seems that the dbus service is required too.
That would affect many other gnome applications too.

See also https://github.com/NixOS/nixpkgs/pull/30271, https://github.com/NixOS/nixpkgs/pull/39997…

Yeah I remember this is breaking macOS - dbus is unavailable IIRC. I can't remember the app I hit this issue with though...

@Mic92, do you mean that making this change would affect other GNOME apps, or just that this issue also affects other GNOME apps?
@matthewbauer, Seahorse is only available on meta.platforms = platforms.linux.

@gmarmstrong I mean that potentially other Gnome applications might be broken ouside of GNOME for the same reason.