PCC: compile module with knowledge of maximum memory size limit to avoid dynamic checks entirely?

A fuzzbug recently came in (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67568) that has a load with a very large (~3GiB) offset. With our bounds-checking code and default setting of a 2GiB guard region, we handle this today with a dynamic bounds-check. PCC currently cannot handle dynamic bounds-checks. So if we want to (i) deploy PCC in our default configuration, and (ii) correctly handle all legal Wasm inputs, we need to find a way to check such instructions.
It occurs to me that if our guard region is large enough, any load/store with an offset larger than the guard region will also have a minimum address that is larger than the biggest possible memory in most "reasonable" production configurations. If we can guarantee that, and compile with that assumption, we can statically compile a i32.load offset=9bazillion instruction into a trap (with a HeapOutOfBounds trap code so it is semantically unchanged to observers). This would allow PCC to check the result as well.
I'd like to file the issue first to get opinions: would it be reasonable to have a maximum_memory_size configuration option that bounds all memories, such that it is an error to grow larger than that, and then use this during compilation to handle the above case? And then set it so that it is at (or ever so slightly smaller than, by the maximum load/store width) the default guard-region size, so we get the nice property that our default configuration never generates dynamic bounds checks?

Here's a minimal tests/disas/ testcase for reference:

compile test
;;! target = "x86_64"
;;! test = "compile"
;;! flags = [ "-Cpcc=y" ]

(module
  (func (result i32)
    i32.const 0
    i32.load offset=3876003872)
  (memory (;0;) 8164 8167))

It occurs to me that if our guard region is large enough, any load/store with an offset larger than the guard region will also have a minimum address that is larger than the biggest possible memory in most "reasonable" production configurations

This isn't quite true in that wasm memory loads create an effective 33-bit address meaning that with a 4G guard region we statically know everything will hit a guard page and no dynamic bounds checks regardless of the offset are needed.
Notably you can't use an offset larger than the memory size, that's a validation error (IIRC).

we can statically compile a i32.load offset=9bazillion instruction into a trap (with a HeapOutOfBounds trap code so it is semantically unchanged to observers). This would allow PCC to check the result as well.

I'll note that I believe this already happens.

so we get the nice property that our default configuration never generates dynamic bounds checks?

I think that makes sense yeah. The pooling allocator actually already has this configuration but that's purely runtime configuration, it's not compile-time configuration, and this would be a compile-time-config option (that then must be matched at runtime as well)

4G guard region

Ah, the default is a 2GiB guard region (post, and separately 2GiB pre); that's what I'm assuming we need to find a workable PCC configuration for, in order not to impact virtual memory overhead.

I'll note that I believe this already happens.

@alexcrichton please see the test case above, where it doesn't, for a default wasmtime config (2GiB guard region) -- is there some other setting we need to enforce to get this property? In particular I note this case in the seven bounds-check cases, which triggers for static memories where the guard is otherwise not large enough. There is the case here that statically becomes a trap, but that's only if the offset and size exceed the memory size bound, and that bound needs to be 4GiB to elide bounds checks otherwise; so a 3GiB offset doesn't fit. That's precisely the gap I want to reify here: a separate "actual maximum size" configuration that is distinct from "virtual memory size".

Notably you can't use an offset larger than the memory size, that's a validation error (IIRC).

This also seems not to be true, per the test case above: its max memory size is 8167 pages (a little under 512MiB) yet it has an offset of ~3GiB. Regardless, we aren't rejecting modules at compile time that have a max memory size somehow incompatible with our memory config, so we can't rely on that here.

Ah yes sorry forgot to mention Wasmtime's default is 2G. I can confirm that with -O static-memory-guard-size=$((1<<32)) the above test case works as-is today. I can also confirm that if the above offset is changed to $((1<<32)) - 4 then it still works, but offsets larger than that actually work with the default 2G region.
I think I was confusing a different optimization we have where when access size + offset is >4G we compile a static trap, but you're right that the maximum size of memory isn't taken into account. That's something we should add! Not that it would "truly" fix this test case though since most memories don't have an upper bound in the real world.
Also sorry for "larger than memory size" I should have clarified I meant the indexing type. The index type here is a 32-bit integer so the offset 1<<32 is rejected by wasm validation. My point being that for 32-bit linear memories we statically know all linear memory offsets are less than 1<<32, so we are guaranteed that everything is a 33-bit address and no more.
Thinking a bit more on this, I think we could perhaps do two different things:

With PCC recommend a 4G guard region. Whether or not that's suitable for virtual memory overheads I think will be situation-specific. For example with mpk there's no issue as you could probably still get more instances today than already. Additionally the 2G guard region today is somewhat arbitrarily the maximum signed distance to go, but there's no reason (pending bugs) that we couldn't switch the default to a 1G or 512M guard region which would already be a significant improvement over today. Mostly I don't think 4G guards should be immediately discarded, but instead weigh the cost with other factors.
I don't really know PCC at all, but is it possible to codegen a different pattern for bounds-checked offsets? For example with a 2G guard region the risk is that the access lies outside the guard region, but we could perform manual arithmetic with a manual trap if the access is >=6G and then perform a normal trapping load if it's <= 6G. I'd hope that in this situation it'd look to PCC like the address was a normal load, but I don't know enough about PCC to know if this is possible.

I'm trying to follow along and I have a lot of questions.
First off, how do I tell which heap settings Wasmtime actually picked?
 So is this case "3. General case for static memories" with bound=4GB? And this means we're allocating 4GB of virtual memory and leaving the out-of-bounds parts unmapped?
Wasmtime won't generate even unoptimized CLIF for this with -Cpcc=y so I guess the problem is there was some instruction in the unoptimized version that we can't generate a fact for; what was it?
After optimization, the bounds-check constant-folds away:
Optimized CLIF
;; @0022                               v7 = load.i64 notrap aligned readonly checked v0+80
;; @0022                               v9 = iconst.i64 0xe707_2020
;; @0022                               v10 = iadd v7, v9  ; v9 = 0xe707_2020
;; @0022                               v13 = load.i32 little heap v10

This seems like it should be easy to verify: it's an add of a constant <4GB to a pointer which I gather should be valid up to 4GB. Perhaps it would be a better test case if the index were an i32 parameter, since I assume your concern is for PCC's ability to verify the general case.
If I add the -Ostatic-memory-maximum-size=535232512 flag, then the same example compiles to an unconditional trap. (That number is 8167 pages, the maximum bound of the memory's limits.) So is that the flag that you wanted, Chris?

Ah I think now that heap settings moved out of CLIF you'd need to consult tunables and cross-reference that with flags passed to Wasmtime. (where tunables mostly match Config names but not always)
By default static memories have a max size of 4G (exactly 1<<32) and a 2G guard region (1<<31).

And this means we're allocating 4GB of virtual memory and leaving the out-of-bounds parts unmapped?

Mostly correct yeah. The code generator assumes that 4G (the static memory bound) + 2G (guard region) is allocated and the only mapped pages are those actually in. use by the heap.
I also sort of forget exactly what the static_memory_bound variable is. It selects dynamic/static but I think you're also seeing an interaction that I forgot about (and I suspect at this point you could probably explain it to me as well!)

@jameysharp ah, sure, it was a trivial testcase that was not optimization-proof, but replace the i32.const 0 with a local.get if you like -- the interesting bit is that with a large-enough load offset, we still get a dynamic bounds check, even with a nominally static memory. (Sorry for the confusion!) The bit that is not PCC-able is exactly the dynamic bounds check -- the symbolic reasoning to match up the compare to the select breaks when opts merge more than one access together. The point of the testcase is not exactly literally to trigger a PCC failure but to show that we can still get dynamic bounds checks here.

I also sort of forget exactly what the static_memory_bound variable is. It selects dynamic/static but I think you're also seeing an interaction that I forgot about (and I suspect at this point you could probably explain it to me as well!)

Reading enough of the implementation to explain this may help me understand what Chris is asking for, so here goes.
Tunables::static_memory_bound is both:

the maximum number of 64kB pages that a memory can use while still using the static memory heap style,
and the amount of virtual address space that the runtime will reserve (before guard pages) for static memories.

If at compile time we know that a static memory style was selected, and the constant offset plus access width given in the memory access is greater than the static memory bound in bytes, then it doesn't matter what index we add to that offset: The result will always be greater than the current size of the memory—which must be no bigger than the static memory bound—and so the access should always trap.
Chris, what you want is 4GB of guard pages and a static memory bound of reasonable size. For example if an embedder doesn't allow more than 128MB of linear memory, the static memory bound should be set to 128MB, or 2048 pages. The static memory bound will turn overly-large offsets into traps, and for smaller offsets, the 4GB of guard pages will ensure that any i32 index doesn't need a bounds-check.
We could also just trap when the offset is too large for any memory which is declared with a maximum limit, whether it's static or dynamic. If the static memory bound is used as an implied maximum limit when the module doesn't declare one, then that would remove the need for the current "case 1" for static memories. However that doesn't change that we need 4GB of guard pages to avoid bounds-checks when the offset is smaller.

The bit that is not PCC-able is exactly the dynamic bounds check -- the symbolic reasoning to match up the compare to the select breaks when opts merge more than one access together.

I'd still like to understand this better. This error is being reported before any optimization pass runs, so optimizations merging accesses isn't the direct cause, right? Is it that you need a new kind of fact which you then can't define a merge operation for? Which specific instruction do you need this fact for? Is it the select_spectre_guard which could produce either a known-valid pointer or a known-inaccessible one?

I'd still like to understand this better. This error is being reported before any optimization pass runs, so optimizations merging accesses isn't the direct cause, right? Is it that you need a new kind of fact which you then can't define a merge operation for? Which specific instruction do you need this fact for? Is it the select_spectre_guard which could produce either a known-valid pointer or a known-inaccessible one?

Yes, but again, we're a few steps removed from the actual issue in this test case. The point of the test case is to show that dynamic bounds checks still occur; it says nothing about PCC, is not designed to be tested with PCC, has never even heard of PCC, etc etc.
The underlying cause for all of this is that, yes, when optimizations run, multiple dynamic checks get merged together (the same base address is used for multiple accesses because of GVN, and the facts merge), and when symbolic facts merge, we either need to merge symbolic worlds (unscalable) or a consistent merging strategy; and separately, sometimes optimizations around constant addresses are smart and there's no stable thing with an identity to hang the comparison reasoning off if, because constants get absorbed directly into instructions or folded away.
I haven't shown any test case that demonstrates that, because it's deep in fuzzbugs and I really don't have the bandwidth to write it up properly (sorry). You're of course welcome to dig into it; the pcc_memory integration test will trigger the issue if you enable bounds-checking configurations. But it's a deep research question and IMHO the right course is to save it for later (and I will not be able to help further right now).
The point of all this is that we think we can do PCC if we only have implicit (virtual memory)-based bounds checking. So I pushed a recursion level and instantiated a subproblem: let's create a mode where codegen never does explicit bounds checks. Then we don't have to worry about the above. That's this specific issue, and the test case above demonstrates that: we still get dynamic bounds checks even in the default configuration (and even under opts if you replace my silly const-0 with an arg input). Does that make sense?

Okay, that makes sense.
Does my suggested configuration sound reasonable to you? I don't think we need any new configuration options to do what you want. Just set -Ostatic-memory-maximum-size to e.g. 128MB and -Ostatic-memory-guard-size to 4GB. Then I believe you'll never get bounds checks.

I think so actually -- thanks for digging into this! If I understand right, one way to see it is that we've been "holding it wrong" wrt the two knobs and how they divide the total space: if the embedder has a reasonable per-instance limit significantly below 4GiB then most of the address space should be within the guard, not the possibly-available static memory region.
I think that's sufficient to close the issue actually; when I pick PCC back up I'll plan to fuzz under such a configuration.


Add detailed info to offer trace

What was wrong?
Updated trace_offer to return more detail about what happened during the Offer, specifically to identify between cases where the offered key was rejected, or the transfer failed, which is helpful to know inside the state bridge.
How was it fixed?

added OfferTrace type to capture more information about failed trace_offer requests

To-Do

[ ] Clean up commit history and use conventional commits.

Blocked by #1485, since that's updating the content_keys type used here, but this is ready for review, and I'll make the adjustments once that gets merged

Blocked by #1485, since that's updating the content_keys type used here, but this is ready for review, and I'll make the adjustments once that gets merged

merged


fix oci image index script

The script does not work for multiarch images as it does not append to the manifests. Instead it replaces with the latest one which results in the last architecture to be the only one supported. JQ has an option to append an element to an array.
Fixes #638

We at @myopenfactory deployed these changes into our monorepo and confirmed that it works like the 1.x versions.

This is a duplicate of #633.

This is a duplicate of #633.

Oh my bad, did look over the PRs but did not associate this. Let's try to get the other one in. Maybe you could reference the ticket #638 to solve by your ticket.


Toggle mjs files to javascript/auto type

https://github.com/graphql/graphql-js/issues/1272
Wow, this new testing setup is really handy.

Landing this and releasing.

Hello,
this is still broken for me.
import graphql, { parse, buildASTSchema, buildClientSchema } from 'graphql'
console.log(graphql, parse, buildASTSchema, buildClientSchema)
// ouput : /static/media/index.497f5ee2.mjs undefined undefined undefined

I'm using<EMAIL_ADDRESS>


Feature/source override

Are you a current Fivetran customer?

Yes, original PR created by @djbelknapdbs
What change(s) does this PR introduce?

This PR allows for users to set a variable within their dbt_project.yml that overrides the salesforce source name used within the macros in this package.
Does this PR introduce a breaking change?

[ ] Yes (please provide breaking change details below.)
[X] No  (please provide explanation how the change is non breaking below.)

Is this PR in response to a previously created Issue

[X] Yes, Issue #6
[ ] No

How did you test the PR changes?

[X] CircleCi
[ ] Other (please provide additional testing details below)

Select which warehouse(s) were used to test the PR

[X] BigQuery
[X] Redshift
[X] Snowflake
[X] Postgres
[ ] Databricks
[ ] Other (provide details below)

Provide an emoji that best describes your current mood

👽

This has been applied and better implemented within PR #14


Add a ImageRepoURL field to CreateLBFargateAppInput

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@SoManyHs I'll fill that value once the interfaces in this PR https://github.com/aws/amazon-ecs-cli-v2/pull/289#pullrequestreview-312963184 are implemented :)


Travis build error with bundler

Example of failed build:
https://travis-ci.org/bitcoinops/bitcoinops.github.io/builds/628560320?utm_source=github_status&utm_medium=notification
Affecting
https://github.com/bitcoinops/bitcoinops.github.io/pull/305
and
https://github.com/bitcoinops/bitcoinops.github.io/pull/282
Appears to have started after https://github.com/bitcoinops/bitcoinops.github.io/pull/304 was merged.

Weird.  #304 said it passed its Travis check.


Drop support for Ruby 2.4

Ruby 2.4 has reached EOL. Should we drop support for it? Perhaps this should be done as a part of 1.0 release?
Previous discussion about 2.2 & 2.3 https://github.com/rubocop-hq/rubocop/issues/6945
@bbatsov Mentioned here that we should adhere to same EOL as MRI itself.

Good point. Generally I don't think we have some burning need to drop 2.4, as it doesn't create much maintenance overhead, but dropping it before 1.0 might be a good idea, otherwise we might be stuck with it for a while. @rubocop-hq/rubocop-core as always, I'd love to hear your take on this.

My opinion (I don't know how prevalent it is) is that dropping compatibility with a Ruby version does not require a major version upgrade since it is part of the gem manifest and won't introduce any incompatibility with another Ruby version (as it simply won't install). So my take is that we should continue supporting it, especially since it's not a big overhead, and we can decide to drop support when we feel like it is worthwhile

I looked through old PRs to find out when we dropped support for older rubies:

#3328 dropping 1.9 was merged on 18 Jul 2016.
#4787 dropping 2.0 was merged on 2 Oct 2017.
#5990 dropping 2.1 was merged on 18 Jun 2018.
#7026 dropping 2.2 was merged on 8 May 2019.
#7869 dropping 2.3 was merged on 12 Apr 2020.

If we drop support for old Rubies with the same frequency as new versions are released (once per year), I would expect us to drop support for 2.4 in April/May 2021.

Thanks for looking into this! Seems we've been quite consistent in sticking to unsupported Rubies for at least a year. I guess we should update our support policy then. :-)

RuboCop has customarily provided support for about a year after EOL of Ruby version. And this span seems to be working as it is. I don't think it's time to drop it, as it will impact gems and apps that rely on RuboCop support.
I can take this as part of the maintenance when dropping. I think RuboCop core should provide the community with a margin of transition.

Fair enough. Seems we have a deal then.

I've added the policy to README. Should we add it somewhere else as well? Should it be shown on the docs/website?

Probably we should have some dedicated section on the docs about the Ruby support/compatibility policy, but we can address this down the road. I like this as a reference https://nrepl.org/nrepl/0.7.0/about/compatibility.html (probably because I wrote it myself :D )


Create a changelog file in the docs directory

Done: https://github.com/startwind/forrest/blob/main/docs/changelog.md


No link that describe how to link Cloud Partner Profile with Dev Center Account

In the end of te page it states "Once you have completed the Dev Center registration, link your Cloud Partner Profile with your Dev Center account (see detailed steps here). This step must be completed before publishing your transact offer." But there is not link or reference at see detailed steps here. It think it should be https://docs.microsoft.com/en-us/azure/marketplace/cloud-partner-portal-orig/cloud-partner-portal-manage-publisher-profile?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: e16147fb-8da4-6c41-be0f-b2856043c7c9
Version Independent ID: dab8f3c3-c3ff-883b-406d-405701a53224
Content: Become a Publisher in the storefronts on the marketplace
Content Source: articles/marketplace/become-publisher.md
Service: marketplace
GitHub Login: @ellacroi
Microsoft Alias: ellacroi

@svandenhoven Thanks for the comment! We have shared this feedback with the content author for further review and update as appropriate.
@ellacroi & @v-miclar FYI.

Please reassign this issue to Qian Wen (v-qiwe).

#please-close the link has been added and is now live.

Hi, and what is the right link? We have now two company profiles, one for Dev Center, second for Commercial Marketplace, even more when transitioned from other countries. Is possible to merge them?

@Kazzan could you please provide more details to your question.  Also please direct your ask to<EMAIL_ADDRESS>who will look into this.  Thanks!

Hi @qianw211, I solved this directly with support. Great support from them.
The initial problem is, that the invites to Partner Center (migration from old portal or Co-Sell invite) allow create another organization to someone, who have no rights in the correct one. Even in time when he is the member of same AAD tenant. This should at least warn like "There is already one organization ABCD registered. Do you want to create another one, or request access from someone else?".

#please-close, seems this is already resolved!


[DOC-8681][DOC-8690] Document VIEWSYSTEMTABLE privilege

[DOC-8681][DOC-8690] Document VIEWSYSTEMTABLE privilege
(Also fix a couple small things that caught my eye)
Previews
src/current/v23.1/security-reference/authorization.md
src/current/v23.1/grant.md
src/current/v23.1/revoke.md
src/current/v23.2/security-reference/authorization.md
src/current/v23.2/grant.md
src/current/v23.2/revoke.md
src/current/v23.1/security-reference/authorization.md
src/current/v23.2/security-reference/authorization.md

friendly ping @mdlinville


Warning message appears when pulling from docker registry

when doing docker pull crossbario/crossbar the following message appears after download:
Status: Downloaded newer image for crossbario/crossbar:latest
docker.io/crossbario/crossbar: this image was pulled from a legacy registry.  Important: This registry version will not be supported in future versions of docker.

maybe this is osx related .. works for me without warning:
oberstet@thinkpad-t430s:~$ sudo docker pull crossbario/crossbar
[sudo] password for oberstet:
Using default tag: latest
latest: Pulling from crossbario/crossbar

4d06f2521e4f: Already exists
a3ed95caeb02: Already exists
6077fff8857d: Already exists
822d971f3d78: Already exists
569a32b15f4f: Already exists
00a75c72d838: Already exists
ef5e92f70e36: Already exists
Digest: sha256:3de888a1cd647afe86519da5e8af82beaac1beec892f70bd71565302cf78c56d
Status: Image is up to date for crossbario/crossbar:latest
oberstet@thinkpad-t430s:~$


Many problems

Hi! I was trying to configure bots in some way and ran into certain problems:

How to adjust the power of the lord? I think everyone knows that our standard lord is much stronger than Lord Rat and at the same time weaker than Lord Wolf. Where is it configured?
How to set up the initial number of troops and their types (which are issued to a computer player at the beginning of the game).
In the settings of each AI there are various types of attacks. For example, AttUnitMain1 and AttUnitEngage.  How to make it work on all other subspecies of attacks? This is necessary so that the Wolf, for example, after a large number of attacks, would send not 200 swordsmen and pikemen + 20-30 crossbowmen to the enemy, but 200 swordsmen and pikemen + 100-150 crossbowmen. The option with increasing AttUnitEngage for example to 1000 is not suitable. Since in this case I will not be able to control the exact value of the crossbowmen in each wave (I need accuracy).
P.S. Sorry for my English.

Зелениум ты по-русски понимаешь?

Да

Ура слава Богу))  Согласен с тобой про Лорда. А про армию не понял )) хочешь увеличить количество?

Нет. Хочу сделать точное количество армии. Чтобы расли все значения, а не только AttMainUnit

Не совсем тебя понял

Thank you for opening up an issue. But several of these already have an issue related to them.
The power of the lord is probably not configurable in the AI parameters. The starting troops are also not found in there, 100% sure.
And about the attack increase. That is how the formula works right now. There is no way to change that without great effort and time.

Okay. You can still do without the first 2 things, but what about the 3rd? If it cannot be changed, then you need to at least understand it. I can not do it. If you put AttUnitPatrol for example 100000, then they will not go at all that amount, but about the same as AttMainUnit1. How does it all work ???

Generally speaking, are you already using? I mean AIC files and so on? With what? I just think that there I can find everything I need. Have you used HexEditor?

is suggested in https://github.com/Sh0wdown/UnofficialCrusaderPatch/issues/459 and https://github.com/Sh0wdown/UnofficialCrusaderPatch/issues/156 with a hint how to potentially implement it

Someone said "It is stored in a separate array gStartupTroopCount[19][3][20]
[StartupTroopGroupType (AIType or 18 + LordType for human)][GameType][StartupTroopType]
pgStartupTroopCount = (int*)0x00614358 for v1.3"
Where is this array located? How to find and edit it?

That sounds like a hex editor array not yet implemented into the aic. If you know how to use cheat engine or idafreeware you might be able to figure out the exact value to change from that, but I've never figured out how to browse cheat engine or ida.
What was your solution to problem 3, just for others who might be having the same issue...

The solution was to understand how it all works.
I will describe now.
If you use AttMainUnit and only this, then the units from the AttForceBase section will go all to AttMainUnit. Example: you set the CPU to attack only AttMainUnit1-4, without others (such as AttUnitPatrol). In AttForceBase, set the value to 20. In AttMainUnit, you put Spearman.
Congratulations! Your bot will attack 20 Spearman.
If you use AttMainUnit and, for example, AttUnitPatrol, then AttForceBase will split in half between these 2 types of troops.
For example, you added to the previous example
AttUnitPatrol = EuropArcher
and
AttUnitPatrolMax = 10000.
Congratulations. Now in the first attack, the CPU will attack at ~ 10 archers and ~ 10 spearmen (there is a small spread in one direction or another).
In this case, with an increase in attack power over time, the proportion will remain 1: 1.
If in the last example you put no
AttUnitPatrolMax = 10000
but
AttUnitPatrolMax = 10
then such a proportion will be observed only in the first attack (where there are only 20 units). In the next attack, the number of archers will remain no more than 10, but everything that grows in AttForce will go to AttMainUnit
That's how it all works.

But guys, please help me with questions 1 and 2. It is VERY important. Let me at least find out what programs you used to extract information? As far as I understand it is not HexEditor since in it I could not do something interesting.

ByBurton has answered those two, they are currently not possible using this patcher, you would have to modify the exe or process yourself with a hex editor type program, cheat engine for example. I believe both are on sh0wdown's radar to implement in future versions, but that depends on how much time he has, and how much time it will take to sort out, there are also several other things ppl have requested for future versions and I'm sure he has his own ideas as well, so dont rely on your requests being here soon.

As for the 2rd problem - I completely figured it out. It is no longer relevant.

And first too. Ty for help, guys!


feat: add pipe for loom

name: pull request
about: description demo
ps: there is some video rendering issue in edge, fixing it
feat: #583
/claim #583

description
brief description of the changes in this pr.
related issue: #583
type of change

[ ] bug fix
[x] new feature
[ ] breaking change
[ ] documentation update

how to test
add a few steps to test the pr in the most time efficient way.

if relevant add screenshots or screen captures to prove that this PR works to save us time.
checklist

[ ] MOST IMPORTANT: this PR will require less than 30 min to review, merge, and release to production and not crash in the hand of thousands of users
[ ] i have read the CONTRIBUTING.md file
[ ] i have updated the documentation if necessary
[ ] my changes generate no new warnings
[ ] i have added tests that prove my fix is effective or that my feature works

additional notes
any other relevant information about the pr.

@tribhuwan-kumar whats the status of this PR again?
looks decent to merge already, can do audio later

@tribhuwan-kumar whats the status of this PR again?

same as prior, just need to embed audio in video. i'll do it in free time cz there is some ffmpeg stuff to do!

looks decent to merge already, can do audio later

if you're okay with it then you can merge it!

also, would you want to sell this pipe? dm on discord @louis030195
we can make it easy to charge for it through the pipe store and you get the revenue

i don't get it. "sell this pipe" isn't that bounty is for creating this pipe?

/approve

@tribhuwan-kumar we're setting up things so you can charge for pipes in the pipe store for for example people would have to pay $50 one time payment to use your pipe


move repo to Github organization

This project is now community-maintained. The original author, @UnbounDev is not involved much.
I'm the primary maintainer, but really I'm just another user who happens to have commit and publish rights. Maintenance is being down by a community of users like @edilson so improved a few things in the most recent release.
With the blessing of @UnbounDev I'd like to create a new Github organization, like node-strava and move the project into that neutral namespace, as a step towards possibly expanding the number of users with commit and release rights. @UnbounDev is welcome to maintain commit access there.
I helped the passport-saml project go through a similar transition and it worked quite well. That project is now more active then it ever has been. It's the #1 SAML library for Node and has made rapid improvements lately with better TypeScript support among other features and bug fixes.
Although Strava has their own Swagger-based client, clearly there is ongoing demand for a Node Strava client that is installable from NPM and works in a more conventional way.

Thank you for making this proposal and maintaining this project, @markstos, I am very grateful to you and the community for extending the project's life.
Please create a neutral space, as you see fit, and move the repository into that space. Please let me know if I can assist in any way.

@UnbounDev I have created a "node-strava" organization and invited you as an owner. The next steps are for you to accept the invitation and initiate the transfer there: https://docs.github.com/en/repositories/creating-and-managing-repositories/transferring-a-repository Once that's done, you can choose how much want to be involved in the new organization going forward.  I also invited @edilson as a recent contributor to join as an organization owner as well.

Alright, this repo has a shiny new home: https://github.com/node-strava/node-strava-v3
Thanks again to you, @markstos and @edilson and all the rest of the community!

Thanks @UnbounDev! As part of this transition, I made a couple of other changes:

Renamed master branch to main. Local checkouts will not be automatically updated.
Enabled a branch-protection rule for main so for now that a pull request and peer-review is required. My hope is that @edilson will continue to help with maintenance either by submitting PRs or reviewing them. If we end up with a single active maintainer again, we'll have to re-evaluate the mandatory peer-review rule. It may also be useful to set up Slack for private maintainer conversations, but we can consider that another time.


Get http resources where input not expected

Hi,
How to deal with with Get http resources where input is not expected
Chaki expects to have some input. But how to deal with cases where there is no input necessary especially Gets.
Kindly guide.

Hi you can ignore the input case  with a function like this
func (ct *fooController) getNoReq(ctx context.Context, _ any) (string, error) {
 return "ok", nil
}

you can do the same thing for response as well
func (ct *fooController) getNoReqAndNoResponse(ctx context.Context, _ any) (any, error) {
  return  nil, nil
}


When I include the library an old RecyclerView draws wrong

Before include:
http://imgur.com/j3y3DnJ
After include:
http://imgur.com/xoKAjKE
I've only included in the build.gradle
compile 'com.github.CymChad:BaseRecyclerViewAdapterHelper:v1.9.3'
Do you know why the height of items in my old recyclerview are drawing bad ?
Do you override some class ?

Should be your XML layout height problem，You can try
android:layout_height="wrap_content"

loading layout code
mLayoutInflater.inflate(layoutResId, parent, false);


Add support for hkern and vkern elements.

Obviously mostly useful for traditional typefaces, but I think even for icon fonts would benefit from this feature being supported.
Example

Without kerning pairs

With kerning pairs
<hkern g1="diamond" g2="diamond" k="100" />
<hkern g1="home" g2="home" k="-100" />

This stays open too long, and seems nobody plan to participate. Let's close. This can be reopened anytime if someone wish to implement.


Is it possible to start using this by replacing all ".so" files of the gperftools tcmalloc with this tcmalloc 's .so files?

I'm using the gperftools tcmalloc and want to give the new tcmalloc a try, can I directly do so by replacing all the .so files of the older tcmalloc with this versions .so files?
I'll also replace the header files with the newer header files.
Just wanted to know if this is possible.

The malloc extension API is slightly different. E.g. if you need to access tcmalloc metrics, you will need to customize your code for this version of tcmalloc vs. gperftools tcmalloc. See https://github.com/yugabyte/yugabyte-db/commit/e145d7f0a531fbcae45cdf1ce4c42aa0f28e5c7e, for example.

Also this version of tcmalloc relies on the Abseil library while Gperftools does not.


Add voucher design

Allow to print LNURLw vouchers with a default LNbits design, or use a custom design of your own. The default, simple QR is still available.
You can use the template.svg in static/images/ folder to start a new custom design.

Looking forward to merging and using once conflicts are resolved


Add name of the ftp configuration file description

I'm starting to use the atom. I tried to create the configuration file, but did not create, I do not know why, I could manually create it hand does not know the name.

im also facing this problem, when i click map to remote, i see a message tell that the configuration file was created successfully but i cant find it, at least we need to know the name of the file, we can creat it manually, with following instruction https://atom.io/packages/sftp-deployment


Couldn't send user_command: Error creating user command error

When I used the script, I started reporting errors almost after a few transactions were sent. any suggest?
message: `Couldn't send user_command: Error creating user command: {"payload":{"common":{"fee":"0.001","fee_token":"1","fee_payer_pk":"B62qiUYxxxxxxx","nonce":"266","valid_until":"4294967295","memo":"E4YzpZRQxxxxx"},"body":["Payment",{"source_pk":"B62xxxxxxx","receiver_pk":"B62qihyyyyyyy","token_id":"1","amount":"7964041225"}]},"signer":"B62qiUYSjLkQuvgouQ1xxx","signature":["Signature","7mXK2mYsQydaoNzzzzzxxxxxMnYwbUkst3vLoix9Xd"]} Error: Input nonce 266 either different from inferred nonce 265 or below minimum_nonce 264`,     path: [ 'sendPayment' ]

A relatively simple workaround is to use the suppression feature to suppress payments that were successfully made, and re-run the payout to pickup the addresses that failed. See https://github.com/jrwashburn/mina-pool-payout#handling-special-accounts---suppressing-or-redirecting-payouts. Manual control over .paidblocks is also be necessary to re-try the blocks (remove the entries from .paidblocks to re-process. See the final paragraph in readme.
Note that these payments may be re-tried automatically, so it is best to wait some time between runs and double check to be certain which payments failed.
Also, note that if this workaround is used, it is important to remember to remove the keys from the exclusion list for the next full run.


Flashing Knife

yo danielkrupinski,
when you set the knife model you also need to sets its m_iItemDefinitionIndex to stop that annoying flickering you get when changing knife.
thanks!

how do i fix the bug ptzer?

Fixed now.

Sorry I didn't reply. Been busy.
On Sun, 14 Apr 2019 at 06:40, Daniel Krupiński<EMAIL_ADDRESS>wrote:

Fixed now.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/danielkrupinski/Osiris/issues/80#issuecomment-482882639,
or mute the thread
https://github.com/notifications/unsubscribe-auth/At8TbcbhnGk-sPlfo0v5Q6YrIaa3BaiCks5vgkDSgaJpZM4b2JBW
.


Simple context building on declarative platform config

I don't remember creating this issue, and I see to my horror that the description field is entirely blank.
@thetorpedodog @mlin @atolopko-czi if you have context on whether this is closed, or closeable, or a dup, please let me know.

No idea. :) OK with closing it.  Important needs will resurface as new issues.


Fix CentOS provisioning and E2E tests

What this PR does / why we need it:
Fix CentOS provisioning and E2E tests.
Does this PR introduce a user-facing change?:
Fix CentOS cluster provisioning

/assign @xrstf

/lgtm

/retest


[Class modifiers] No error in CFE when class from a pre-feature library mixes in base mixin

Test  co19/LanguageFeatures/Class-modifiers/versioning_A02_t04 fails on CFE when  class from a pre-feature library mixes in base mixin from a post-feature. All expected errors below are missed in CFE (but reported analyzer)
// class_modifiers_lib.dart
// @dart=3.0
base mixin class BaseMixinClass {}
abstract base mixin class AbstractBaseMixinClass {}
base mixin BaseMixin {}

// versioning_A02_t04.dart
// @dart=2.19

import "class_modifiers_lib.dart";

class WithBaseMixinClass with BaseMixinClass {}
//    ^^^^^^^^^^^^^^^^^^
// [analyzer] unspecified
// [cfe] unspecified

class WithAbstractBaseMixinClass with AbstractBaseMixinClass {}
//    ^^^^^^^^^^^^^^^^^^^^^^^^^^
// [analyzer] unspecified
// [cfe] unspecified

class WithBaseMixin with BaseMixin {}
//    ^^^^^^^^^^^^^
// [analyzer] unspecified
// [cfe] unspecified

abstract class AbstractWithBaseMixinClass with BaseMixinClass {}
//             ^^^^^^^^^^^^^^^^^^^^^^^^^^
// [analyzer] unspecified
// [cfe] unspecified

abstract class AbstractWithAbstractBaseMixinClass with AbstractBaseMixinClass {}
//             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
// [analyzer] unspecified
// [cfe] unspecified

abstract class AbstractWithBaseMixin with BaseMixin {}
//             ^^^^^^^^^^^^^^^^^^^^^
// [analyzer] unspecified
// [cfe] unspecified

@sgrekhov This should also be fixed now with the fix from https://github.com/dart-lang/sdk/issues/52316

Confirmed. Fixed. Thanx!


Cluster-autoscaler: skip scale down if schedulable but not scheduled pods are present

cc: @piosz @fgrzadkowski

LGTM


Provide PHP information in dashboard

Would be nice to have a PHP section on the dashboard that displayed PHP version, fpm usage, etc.

Taking a look at this now.


provider/core: Snapshot infrastructure saved as map rather than string

I changed infrastructure to be saved as a map rather than a string. This makes it easier to parse the infrastructure into proper format for the diff view in deck. @lwander @danielpeach

LGTM

Please squash

I let github do the squash


Documentation: Rating

Figma:
https://www.figma.com/file/xZ0IW0MJO0vnFicmrHiKaY/Components-Post?type=design&node-id=12038%3A38032&mode=design&t=zJsCfjamR0qblDLe-1
Story Book:
TBD
Comments:

built new component
added documentation

Code implementation has already been done with #2411
@rouvenpost or @Vandapanda please update the States in Figma.


Create separate docs based on scopes

What a nice package, I am enjoying it, But one little thing I want to have is:
I have different API scopes in my project, some of them are for interacting with the front app, some of them are for mobile and etc.
The package should have a config for separating scopes and for a big project it is better to categorize APIs.

Hi @pouyasalimi
Thanks for the issue and thanks for using this package. I am glad it made you a little more productive.
This is an issue, I have been battling with, reference here: https://github.com/rakutentech/laravel-request-docs/issues/44#issuecomment-1040233830
Currently, I receive a list of routes as raw from collect(Route::getRoutes())
Sample below
       Illuminate\Routing\Route {#3931
         +uri: "mobile/api/v1/health",
         +methods: [
           "GET",
           "HEAD",
         ],
         +action: [
           "middleware" => [
             "mobileapi",
           ],
           "uses" => "App\Http\Controllers\MobileApi\V1\HealthController@index",
           "controller" => "App\Http\Controllers\MobileApi\V1\HealthController@index",
           "namespace" => "App\Http\Controllers",
           "where" => [],
           "as" => "generated::HJasdasdasdDPsmH",
           "prefix" => "mobile/api",
         ],

The issue here is how to go about categorizing. There can be many keys to choose from in order to do it scope based. Such as, by prefix, by middleware, by methods etc. If you have good suggestion for it.

Aha, I see.
Putting a search box above the page is very helpful, but separating API scops is another thing.
For example, when I have a monolith laravel backend that handles separate scops of APIs and depending on the team, I don't want to share Admin API's with my mobile app team. So just like the Laravel swagger package, we should have multiple config for multiple API scopes and we can build our documents according to the config file.

I have a little free time these days and if you think I can help, let me know.

Yes, that's correct, will send a PR.

Closing for now....Good feature tho.
Only open to pull req on this


How to: same plugin, different policy for different consumer, in the same ingress resource?

Is this a request for help?:
Yes
What keywords did you search in Kong Ingress controller issues before filing this one? (If you have found any duplicates, you should instead reply there.):
annotations

Is this a BUG REPORT or FEATURE REQUEST? (choose one):
FEATURE REQUEST

Kong Ingress controller version:
kong-docker-kubernetes-ingress-controller.bintray.io/kong-ingress-controller:0.0.3
Kubernetes version (use kubectl version):
Client: 1.10.1
Server: 1.8.11
What happened:
I'm trying to apply different rate limit policy to different consumer, but it seems that:

There can be only one configuration of a plugin in a CRD
And there only one key "rate-limiting.plugin.konghq.com" is allowed in annotation field.

I'll try .. sorry about this.
rate-limiting.plugin.konghq.com: |
  add-ratelimiting-to-route


Setup for A/B testing using new DLCS

The updated DLCS is a ground-up rewrite using updated dotnet core framework functionality for ease of maintainability (chief of which is using YARP rather than nginx to have all reverse-proxy logic in 1 place in 1 language).
It's also using a new infrastructure layout and new image-server so we need to do some A/B testing to verify that performance and functionality is appropriate.

Reminder that we will need to set flag introduced in https://github.com/dlcs/protagonist/issues/325 where asset.role == "closed"

Go Live Rollout Plan
We could do this either way but we think it better to:

get new DLCS deployed first and run live with this before...

...replace current DDS with new Born-Digital aware DDS (develop)

[ ] Required tf iiif-stage-new infra/routes is applied

[ ] Determine testing requirements - do we need a iiif-new environment too? (see note below)

[ ] New DLCS works with current DDS main

[ ] New DLCS supports clickthrough pattern auth same as current Deliverator

[ ] Simulated live traffic loads are acceptable (i.e., way better than current!)

[ ] Locust load tests all look good (random selections)

[ ] Specific scripted tests all give same result (known edge cases, auth etc)

[ ] New DLCS works with current born-digital 2022 develop branch

[ ] When should we impose a workflow freeze?

[ ] Can we run Old and New DLCS simultaneously against the same PostgreSQL? Do we want to do this?

Notes
Do we need a iiif-new environment?
Yes if front-end toggles can pick the source of IIIF
Existing environments (only one DLCS and therefore only one DLCS DB, but we use different spaces for stage/prod:
iiif => prod storage, stage dds buckets, space 5
iiif-stage => stage storage, stage dds buckets, space 6
iiif-test => prod storage, stage dds buckets, space 6
In PR tf

iiif-stage-new => neworchestrator / stage-new dds buckets, space 6, "empty" protagonist DB

Do we need a iiif-new environment?

iiif-new => neworchestrator / new-api / prod storage, prod-new (?) dds buckets, space 5 - a backup and restore of the existing PG DB
We do need this for true A/B switching from the front end
But practically it's going to need to serve existing Manifests from S3
These need to have their DLCS paths rewritten before being served as well as their DDS paths; currently the RewriteDomainLinksTo appSetting supports the latter but not the former.

Running against a shared database
Initial testing uses an empty DLCS database
Later testing uses a fully populated copy of the DLCS database
If there is a content freeze for as long as there is toggle-able A/B testing then that copy is OK
If we want to re-enable workflow before switching over to all new DDS then they will need to share the same postgres database. This is possible but needs further assessing.


Incoming change to worker.config.json from upstream

Is your feature request related to a problem? Please describe.
N/A
Describe the solution you'd like
When the change for https://github.com/Azure/azure-functions-host/issues/3214 is made, update our worker.config.json to match.
Describe alternatives you've considered
N/A
Additional context
N/A

Upstream has made the changes.  This needs to be fixed ASAP.


Do not replace method covers annotations with attributes in CoversAnnotationWithValueToAttributeRector

Currently, this class refactors all occurences of cover annotations, as described in the example. However, method-based attributes are not supported and never will be (see https://github.com/sebastianbergmann/phpunit/issues/5175 and https://github.com/sebastianbergmann/phpunit/issues/4502).
Applying this rule to a testsuite that uses method-based annotations will now result into a broken tests suite (Attribute "PHPUnit\Framework\Attributes\CoversClass" cannot target method (allowed targets: class)).
I think there are two possible solutions:

ignore all method-based annotations altogether, possibly with some warning message provided to users to notify them on this behavior
move all the annotations to be class-based, as using multiple covers attributes on a class is allowed

Hey there!
The issue is not Rector failing, but actually Rector producing code that makes PHPUnit fails. This already happens on the documented example in this repository:
 use PHPUnit\Framework\TestCase;
+use PHPUnit\Framework\Attributes\CoversClass;
+use PHPUnit\Framework\Attributes\CoversFunction;

-/**
- * @covers SomeClass
- */
+#[CoversClass(SomeClass::class)]
 final class SomeTest extends TestCase
 {
-    /**
-     * @covers ::someFunction
-     */
+    #[CoversFunction('someFunction')]
     public function test()
     {
     }
 }

should be
 use PHPUnit\Framework\TestCase;
+use PHPUnit\Framework\Attributes\CoversClass;
+use PHPUnit\Framework\Attributes\CoversFunction; # OPTIONALLY

-/**
- * @covers SomeClass
- */
+#[CoversClass(SomeClass::class)]
+#[CoversFunction('someFunction')] # OPTIONALLY
 final class SomeTest extends TestCase
 {
-    /**
-     * @covers ::someFunction
-     */
     public function test()
     {
     }
 }

(Note: I've attempted to use the demo to run the code, but could not get it to apply this rule, see https://getrector.com/demo/e7acfebb-4fe4-42b0-aa5e-f92ecce676f8).

Seems the opening PHP tag was missing: https://getrector.com/demo/94bd6a93-2b06-4823-8a9a-be9058fad69d
Is that one correct false positive?

Ah that explains a lot! I've created https://getrector.com/demo/34c92edd-6a12-46e2-a1d3-fcbb6003a8e0 now, which currently has 2 changes but should have either:

no changes at all
result in

 <?php

 use PHPUnit\Framework\TestCase;
@@ -4,16 +4,12 @@

+#[\PHPUnit\Framework\Attributes\CoversFunction('someFunction')]
+#[\PHPUnit\Framework\Attributes\CoversClass(SomeClass::class)]
 final class SomeTest extends TestCase
 {
-    /**
-     * @covers ::someFunction
-     */
     public function test()
     {
     }

-    /**
-     * @covers SomeClass
-     */
     public function another_test()
     {
     }

Thanks for the amazing diffs. That made everything easier to understand for me :+1:

Thanks for solving this; hopefully will find some time to try this out soon

If I change it this way I get an error:
Function "::someFunction" is not a valid target for code coverage

Or am I missing something?

If I change it this way I get an error:
Function "::someFunction" is not a valid target for code coverage

Or am I missing something?

Some things, like class functions combined with @coversDefaultClass, are no longer supported: see https://github.com/sebastianbergmann/phpunit/issues/4502. I'm not sure if there some more general way to detect those and handle them differently (i.e. delete them altogether, as they are no longer supported), but the solution implemented to resolve this issue at least moves the code from "not running at all" to "running with warnings" in these cases.
If you have a suggestion for improvement, you might want to create a separate issue for that.


Adding inline comments is not working anymore

Most likely due to https://github.com/eXolnet/redmine-code-audit/commit/90450affa2eca30616b0097e7c72fed997e60e8e#diff-9b2ab1c1505d299b2274bb5a152659a4R5.

Here's an example of the data sent when we post the form:
utf8:✓
authenticity_token:TnwfSZwpZSHxQcBLqJao2/LpS52tP51e3Qv4SULAs+o=
inline_comment[0][line_begin]:3
inline_comment[0][line_end]:3
inline_comment[0][change_id]:null
inline_comment[0][content]:Test
audit_action:
audit_comment:General
commit:Save

The change_id is null. I think it might be the source of our problem.

@xel1045 you should probably not work on that stuff since I've already fixed it in my branch :stuck_out_tongue:.

Well.. It's fixed on develop.


Document The Vaadin Way for working with service workers

Service workers are useful for cases where extra caching and progressive improvement are needed, such as offline and poor network connectivity situations. Right now, there's no clear way on how service workers should interact with Hilla applications and Hilla endpoints. The Vaadin Way could document opinionated best practices for working here.
Some ideas for topics:

Working with CSRF tokens in service workers
Using Google WorkBox - yes, no, sometimes?
Cache strategies

What to cache and what not to

Passing message events - what to send and what not to send
How to account for Vaadin specifics in sw.ts

message event handling order - should Vaadin's internal messages be handled first?

Communication between the Hilla app and the service workers
Constructing callbacks
Message bus -style communication patterns

Example: chat messages that are synchronized in the background and pushed to the UI

Error handling in service workers

Is it technically possible to do all this right now, or are there any limitations in Hilla that must be addressed first?

Is it technically possible to do all this right now, or are there any limitations in Hilla that must be addressed first?

Technically yes, but it requires hacky workarounds right now. Related issues:
https://github.com/vaadin/hilla/issues/2788
https://github.com/vaadin/hilla/issues/2789
https://github.com/vaadin/hilla/issues/2790
https://github.com/vaadin/hilla/issues/2791


{Compute} Remove credentials from recordings

#14344, #14532 contains access tokens (credentials) in x-ms-authorization-auxiliary headers.
x-ms-authorization-auxiliary will not be recorded anymore since https://github.com/Azure/azure-python-devtools/commit/8915c3fc276242b6818462b50c9340fd925e0842, added by https://github.com/Azure/azure-python-devtools/pull/46. This change is released in azure-devtools 1.1.0.
Azure CLI didn't take this change until #14477 which bumped azure-devtools to 1.2.0 and is merged after #14344, #14532, so this issue is already fixed. Re-running the test won't record x-ms-authorization-auxiliary anymore.
Actually, if a test is marked as @live_only, no recording should be generated at all! This should be improved in the test framework.

@qwordy You could use LiveSenarioTest instead of ScenarioTest to avoid recording if you only want to run the test in live mode.https://github.com/Azure/azure-cli/blob/da4239a3b5ae9e2cc74785522bbb11f27b26eaf4/src/azure-cli/azure/cli/command_modules/vm/tests/latest/test_vm_commands.py#L4920

VMCrossTenantUpdateScenarioTest

It was written by XXX on 7/29/2020.

I remember there is bug in LiveSenarioTest

@qwordy You could use LiveSenarioTest instead of ScenarioTest to avoid recording if you only want to run the test in live mode.
https://github.com/Azure/azure-cli/blob/da4239a3b5ae9e2cc74785522bbb11f27b26eaf4/src/azure-cli/azure/cli/command_modules/vm/tests/latest/test_vm_commands.py#L4920

Thank you so much!


Create Readme.txt

@maxinvests please don't send in this kind of spam pr's, it will get denied and you are wasting everyone's time


Compartmentalize name ownership

Add the ability to generate new addresses on the fly and specify what address should be used to register a name.

Make sure the bitcoin change from one name op tx isn't sent to an address that's been used for another name, to ensure compartmentalization.

One potential solution is to introduce the notion of "accounts", similar to the way the Mycelium bitcoin wallet works. Each account is its own branch of the master HD wallet and is used to create a bunch of derivative addresses that are completely isolated from addresses generated by other accounts. This way change from one account never unintentionally spills over into other accounts.

@john-light we merged the CLI repo with blockstack-core and I moved this issue there. Closing this copy of the issue.


Rare issue in HybridMediaPlayer while using setDataSource()

Hii @h6ah4i
I just encountered an issue while using HybridMediaPlayer at setDataSource(Context context,Uri uri) Method.
Please note the parameters I am using for code inspection.
Here is the logcat:
Fatal Exception: java.lang.NullPointerException
com.h6ah4i.android.media.opensl.OpenSLMediaPlayer.setDataSource (SourceFile:134)

The line 134 points here:
public void setDataSource(FileDescriptor fd) throws IOException, IllegalArgumentException, IllegalStateException {
        int nativeFD = checkAndObtainNativeFileDescriptor(fd);
        this.checkNativeImplIsAvailable();
        int result = setDataSourceFdImplNative(this.mNativeHandle, nativeFD); //line 134 where null pointer exception is thrown
        parseResultAndThrowException(result);
    }

Please have a look on the issue.
Looking forward to hear from you.
Best Regards

@hunnydarapstar Hi. Please look at the following diagram. Maybe, you are using setDataSource(Context context, Uri uri), and the NullPointerException was thrown because the passed context parameter was null.

https://github.com/h6ah4i/android-openslmediaplayer/blob/0.7.2/library/src/main/java/com/h6ah4i/android/media/opensl/OpenSLMediaPlayer.java#L134


Problem with running program

So, uh i downloaded it and i cant figure out which file to run to get the save editor open, could you tell me which folders to go into in the unzipped file and what to run to get save editor open?

You probably downloaded a ZIP of the repository (the source code).
You want to grab the latest saveedit-1.x.x.zip from releases.
https://github.com/gibbed/Gibbed.Borderlands2/releases/latest

Thank you gibbed!

Did that fix your issue?

yes thank you!


Angular material webpage html-css error

Bug

https://material.angular.io/components/sidenav/overview
on going through the above angular material's webpage , found that there is an overflow issue which can be solved by making the main div's display block from inline-flex
// previously
.docs-component-overview {
display: inline-flex;
}
// my solution
.docs-component-overview {
display: block;
}

just go to below link to see the error
https://material.angular.io/components/sidenav/overview

There's a seperate repository for the Material website, see https://github.com/angular/material.angular.io. You can report the issue there.

Closing, if you would like to open the issue in the material.angular.io repo it can be considered there.


[6.3.1] Missing lib folder

Hi, published package not working

I have the same issue. When I try to build with 6.3.0, it works fine. But with redux-form 6.3.1, I get:
Module not found: Error: Cannot resolve module 'redux-form'

Verified package.json is listing the following main config which doesn't exist:
https://github.com/erikras/redux-form/blob/master/package.json#L5
 5 |  "main": "./lib/index.js",
  ...

same, from my CI pipeline:
Error: Cannot find module './lib/immutable'
    at Function.Module._resolveFilename (module.js:455:15)
    at Function.Module._load (module.js:403:25)
    at Module.require (module.js:483:17)
    at require (internal/module.js:20:19)
    at Object.<anonymous> (/var/lib/jenkins/workspace/CICEXP/checkout/PRA/node_modules/redux-form/immutable.js:1:18)

also, thanks for your awesome lib :)

+1

+1

Any particular reason why the npm publish action happens outside of Travis? Had it been in Travis, the latest (broken) version wouldn't have been published.

travis failed for the tag build though
https://travis-ci.org/erikras/redux-form/builds/182395135

Exactly, Travis doesn't run publish tasks if the build fails or errors.

I also ran into this issue as well. You can just manually place the lib file in most cases from a previous build or install of the node module into the reduxform node module. That is working for me now until they have the repo updated for us.
Thanks for the great library guys it is very useful and saves me quite a bit of time!

+1

+1

GAH! I started using np to bump versions and publish. Will fix ASAP.

The Travis thing was because yarn failed. https://github.com/yarnpkg/yarn/issues/2199

Fixed in v6.3.2. Apologies!! 😳

Thank you @erikras!

Might be a different instance, but I get this when compiling examples/simple-form:
ERROR in ./src/index.js
Module not found: Error: Cannot resolve module 'redux-form' in /Users/EricFields/Development/redux-form/examples/simple/src
 @ ./src/index.js 15:17-38

ERROR in ./src/SimpleForm.js
Module not found: Error: Cannot resolve module 'redux-form' in /Users/EricFields/Development/redux-form/examples/simple/src
 @ ./src/SimpleForm.js 11:17-38


No Npcs after death

If my character was died near npcs, all npcs are disappear

same problem fix it please

@gonalez Could u fix it please, It a serious bug.

I have the same problem


Zims rebased ssh plus config support

This commit contains finally agreed SSH key support and support for global and local config
SSH key Support:
if device_key_path present in configuration file implementation will attempt to read the key
if key not read successfully password will be used (if presen)
if no key and no password, SCP and SSH will fail
Config support

config template and config file postfix specified in options
local and global config combined and sticked into gulp instance
gulp init task implemented

@aovechki @ZhijunZhao @xscript


Add resource about emoji search in Elasticsearch

Disclaimer: I'm the author.
This article explain how to allow search with emoji, like they do on Yelp for exemple.

cool! thank you, @damienalexandre!!!


Ciborium and Cleanups

Port this crate from cert_cbor to ciborium.
Move Jsonu64 module to its own file

Motivation
serde_cbor has been unmaintained forever now, we've avoided switching to ciborium because it would not support deserialization to a borrowed string, which serde_cbor did. However, I'd rather pass cargo deny advisories than continue with this.
We can fix any issues arising from this when we make mobilecoin.git use this crate.

Current dependencies on/for this PR:

main

PR #2   👈

This comment was auto-generated by Graphite.

this issue slightly worries me: https://github.com/enarx/ciborium/issues/51

Can we try adding some more structs here, like nested structs, and enums? The test coverage on this seems pretty light compared to the variety of ways in which we use it

this issue slightly worries me: enarx/ciborium#51

Trying to use the bytes they provide in that issue results in Err(Semantic(None, "missing field pubkey"))
    #[derive(PartialEq, Serialize, Deserialize, Debug, Clone)]
    struct S {
        pubkey: [u8; 32]
    }

    #[test]
    fn try_51() {
        extern crate std;
        let s = S {
            pubkey: [189, 182, 166, 93, 174, 37, 6, 247, 39, 203, 228, 101, 121, 197, 203, 42, 98, 138, 145, 12, 12, 76, 145, 168, 132, 185, 90, 18, 54, 28, 248, 170],
        };
        let serialized = serialize(&s).unwrap();

        // Data from https://github.com/enarx/ciborium/issues/51
        // Uncomment the lines to see the error
        // let data = vec![161, 102, 112, 117, 98, 75, 101, 121, 88, 32, 189, 182, 166, 93, 174, 37, 6, 247, 39, 203, 228, 101, 121, 197, 203, 42, 98, 138, 145, 12, 12, 76, 145, 168, 132, 185, 90, 18, 54, 28, 248, 170];
        // assert_eq!(format!("{r:?}"), "sushi");
        let data = vec![161, 102, 112, 117, 98, 107, 101, 121, 152, 32, 24, 189, 24, 182, 24, 166, 24, 93, 24, 174, 24, 37, 6, 24, 247, 24, 39, 24, 203, 24, 228, 24, 101, 24, 121, 24, 197, 24, 203, 24, 42, 24, 98, 24, 138, 24, 145, 12, 12, 24, 76, 24, 145, 24, 168, 24, 132, 24, 185, 24, 90, 18, 24, 54, 24, 28, 24, 248, 24, 170];
        assert_eq!(serialized, data);
        let r = ciborium::de::from_reader::<S, _>(&data[..]);
        assert_eq!(format!("{r:?}"), "sushi");
        assert_eq!(r.is_ok(), true);
        assert_eq!(r.unwrap(), s);

    }

@nick-mobilecoin This is ready to go if you could re-review (for changes).


make Xcode selection for CI deterministic

In CI runs the script ./scripts/swift-version.sh to select the version of Xcode used for swift compiles. For most situations it is near-deterministic, but the listing order depends on Spotlights ordering (so a black box to us), as well as what the last version of Xcode selected on the system was.
We should make this completely deterministic, either by being explicit on the Swift->Xcode mappings, or by skipping the currently selected check and making the ordering completely deterministic. I lean to making this explicit, as it will better point out failures in the setup.

Fixed in 942fd88.


Work around BoringSSL bitcode linker warning

When linking envoy-mobile to an iOS app that ships with bitcode, you get
these warnings from the linker:
ld: warning: Linker asked to preserve internal global: 'sk_CRYPTO_BUFFER_call_free_func'
ld: warning: Linker asked to preserve internal global: 'sk_X509_call_free_func'
ld: warning: Linker asked to preserve internal global: 'sk_X509_call_free_func'
ld: warning: Linker asked to preserve internal global: 'sk_X509_call_free_func'
ld: warning: Linker asked to preserve internal global: 'sk_X509_call_free_func'

I have submitted an upstream patch to fix this
https://boringssl-review.googlesource.com/c/boringssl/+/37804 in the
meantime I've applied that patch here.
Signed-off-by: Keith Smiley<EMAIL_ADDRESS>

https://github.com/lyft/envoy-mobile/issues/471

thanks @keith!

Lets hold on this until we have some clarity upstream about this fix

Going to update the upstream conversation but this is fine for now


Bundle the extension using Webpack

See: https://code.visualstudio.com/api/working-with-extensions/bundling-extension

Finished in 4ad26dd0837f4e578bfb7e0b3486ba00e40fb654


add optiono to subset by samples

Need to subset a Table by samples, so just adding the functionality here.

Does this work? I assume it works, but this is scripts/subset_hail_table.py - and Hail Tables are sites-only representations of VCFs (no sample columns)


fix: Run serverpod generate in newly created projects.

Changes
Run serverpod generate when creating new projects
Pre-launch Checklist

[ ] I read the Contribute page and followed the process outlined there for submitting PRs.
[ ] This update contains only one single feature or bug fix and nothing else. (If you are submitting multiple fixes, please make multiple PRs.)
[ ] I read and followed the Dart Style Guide and formatted the code with dart format.
[ ] I listed at least one issue that this PR fixes in the description above.
[ ] I updated/added relevant documentation (doc comments with ///), and made sure that the documentation follows the same style as other Serverpod documentation. I checked spelling and grammar.
[ ] I added new tests to check the change I am making.
[ ] All existing and new tests are passing.
[ ] Any breaking changes are documented below.

If you need help, consider asking for advice on the discussion board.
Breaking changes
If you have done any breaking changes, make sure to outline them here, so that they can be included in the notes for the next release.

Closing for now, still think this is a good idéa but we can add this in the future when we need it.


Evaluate exported pkgs against repo baseline nixos

The baseline pkgs for own consumption is nixos, only pkgs/override.nix
provides a mechanism to "backport" packages from unstable.
Therefore, we should expose our packages to others in the same
context as we use them ourselves.

I was about to open an issue, yet I figured it would be more helpful to directly propose a PR, instead. Maybe this needs more preliminary discussion, though.
See also #76 (part about master vs nixos overlays)

Proporsal together with #78
fix #76

Probably a good idea. Won't merge until we fully decide on a solution to #76.

Probably a good idea. Won't merge until we fully decide on a solution to #76.


SchematicTestRunner.runExternalSchematic fails with "The encoded data was not valid for encoding utf-8"

Command
other
Is this a regression?

[X] Yes, this behavior used to work in the previous version

The previous version in which this bug was not present was
17.0.0
Description
With v18.0.0-rc.1 under some conditions (which  exactly is unclear to me) SchematicTestRunner.runExternalSchematic fails with the following error: "The encoded data was not valid for encoding utf-8"
See e.g. https://github.com/daniel-sc/ng-extract-i18n-merge/actions/runs/9091527710/job/24986317152?pr=111
The exact same test runs fine with angular 17: https://github.com/daniel-sc/ng-extract-i18n-merge/actions/runs/9091527710/job/24986316954?pr=111
This is the line were it fails: https://github.com/daniel-sc/ng-extract-i18n-merge/blob/master/schematics/ng-add/index.spec.ts#L45 (unsure why the first invocation succeeds..)
Minimal Reproduction

Checkout branch angular-18 from https://github.com/daniel-sc/ng-extract-i18n-merge
remove package-lock.json
run npm i --save-exact "<EMAIL_ADDRESS>"<EMAIL_ADDRESS>"<EMAIL_ADDRESS>"<EMAIL_ADDRESS>"<EMAIL_ADDRESS>"<EMAIL_ADDRESS>"<EMAIL_ADDRESS>run npm run build
run npm run test-coverage

Exception or Error
The encoded data was not valid for encoding utf-8
TypeError: The encoded data was not valid for encoding utf-8
    at TextDecoder.decode (node:internal/encoding:449:16)
    at C:\projekte\ng-extract-i18n-merge\node_modules\@angular-devkit\schematics\src\rules\template.js:38:44
    at C:\projekte\ng-extract-i18n-merge\node_modules\@angular-devkit\schematics\src\rules\base.js:135:23
    at C:\projekte\ng-extract-i18n-merge\node_modules\@angular-devkit\schematics\src\rules\base.js:89:20
    at C:\projekte\ng-extract-i18n-merge\node_modules\@angular-devkit\schematics\src\rules\base.js:113:30
    at C:\projekte\ng-extract-i18n-merge\node_modules\@angular-devkit\schematics\src\tree\host-tree.js:278:13
    at C:\projekte\ng-extract-i18n-merge\node_modules\@angular-devkit\schematics\src\tree\host-tree.js:51:61
    at Array.forEach (<anonymous>)
    at HostDirEntry.visit (C:\projekte\ng-extract-i18n-merge\node_modules\@angular-devkit\schematics\src\tree\host-tree.js:51:43)
    at HostCreateTree.visit (C:\projekte\ng-extract-i18n-merge\node_modules\@angular-devkit\schematics\src\tree\host-tree.js:277:19)
    at C:\projekte\ng-extract-i18n-merge\node_modules\@angular-devkit\schematics\src\rules\base.js:109:14
    at callRuleAsync (C:\projekte\ng-extract-i18n-merge\node_modules\@angular-devkit\schematics\src\rules\call.js:77:24)
    at C:\projekte\ng-extract-i18n-merge\node_modules\@angular-devkit\schematics\src\rules\call.js:69:63
    at doInnerSub (C:\projekte\ng-extract-i18n-merge\node_modules\rxjs\src\internal\operators\mergeInternals.ts:71:15)
    at outerNext (C:\projekte\ng-extract-i18n-merge\node_modules\rxjs\src\internal\operators\mergeInternals.ts:53:58)
    at OperatorSubscriber._this._next (C:\projekte\ng-extract-i18n-merge\node_modules\rxjs\src\internal\operators\OperatorSubscriber.ts:70:13)
    at OperatorSubscriber.Object.<anonymous>.Subscriber.next (C:\projekte\ng-extract-i18n-merge\node_modules\rxjs\src\internal\Subscriber.ts:75:12)
    at C:\projekte\ng-extract-i18n-merge\node_modules\rxjs\src\internal\operators\mergeInternals.ts:85:24
    at OperatorSubscriber._this._next (C:\projekte\ng-extract-i18n-merge\node_modules\rxjs\src\internal\operators\OperatorSubscriber.ts:70:13)
    at OperatorSubscriber.Object.<anonymous>.Subscriber.next (C:\projekte\ng-extract-i18n-merge\node_modules\rxjs\src\internal\Subscriber.ts:75:12)
    at C:\projekte\ng-extract-i18n-merge\node_modules\rxjs\src\internal\observable\innerFrom.ts:90:24

Your Environment
Angular CLI: 17.3.3
Node: 20.12.0
Package Manager: npm 10.5.0
OS: win32 x64

Angular: 18.0.0-rc.1
... compiler-cli, core

Package                      Version
------------------------------------------------------
@angular-devkit/architect    0.1800.0-rc.1
@angular-devkit/core         18.0.0-rc.1
@angular-devkit/schematics   18.0.0-rc.1
@schematics/angular          18.0.0-rc.1
typescript                   5.4.5

Anything else relevant?
No response

This is caused by a long standing bug in Jest which causes instanceof not to work correctly.
See: https://github.com/jestjs/jest/issues/2549

@alan-agius4 thanks for the pointers!
It would be great if this could be adressed, as this breaks unit tests of schematics with jest and probably impacts some libraries..


WIP - Update glob to v10

Summary
I've spotted #384 and I thought I give it a try. Shakapacker currently depends on an older version of glob which has a dependency with known memory leak. The author of glob ditched that dependency aleady and made "Massive performance improvements." in v9. In v10 the only breaking change is that there are no default exports, but Shakapacker is not using that anyways, so I thought it's better to use the latest, maintained version of the package instead of pinning it to v9.
Though it seems to be an easy upgrade, it has some possible drawbacks: the new glob versions requires at least nodejs v16. Shakapacker currently supports ^12.13.0 || ^14 || >=16. The official support for v12 ended more than a year ago, for v14 it ended 7 months ago (reference), so I don't think it should cause a problem.
Pull Request checklist

[x] ~Add/update test to cover these changes~
[x] ~Update documentation~
[ ] Update CHANGELOG file

Other Information
Glob in dev dependencies
There are some dev dependencies that still require glob v7, but those shouldn't affect shakapacker users. As far as I read about these packages, they are going to upgrade glob as soon as they can drop nodejs 14, for example https://github.com/jestjs/jest/pull/14509
$> yarn why glob
yarn why v1.22.19
[1/4] Why do we have the module "glob"...?
[2/4] Initialising dependency graph...
[3/4] Finding dependency...
[4/4] Calculating file sizes...
=> Found<EMAIL_ADDRESS>info Has been hoisted to "glob"
info This module exists because it's specified in "dependencies".
info Disk size without dependencies: "1.21MB"
info Disk size with unique dependencies: "3.25MB"
info Disk size with transitive dependencies: "4.2MB"
info Number of shared dependencies: 21
=> Found "<EMAIL_ADDRESS>info This module exists because "jest#@jest#core#@jest#reporters" depends on it.
info Disk size without dependencies: "64KB"
info Disk size with unique dependencies: "204KB"
info Disk size with transitive dependencies: "300KB"
info Number of shared dependencies: 8
=> Found<EMAIL_ADDRESS>info This module exists because "jest#@jest#core#jest-config" depends on it.
info Disk size without dependencies: "64KB"
info Disk size with unique dependencies: "204KB"
info Disk size with transitive dependencies: "300KB"
info Number of shared dependencies: 8
=> Found<EMAIL_ADDRESS>info This module exists because "jest#@jest#core#jest-runtime" depends on it.
info Disk size without dependencies: "64KB"
info Disk size with unique dependencies: "204KB"
info Disk size with transitive dependencies: "300KB"
info Number of shared dependencies: 8
=> Found<EMAIL_ADDRESS>info This module exists because "jest#@jest#core#rimraf" depends on it.
info Disk size without dependencies: "64KB"
info Disk size with unique dependencies: "204KB"
info Disk size with transitive dependencies: "300KB"
info Number of shared dependencies: 8
=> Found<EMAIL_ADDRESS>info This module exists because "jest#@jest#core#@jest#transform#babel-plugin-istanbul#test-exclude" depends on it.
info Disk size without dependencies: "64KB"
info Disk size with unique dependencies: "204KB"
info Disk size with transitive dependencies: "300KB"
info Number of shared dependencies: 8

@kiskoza can you solve the JS test failures?

The issue with the CI was that I removed nodejs v14 from the supported engine versions, but the CI was configured to use that in the matrix. I removed v14 from the CI matrix.

This sounds good, but do we need to update the major version of shakapacker to require node 16+?
CC: @G-Rath

It looks like this is the only use of glob:
https://github.com/shakacode/shakapacker/blob/0f892759d39bc00c9afc6d2980c06556e73a603c/package/environments/base.js#L16-L25
Technically this includes user input via source_path and source_entry_path on the first line of that snippet which if it included a glob would be treated as such but I can't find any evidence of those ever expecting to have a glob and I would expect other things would break if you actually tried to use a glob in them.
As such, I think we could just remove glob entirely in favor of a recursive fs.readdirSync along the lines of:
const getSourceFiles = (rootPath, includeNested) => {
  if(!includeNested) {
    return fs.readdirSync(rootPath);
  }

  return fs.readdirSync(rootPath, { withFileTypes: true }).flatMap(dirent => {
    if (dirent.isDirectory()) {
      return getSourceFiles(join(rootPath, dirnet.name), true);
    }

    return dirnet.name;
  })
}

(that's very rough since I didn't want to steal @kiskoza's thunder, though I am happy to pick this up if folks want 🙂)
That would probably be a bit slower than glob but it shouldn't be horrible since glob has to be doing something similar to that, and Node v18/v20 added recursive so eventually we'd be able to greatly simplify that too.
That should result in addressing #384 without requiring dropping any node versions or a new major version (though @justin808 since we're expecting to do a new major sometime in next few months, I plan to talk about dropping support for Ruby 2.x and so it still might be worth dropping support for older versions of Node too, but we can talk about that separately)

The Node code is used only at build time, so I doubt a memory leak would affect much.
I'd say we defer until v8 and drop support for older Node.

I think that's fine on the short-term but long-term it still seems like a win to drop a dependency entirely so if anyone wants to pick that up I'm happy to review it - otherwise I'll pick it up sometime either after v8 or if it starts getting flagged by osv-scanner and dependabot

Bundler has dropped support for Ruby 2.6 and 2.7 in their recent 2.5.0 release. Since Bundler is a far more prominent package than Shakapacker, I think we can conclude from their practices that we can drop support for Node 14 without making a major release.
But also I think replacing glob with fs.readdirSync is a good option, if not a better one.

I think we can conclude from their practices that we can drop support for Node 14 without making a major release.

I can't find anything about their versioning that indicates why they didn't consider dropping the Ruby versions a breaking change but under traditional semver (which they might not be following) it is a breaking change, but I can say in the Node community at least dropping support for a version of Node is considered a breaking change - there can be some wiggle room such as if the versions are super old and a package doesn't have an explicit "engines" field, but overall I don't think this change is critical enough to be worth dropping that support especially when we're planning to do a new major soon anyway and we agree that the fs.readdirSync is the better path 🤷

Thanks for all your input on the topic. It feels like we shouldn't make this change yet as dropping support for older nodejs versions require a major version bump in the npm package, but this dependency is hardly used and the change is not big enough to be the reason to drop older nodejs, the original issue could be solved in other ways as well.
I'm happy to either leave it open or just close it as it's already referenced from the issue and could be reopened later when we need it

Is there any argument against switching to fs.readdirSync?

Been no movement on this one so opened my own stab for this at https://github.com/shakacode/shakapacker/pull/435


subscription.on message wasn't received of messages after 10 mins or more.

OS: Debian
Node.js version:  8.4.0
npm version:   5.4.0
google-cloud-node version:  0.56.0
@google-cloud/pubsub version: 0.14.0

Steps to reproduce
const pubsub = require ( '@google-cloud/pubsub' )( connectOption )

const subscription = pubsub.topic ( topicName ).subscription ( subScription )

subscription.on ('message', message => {
    message.ack()
    ....
})

subscription.once('error', err => {
    ....
})

I received all un-received messages when I restart it.
I checked the connectionPool object when it have not received message:
connectionPool.isPaused [false]
connectionPool.isOpen [true]
connectionPool.connections [5]
Thank you.

Thanks for reporting! We've had multiple reports of similar behavior and we're currently investigating this issue. In the meantime you might want to use pubsub v0.13.x as a temporary workaround.

Thank you.
I did a temp code to use the pubsub v0.14.0 like this:
const pubsub = require ( '@google-cloud/pubsub' )( connectOption )

const listenSub = () => {
    let subscription = pubsub.topic ( topicName ).subscription ( subScription )
    subscription.on ( 'message', message => {
        message.ack()
        ....
   })

    subscription.once('error', err => {
        ....
    })

    //        each 10 mins restart
    const tt = setTimeout (() => {
	subscription.close().then(() => {
		listenPubSubMaster = null
		return listenSub ()
    }, 600000 )
}
listenSub()

We see a similar behavior, although it takes a few hours. Eventually the subscription stops processing new messages and we have to restart the subscription. This only started after upgrading to 0.14.

I too am seeing the same problem.  We set up a forever running process on GKE and it seems to go unresponsive after some time.  I came to the same conclusion of putting a restarter.
Let's hope this gets resolved soon!

This is definitely a result of moving towards the new streaming API, I'm working on resolving this issue, but in the mean time I suggest downgrading to 0.13. I'll update this issue once it's been resolved.

@callmehiphop FYI, we resolved this in Go by adding gRPC keepalive. I'm not sure how the code should look like in Node though. https://code-review.googlesource.com/#/c/gocloud/+/17070/

@pongad thanks for the update! I believe we just have to set the grpc.keepalive_time_ms option.

@QTGate @mscottx88 @caseyduquettesc I've opened a PR (#2627) which I believe will fix the issue you are seeing, would any of you mind testing it out? Thanks!

I'm willing to give it a go, for sure!
On Mon, Sep 25, 2017, 7:17 AM Dave Gramlich<EMAIL_ADDRESS>wrote:

@QTGate https://github.com/qtgate @mscottx88
https://github.com/mscottx88 @caseyduquettesc
https://github.com/caseyduquettesc I've opened a PR (#2627
https://github.com/GoogleCloudPlatform/google-cloud-node/pull/2627)
which I believe will fix the issue you are seeing, would any of you mind
testing it out? Thanks!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/GoogleCloudPlatform/google-cloud-node/issues/2598#issuecomment-331895506,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AT6MMUjOHHVuq1ZzzlIViODrOF50HIecks5sl7X8gaJpZM4PQKSI
.

@callmehiphop I applied the two fixed files to my deployment but now I am seeing duplication of message delivery.  That is, I am seeing the same message being published more than one time.

@jganetsk I'm noticing that messages that are not acked before a stream is closed will be redelivered when the next stream is opened regardless of whether or not the ack deadline was hit. Is that the expected behavior? If so I'm assuming we need to filter these messages out.

@callmehiphop I don't think you should be filtering any messages out. In general, it is possible to receive duplicates, and it is possible that they come before the ack deadline was hit. If this happens consistently, that's a problem that we should look at. How much time passes between delivery and redelivery? Does it correspond to the default ack deadline?

We will take a look at this, thanks for reporting it.

@jganetsk as far as I can tell it only happens when closing and re-opening a stream. I'm putting together an elaborate test and I'll give you the details of my findings once I have some to share.

There appears to be 2 different scenarios in which messages will be redelivered before they have been acked.
Streams being closed.
In the event of a stream being closed it appears that all un-acked messages will be redelivered to a different stream regardless of the current ack deadline.

Test 1: Set the initial streamAckDeadlineSeconds to 60 (1 minute) and close a stream after 30 seconds. Result: All messages were redelivered despite not hitting the deadline.
Test 2: Modify the deadline just before closing the stream. Result: All messages were redelivered even though the deadline was successfully modified.

Race Condition
I believe we occasionally hit a race condition when attempting to modify an existing ack deadline. Currently we calculate the ack deadline using the 99th percentile of ack times (p99). We then calculate a delay to determine when we should modify the ack deadline (Math.random() * n99 * 0.9).
If our p99 value is 10, then it is possible we will not modify the ack deadline until roughly 1 second before it hits. The redelivery cases I have observed occurred when the delay was within 1-2 seconds of the p99 value.
@lukesneeringer have you seen anything similar to what I'm describing above in the Python client? If this is indeed the reason why a redelivery occurs, is it reasonable to calculate the sleep time at a lower percent?

Here is my solution to the race condition you describe.  A simple activeMessages tracking mechanism, combined with async shutdown that is event-driven.
`'use strict';
const config = require('@google-cloud/rcloadenv');
const cp = require('child_process');
const fs = require('fs-extra');
const logger = require('winston');
const pubsub = require('./providers/pubsub');
const uuid = require('uuid/v4');
const { name: configurationStorageName } = require('../package.json');
/**

Listen for and process incoming messages.

@returns {Function}
An async function that can be called to stop listening for messages.
*/
const listen = () => {
let activeMessages = 0;
logger.info(Subscription started at ${new Date().toISOString()}.);

const subscription = pubsub
.get()
.topic(process.env.SUBSCRIBE_TOPIC_NAME)
.subscription(process.env.SUBSCRIBE_SUBSCRIPTION_NAME, {
flowControl: {
maxMessages: +process.env.MAX_CONCURRENT_MESSAGES
}
});
subscription.on('error', (error) => {
logger.error(error);
});
subscription.on('message', (message) => {
const { email, pwd, uid } = message.attributes;
logger.info(Message received at ${new Date().toISOString()} for uuid ${uid}.);
const sessionId = uuid();
const downloadPath = `${__dirname}/../downloads/${sessionId}`;
const profilePath = `${__dirname}/../profiles/${sessionId}`;

const args = [
  uid,
  email,
  pwd,
  sessionId,
  downloadPath,
  profilePath
];

const options = {
  cwd: process.cwd(),
  env: process.env,
  execArgv: [],
  silent: true
};

logger.info(`Starting session ${sessionId}`);
const downloader = cp.fork(`${__dirname}/download.js`, args, options);
activeMessages++;

downloader.stdout.on('data', (data) => {
  const text = new Buffer(data).toString('utf8');
  logger.info(text);
});

downloader.on('exit', async (code) => {
  logger.info(`Session ${sessionId} completed with exit code ${code}.`);

  try {
    await Promise.all([fs.remove(downloadPath), fs.remove(profilePath)]);
  } catch (error) {
    logger.error(`Could not cleanup path(s) in session ${sessionId} because of ${error}.`);
  }

  try {
    message.ack();
  } finally {
    activeMessages--;
  }
  if (activeMessages === 0) {
    subscription.emit('importer-idle');
  }
});

});
return () => {
return new Promise((resolve, reject) => {
// stop the flow of incoming messages
subscription.removeAllListeners('message');
  const shutdown = async () => {
    try {
      await subscription.close();
      logger.info(`Subscription ended at ${new Date().toISOString()}.`);
      resolve();
    } catch (error) {
      reject(error);
    }
  };

  if (activeMessages !== 0) {
    subscription.on('importer-idle', shutdown);
  } else {
    shutdown();
  }
});

};
};
/**

Run the process.

@returns {void}
*/
const run = async () => {
logger.level = process.env.LOG_LEVEL;

try {
await config.getAndApply(configurationStorageName);
} catch (error) {
logger.error(error);
throw error;
}
let unlisten = listen();
setInterval(async () => {
try {
await unlisten();
} catch (error) {
logger.error(error);
} finally {
unlisten = listen();
}
}, +process.env.RESTART_INTERVAL);
};
module.exports = {
run
};
`

OS: Debian
Node.js version:  8.7.0
npm version:   5.5.1
@google-cloud/pubsub version: 0.14.5

Steps to reproduce
const pubsub = require ( '@google-cloud/pubsub' )( connectOption )

const listenSub = () => {
    let subscription = pubsub.topic ( topicName ).subscription ( subScription )
    subscription.on ( 'message', message => {
        message.ack()
        ....
   })

    //        each 10 mins restart
    const tt = setTimeout (() => {
	return subscription.close().then(() => {
	    listenPubSubMaster = null
	    return listenSub ()
    }, 600000 )
}

listenSub()

I did make new subscription each 10 mins.
But looks the @google-cloud/pubsub not release memory when pubsub exit with subscription.close().

OS: Debian

Node.js version: 6.11.1

npm version: 3.10.10

version: 0.14.5

I am also seeing the memory increase and not being released after implementing the subscriber refresh.

@callmehiphop will this be addressed by #2716 or another issue/PR you've already worked on?

No I am not. Even the new v0.14.6. Also have memory problem. I plan do subscription.on gCloud API's by my self.

@callmehiphop will this be addressed by #2716 or another issue/PR you've already worked on?

I believe it will!

I am using PubSub node.js client version 0.14.5 and I sometimes have CloudFunctions that when triggered are no longer able to add messages to a topic or pull messages from a subscription. The fix is to redeploy the CloudFunction which mostly works but not always. The issue is intermittent.

Apologies for asking the obvious...!
But is this now fixed as #2716 has been released with 0.14.8?
Cheers!

@rhodgkins which parts? I think there are a number of issues being described here.

@callmehiphop the issue around no messages being received on a subscription after a period of time.

@rhodgkins from what I understand the only environment in which this occurs is GKE, but I also know a fix was deployed on the GKE side of things that should have resolved it. I haven't seen or heard of any reports verifying this though.

This issue was moved to googleapis/nodejs-pubsub#10

Recently we start experimenting the problem using version 0.13.0, Does anybody has this problem?


Add Filtering Logic

For support of STEVE filtering in the NCHS VRDR Test Plan.  Similar to the same for C# library:
https://github.com/nightingaleproject/vrdr-dotnet/issues/397

From discussion with STEVE, the filtering is only needed for NAPHSIS and thus will only be part of the C# library and is not needed for the jurisdiction-facing Java library.


pivx: init at 3.1.1

Motivation for this change
Having another altcoin ready for usage.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

@GrahamcOfBorg build pivx

@joachifm try the following:
@GrahamcOfBorg build altcoins.pivx

@joachifm ping. Also updated to 3.2.0.

@GrahamcOfBorg build altcoins.pivx


Floating number validation

What kind of issue is this? (put 'x' between the square brackets)

[ ] Question. This issue tracker is not the place for questions. If you want to ask how to do
something, or to understand why something isn't working the way you expect it to, use
http://stackoverflow.com/questions/ask .
Provide working code, starting from http://codepen.io/marcandre/pen/jqbzyN?editors=101.
We monitor the tag parsley.js.

[x] Bug report. If you’ve found a bug, you must provide a minimal example in a CodePen,
starting from http://codepen.io/marcandre/pen/jqbzyN?editors=101 .

[ ] Feature Request. Make sure there's no good way to do what you want first;
consider asking on http://stackoverflow.com/questions/ask first.

--
I think I found a bug with floating number validation. Here is a CodePen to illustrate it:
http://codepen.io/cgrimal/pen/jApoyO?editors=1010
The data-parsley-type="number" is not validating any number input with a .
I read the docs, and tried many things but I can't make it work.
One thing that may help you: it works if you use release 2.2.0-rc4 and the problem starts with release 2.2.0.
I'll use 2.2.0-rc4 for the time being but I'm willing to understand if it is a genuine bug of the lib or me!
Thanks guys,
Clément

Mmm, right, the issue is with the way parameters are passed. If you use parsley-type-min='0.1' (since it's the min parameter to the type validator you want to specify) it'll work as expected.
I'm too jetlagged to think what is the perfect solution to address this in general. If I'm not mistaken, <input type="text" step="0.1"> is not valid HTML5, so I'm not sure we should support it.

PS: Thanks for the CodePen :smile:

Merci @marcandre ! (I'll stick to english even though I think we're both french)
I added the data-parsley-min="0.1" attribute to my input but it still doesn't accept any float value. I also remove the step="0.1" attribute but I don't understand the "Number" section of the documentation where it says:

Validates that a value is a valid number according to the given step, min and original value.

Anyway, that's not my real issue, so probably not worth looking too much into it.
There is no rush so please, take your time to recover from the jet-lag :sunglasses:, I'm happy with 2.2.0-rc4 for now!

It's data-parsley-type-min, not data-parsley-min

BTW, latest is v2.4.3

Has there been any closure on this issue? I can't seem to validate decimals/floating numbers either.
<input data-parsley-type="number" required type="text" />

Results in invalid value errors. I'm using version 2.4.4

Ok, so here's the deal.
One can use type="number" step="..."  ..., with all the caveats. This will be "mapped" so that it behaves as data-parsley-type="number" data-parsley-type-step="..." ...
Note the data-parsley-type-step attribute here... Indeed, all arguments to data-parsley-validatornamehere have form data-parsley-validatornamehere-heresanargument.
So it's normal that <input data-parsley-type="number" required type="text" value="0.1"/> won't validate, as per spec, even with step="..."; the only time where the step attribute is looked at is with the automatic mapping of HTML5 attributes ike type="number".
<input data-parsley-type="number" data-parsley-type-step="0.01" required type="text" value="0.1"/> or similar will validate.
Does this answer everyone's questions here?
Suggestions, or help with documentation / examples / anything always appreciated

The only thing that needs to happen is when I specify that I want something validated as a number, however parsley wants to a define a number, which as I see it, should be anything that returns !isNaN(foo) == true.
I shouldn't be worrying about other specs that aren't part of the parsley library interrupting things.
Would I be able to overwrite a built in validator? I'll just overwrite the data-parsley-type="number" validator with:
window.Parsley
  .addValidator('number', {
    requirementType: 'number',
    validateString: function(value) {
      return !isNaN(value);
    },
    messages: {
      en: 'This value should be a number.'
    }
  });

@danhardman  Please realize that "1.e-9" is a "number", according to your definition, even though for most uses it's would not be acceptable.
Overwriting builtin validators is not supported.
You are free to want to completely ignore HTML5 validators & related specs, but that has not been Parlsey's stance, although we do deviate from the standard from time to time (e.g. pattern).
I think that if the default step was 'any' for data-parsley-type="number", it would mostly get out of your way, no?

@marcandre 1.e-9 is a perfectly fine number, I don't see the problem. If I wanted only [0-9] values, I'd use the digits validator or integer.
Would you be able to give an example of where that behaviour wouldn't be desired?
I just find it very confusing that data-parsley-type="number" doesn't work for all numbers. There's too many assumptions being made

Would you be able to give an example of where that behaviour wouldn't be desired?

I'd venture that most numbers entered are prices. I can think of GPAs, lab results in the medical field, etc.

@marcandre The exact reason I'm having this issue is because validation on price values isn't working because it doesn't properly support decimals.
If you don't want to allow decimals in validation, that should be the hoop to jump through, rather than trying to convince parsley that a number is in fact a number.
As I said, if I wanted whole numbers, I'd use data-parsley-type="digits" or data-parsley-type="integer". That makes much more sense.

For price values, it is much better to specify data-parsley-type-step="0.01", so 1.23 is a valid price but 1.234 isn't, don't you think?

I validate back-end where it matters and values get treated as 2.d.p. If someone wants to be funny, they can be, it'll only round the value up/down and display the value as it should be.
Point still stands, 1.234 should be considered valid by data-parsley-type="number" and if I want to limit to 2.d.p, I should have to do something extra.

So I conclude that you are :+1: on changing the default step from 1 to 'any' in case of data-parsley-type="number". I won't change the default step for <input type=number> though.

@marcandre Ah, yeah sorry that kinda got lost in everything.

Default changed in master, will release soon.