Error: Produced invalid attestation for slot N: Signature is invalid

Seen this on three separate occasions on the Witti network today:
2020-06-21 00:51:28.318+00:00 | ValidatorApiChannel-0 | INFO  | teku-validator-log | ^[[34mValidator   *** Published attestation  Count: 1, Slot: 180257, Root: e029d2..8b75^[[0m
2020-06-21 00:51:28.342+00:00 | ValidatorApiChannel-0 | ERROR | teku-validator-log | ^[[31mValidator   *** Produced invalid attestation for slot 180257: Signature is invalid^[[0m
2020-06-21 00:51:32.323+00:00 | ValidatorApiChannel-0 | INFO  | teku-validator-log | ^[[34mValidator   *** Published aggregate    Count: 1, Slot: 180257, Root: e029d2..8b75^[[0m

This looks a bit weird.
It's possibly related to a large number of warnings in my logs about attestations with invalid signatures:
2020-06-21 01:14:12.409+00:00 | SlotEventsChannel-3 | WARN  | StateTransition | State Transition error
tech.pegasys.teku.core.exceptions.BlockProcessingException: tech.pegasys.teku.core.exceptions.BlockProcessingException: Invalid attestation: Signature is invalid
        at tech.pegasys.teku.core.StateTransition.initiate(StateTransition.java:87) ~[teku-ethereum-core-0.11.5-SNAPSHOT.jar:0.11.5-dev-d95fc0d1]
        at tech.pegasys.teku.core.ForkChoiceUtil.on_block(ForkChoiceUtil.java:224) ~[teku-ethereum-core-0.11.5-SNAPSHOT.jar:0.11.5-dev-d95fc0d1]
        at tech.pegasys.teku.statetransition.forkchoice.ForkChoice.onBlock(ForkChoice.java:65) ~[teku-ethereum-statetransition-0.11.5-SNAPSHOT.jar:0.11.5-dev-d95fc0d1]
        at tech.pegasys.teku.statetransition.blockimport.BlockImporter.importBlock(BlockImporter.java:70) ~[teku-ethereum-statetransition-0.11.5-SNAPSHOT.jar:0.11.5-dev-d95fc0d1]
        at tech.pegasys.teku.sync.BlockManager.importBlock(BlockManager.java:122) ~[teku-sync-0.11.5-SNAPSHOT.jar:0.11.5-dev-d95fc0d1]
...
Caused by: tech.pegasys.teku.core.exceptions.BlockProcessingException: Invalid attestation: Signature is invalid
        at tech.pegasys.teku.core.BlockProcessorUtil.verify_attestations(BlockProcessorUtil.java:399) ~[teku-ethereum-core-0.11.5-SNAPSHOT.jar:0.11.5-dev-d95fc0d1]
        at tech.pegasys.teku.core.blockvalidator.SimpleBlockValidator.validatePreState(SimpleBlockValidator.java:78) ~[teku-ethereum-core-0.11.5-SNAPSHOT.jar:0.11.5-dev-d95fc0d1]
        at tech.pegasys.teku.core.blockvalidator.BatchBlockValidator.validatePreState(BatchBlockValidator.java:43) ~[teku-ethereum-core-0.11.5-SNAPSHOT.jar:0.11.5-dev-d95fc0d1]
        at tech.pegasys.teku.core.blockvalidator.BlockValidator.validate(BlockValidator.java:88) ~[teku-ethereum-core-0.11.5-SNAPSHOT.jar:0.11.5-dev-d95fc0d1]
...

I have dozens of these mesages throughout the day. I don't know how invalid attestations are getting into blocks. In any case, there seem to be many badly signed attestations floating around, and we need to make sure we don't try to build them into aggregates.

Observation:
Validator produced attestation for the slot 68550
20:47:21.105 INFO  - Slot Event  *** Slot: 68549, Block: c2039e..2c6e, Epoch: 2142, Finalized checkpoint: 2140, Finalized root: 64cacf..464c, Peers: 54
20:47:29.264 INFO  - Slot Event  *** Slot: 68550, Block:    ... empty, Epoch: 2142, Finalized checkpoint: 2140, Finalized root: 64cacf..464c, Peers: 54
produceAttestationsForCommittee: 68550, 4
Invalid attestation: Attestation{aggregation_bits=000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000, data=AttestationData{slot=68550, index=4, beacon_block_root=0xc2039e84736babf548cdcda96dda143c71c91d075cc259faee2d4caaa56a2c6e, source=Checkpoint{epoch=2141, root=0xec7ab50c51657e2122cb0505ed0449d79cd3850c20e5f08647760cbfdd358af3}, target=Checkpoint{epoch=2142, root=0x46fa9b5dff989007afaf64ef289f8558d8fe9dc7606ae0449291e6675abb3e15}}, signature=0x84f5d612fbbe92b2fe22096dbfe50f97fd5d5fa099b4085e8899a2bce4bab889efa01ca7cfb85fea334de13e4184c17106eab14d5949ca13f3a54b10d8fc3bbedb634ef63e946ad412170d369da27a3db0410b27e62670e2d29bb22559314e4c}
20:47:29.467 ERROR - Validator   *** Produced invalid attestation for slot 68550: Signature is invalid

While API returns the 68549 slot for validator attestation:
{
  "epoch": "2142",
  "pubkeys": [
    "b6e880d58ec0d07a1490e65d3e38820d8dff3f093da8dacbb585cf884c403051def1544b7cda27870b90be4c8e53fea0"
  ]
}

[
  {
    "validator_pubkey": "0xb6e880d58ec0d07a1490e65d3e38820d8dff3f093da8dacbb585cf884c403051def1544b7cda27870b90be4c8e53fea0",
    "aggregator_modulo": 8,
    "validator_index": 1974,
    "attestation_committee_index": 2,
    "attestation_committee_position": 59,
    "block_proposal_slots": [],
    "attestation_slot": "68549"
  }
]

Report of this "in the wild" on the Altona testnet.

As @benjaminion pointed out I experienced this bug on the altona testnet. Find attached the log file of my node.
teku.log

One interesting thing to note is that all of these signatures are invalid because the expected validator index for all these attestations is all sorts of different validator indices, yet the actual attestation validator index is always either 1266 or 1267.

One interesting thing to note is that all of these signatures are invalid because the expected validator index for all these attestations is all sorts of different validator indices, yet the actual attestation validator index is always either 1266 or 1267.

@cemozerr if we are producing an attestation for wrong slot (as I mentioned above) then the attester index would be calculated incorrectly. Am I following correctly?

Another observation: ValidatorDuties.Duties instance is created with invalid attestationSlot. This time the slot was not previous to the correct one but a dozen of slots away.
Will continue observations...

https://github.com/PegaSysEng/teku/blob/b895c2553d3b5ee0c5c1876eecde339ce6a6c433/validator/coordinator/src/main/java/tech/pegasys/teku/validator/coordinator/ValidatorApiHandler.java#L124-L129
Can it be the reason:
when we are calculating new duties (I guess this is done on epoch boundary) the latest block N in the previous epoch is not yet processed, so the getLatestStateAtSlot() returns us the state for the block N-1. We are getting the state N' via processing slots on top of the state N-1 and submitting validator duty. After that the block N is finally received, the head state rebranches to N but the validator duty is not get rebranched and remains scheduled on a wrong head as if no block N exists.

Can it be the reason:

I think that is unlikely because looking at the logs, this issue happens 18 epochs in a row, which means that we'd have to have the scenario you've described 18 epochs in a row.  However, throughout those 18 epochs, we also don't seem to have the latest block at the correct time of the slot, so maybe that was still a possibility.

I caught some cases with more info, below is one of them with debug info and my comments:
00:48:52.152 INFO  - Slot Event  *** Slot: 127357, Block: d9a87d..53d9, Epoch: 3979, Finalized checkpoint: 3977, Finalized root: ba9b44..c304, Peers: 40
00:49:04.159 INFO  - Slot Event  *** Slot: 127358, Block:    ... empty, Epoch: 3979, Finalized checkpoint: 3977, Finalized root: ba9b44..c304, Peers: 40
00:49:16.176 INFO  - Slot Event  *** Slot: 127359, Block:    ... empty, Epoch: 3979, Finalized checkpoint: 3977, Finalized root: ba9b44..c304, Peers: 40
00:49:19.381 WARN  - Eth1 service down for 4621s, retrying
00:49:24.127 INFO  - Epoch Event *** Epoch: 3980, Justified checkpoint: 3978, Finalized checkpoint: 3977, Finalized root: ba9b44..c304
// we producing state for slot 127360 on top of existing head state for block 127358 by making slot transitions
// Our state for slot 127358 is the canonical state https://beaconcha.in/block/127358
ValidatorApiHandler.getDuties (line 128): latestState: 0x4c474059e2f9917d76f8192d9d08f28ffb3134d78a6887cbbb17479d4ce9cf39, 127358, for slot: 127360
// we are creating duties based on the state which is not canonical  (0xd5a8b1): https://beaconcha.in/block/127360
ValidatorApiHandler.createValidatorDuties (line 303): committee=[26911, 10421, 5801,... , 11251, 21333, 1348], 3,127419
    state.hashTreeRoot()=0xd5a8b1fa71129ce95d494394ca057d3bbdf94c22a50a0018437b310fb11a8009
    epoch=3981, validatorIndex=1974
ValidatorApiDutyLoader.scheduleAttestationDuties: 127419, 1974
new AttestationProductionDuty: 127419
AttestationProductionDuty.addValidator: validatorIndex=1974, validatorPubKey=0xb6e880d58ec0d07a1490e65d3e38820d8dff3f093da8dacbb585cf884c403051def1544b7cda27870b90be4c8e53fea0
00:49:30.036 INFO  - Slot Event  *** Slot: 127360, Block: 53a13e..2b84, Epoch: 3980, Finalized checkpoint: 3978, Finalized root: 2aeb78..dfb3, Peers: 40
00:49:40.154 INFO  - Slot Event  *** Slot: 127361, Block: c1194b..51ee, Epoch: 3980, Finalized checkpoint: 3978, Finalized root: 2aeb78..dfb3, Peers: 40
{3=tech.pegasys.teku.validator.client.duties.AttestationProductionDuty$Committee@6d0683e2}
00:49:49.383 WARN  - Eth1 service down for 4651s, retrying
00:49:52.162 INFO  - Slot Event  *** Slot: 127362, Block: 805dc9..3830, Epoch: 3980, Finalized checkpoint: 3978, Finalized root: 2aeb78..dfb3, Peers: 40

[... skipped ...]

01:01:16.153 INFO  - Slot Event  *** Slot: 127419, Block:    ... empty, Epoch: 3981, Finalized checkpoint: 3979, Finalized root: 84ddf4..49f5, Peers: 40
{3=tech.pegasys.teku.validator.client.duties.AttestationProductionDuty$Committee@f85eb7f}
01:01:16.275 ERROR - Validator   *** Produced invalid attestation for slot 127419 with expected validator index: 12806 and actual validator index: 1974 . Invalid reason: Signature is invalid

So looks like we generated duties based on the non-canonical state and didn't regenerate them after reorg

Another observation:
15:12:52.140 INFO  - Slot Event  *** Slot: 131677, Block: 697e08..85b9, Epoch: 4114, Finalized checkpoint: 4112, Finalized root: d0cf61..6ee8, Peers: 40
15:12:56.580 WARN  - Eth1 service down for 56438s, retrying
Importing block: SignedBeaconBlock{message=BeaconBlock{root=0x228faffc928c95c614753f970d030ea1dd1b7432ceeaab0fb5b2621fba25c371, slot=131678, proposer_index=26568, parent_root=0x697e0886f8bdc04091dc36e0a8e0a88c2674b10e544b3a75d5e64090d42385b9, state_root=0x171c9aeb6462ceaa11904a85fdb12ca4067dd0645f9c16226c538cdf12d471a7, body=0x95ca4da4e6c9e677d3835d8cba62348660c54e3b62bbe152a2035f1a6f944c86}, signature=0x892e07056e3b8159a0a2264c5243fd7101eb7dd6e58622ebb6a4b24a81a16aba385d46cba39c43f5ab8e96b310ec1b5c0e28ad9dc1215a5152e87efc5bc2e78ebea6342e5210d9d8661d13584d1b3c45d48a63fa723f75b042435f458c4e9299}
Importing block: SignedBeaconBlock{message=BeaconBlock{root=0x228faffc928c95c614753f970d030ea1dd1b7432ceeaab0fb5b2621fba25c371, slot=131678, proposer_index=26568, parent_root=0x697e0886f8bdc04091dc36e0a8e0a88c2674b10e544b3a75d5e64090d42385b9, state_root=0x171c9aeb6462ceaa11904a85fdb12ca4067dd0645f9c16226c538cdf12d471a7, body=0x95ca4da4e6c9e677d3835d8cba62348660c54e3b62bbe152a2035f1a6f944c86}, signature=0x892e07056e3b8159a0a2264c5243fd7101eb7dd6e58622ebb6a4b24a81a16aba385d46cba39c43f5ab8e96b310ec1b5c0e28ad9dc1215a5152e87efc5bc2e78ebea6342e5210d9d8661d13584d1b3c45d48a63fa723f75b042435f458c4e9299}
Imported block: SignedBeaconBlock{message=BeaconBlock{root=0x228faffc928c95c614753f970d030ea1dd1b7432ceeaab0fb5b2621fba25c371, slot=131678, proposer_index=26568, parent_root=0x697e0886f8bdc04091dc36e0a8e0a88c2674b10e544b3a75d5e64090d42385b9, state_root=0x171c9aeb6462ceaa11904a85fdb12ca4067dd0645f9c16226c538cdf12d471a7, body=0x95ca4da4e6c9e677d3835d8cba62348660c54e3b62bbe152a2035f1a6f944c86}, signature=0x892e07056e3b8159a0a2264c5243fd7101eb7dd6e58622ebb6a4b24a81a16aba385d46cba39c43f5ab8e96b310ec1b5c0e28ad9dc1215a5152e87efc5bc2e78ebea6342e5210d9d8661d13584d1b3c45d48a63fa723f75b042435f458c4e9299}
15:13:04.152 INFO  - Slot Event  *** Slot: 131678, Block: 228faf..c371, Epoch: 4114, Finalized checkpoint: 4112, Finalized root: d0cf61..6ee8, Peers: 40
{5=tech.pegasys.teku.validator.client.duties.AttestationProductionDuty$Committee@6a26fb29}
// No yet block 131679 so the head state is generated atop of block 131678
15:13:16.153 INFO  - Slot Event  *** Slot: 131679, Block:    ... empty, Epoch: 4114, Finalized checkpoint: 4112, Finalized root: d0cf61..6ee8, Peers: 40
Importing block: SignedBeaconBlock{message=BeaconBlock{root=0xb37865ebafd4b4620be6042901108a562ca78ce5b70aaf7db3c5e573c64a20b7, slot=131679, proposer_index=4950, parent_root=0x228faffc928c95c614753f970d030ea1dd1b7432ceeaab0fb5b2621fba25c371, state_root=0x7401873c0412e6adc15c2d35a317af41d6d27a983398abfcb8c5c7d850b8fc96, body=0x7b17976b56409d7ea393b425d04c784d3dda372013c61fbef2af569a0314aba9}, signature=0xa042e4e713bd251dcb6bcb0952008dc3d35b93e63af2ee7a3b23e63eee517c727138d539eb0283121506e46fd880d5a20a314230bc7ee77e74c7ae32a22cd2e800b9873fa8d6f42fd35e71c72cf492d659a82706468184a13a9be2beb54872fc}
// received and imported block 131679
Imported block: SignedBeaconBlock{message=BeaconBlock{root=0xb37865ebafd4b4620be6042901108a562ca78ce5b70aaf7db3c5e573c64a20b7, slot=131679, proposer_index=4950, parent_root=0x228faffc928c95c614753f970d030ea1dd1b7432ceeaab0fb5b2621fba25c371, state_root=0x7401873c0412e6adc15c2d35a317af41d6d27a983398abfcb8c5c7d850b8fc96, body=0x7b17976b56409d7ea393b425d04c784d3dda372013c61fbef2af569a0314aba9}, signature=0xa042e4e713bd251dcb6bcb0952008dc3d35b93e63af2ee7a3b23e63eee517c727138d539eb0283121506e46fd880d5a20a314230bc7ee77e74c7ae32a22cd2e800b9873fa8d6f42fd35e71c72cf492d659a82706468184a13a9be2beb54872fc}
15:13:24.122 INFO  - Epoch Event *** Epoch: 4115, Justified checkpoint: 4113, Finalized checkpoint: 4112, Finalized root: d0cf61..6ee8
// however duties are generated atop of block 131678 state
getDuties: latestState: 0x171c9aeb6462ceaa11904a85fdb12ca4067dd0645f9c16226c538cdf12d471a7, 131678, for slot: 131680
createValidatorDuties: [28564, 18255, 13027, 28049, 28882, 5841, 17508, 17385, 18968, 7343, 9556, 5601, 15378, 17430, 27745, 21579, 27064, 27754, 18023, 26646, 1974, 16423, 2511, 17909, 27172, 1274, 11436, 15386, 15669, 3268, 24260, 14457, 23294, 16044, 8446, 23678, 24245, 19756, 12978, 5184, 10114, 8194, 18569, 9684, 11696, 28209, 13353, 28788, 5650, 3997, 2923, 7594, 212, 141, 677, 21904, 18779, 27142, 27635, 29636, 20206, 3561, 19162, 13620, 28541, 27479, 13404, 20537, 2901, 1877, 12216, 4383, 19886, 25574, 26949, 10175, 15285, 759, 6586, 22340, 527, 21075, 12752, 28275, 13240, 1628, 9388, 12948, 18055, 25960, 19590, 14020, 6303, 12079, 17491, 2569, 8369, 23455, 18440, 3439, 12149, 29464, 12420, 9634, 14406, 28475, 27906, 1249, 20128, 20023, 20602, 22078, 125, 6137, 506, 18334, 8639, 902, 19846, 13630, 16178, 8324, 2300, 21701, 1436, 26409, 875, 5084, 26550, 4988, 10886, 27302, 11334], 3,131718
0x7939d93bcf43c96178419ca48dbe7774f6d8d98dcc489ca5f2fe0ba2efa46303
4116, 1974
scheduleAttestationDuties: 131718, 1974
new AttestationProductionDuty: 131718

I have a thesis: We don't receive/process the latest block for a certain epoch at the time of processHead() (i.e. at SlotProcessor.processSlotAttestation()). So our chainHead stays set for the latest block - 1 during the slot in which we will produce duties. After processHead() is completed, during the same slot, we do receive the block for the latest slot, and thus ProtoArray gets updated with the block for the slot. No issues yet. Later on, when we process the head again, we should know that actually our chainHead is now different for the previous slot where we calculated duties. But we don't, because the hasReorgedFrom method looks into the block root at the latest slot using getBlockRootBySlot, which a few levels down, uses ProtoArray to get the block root of the slot. So from getBlockRootBySlot, the block, which from chainHead updating side was empty, was always there at the right time, and there were no reorgs! Yay!

I have a feeling that the best way to solve this would be to make the current chainHead variable in RecentChainData redundant and use ProtoArray to get the chainHead information in constant time.

@cemozerr Yep this is exactly what I see in debugger now. Waiting for your potential fix to confirm 👍

From my perspective this should be definitely treated as chain reorg.
[Block 39] --> [40]
     \-------> [Block 40] --> [Block 41]

So in this example our previous head state was [40] (i.e. a state with just slot transition from [Block 39] state), but then the head has changed to the state [Block 40] (slot transition + block) and then to [Block 41]

Reopening this issue as it seems to have started occurring again for each validator once every 24 hours on my Altona node. Interestingly, it only happens at midnight.

Closing this as the error I saw on my Altona node was due to the failed updating of my Teku node, and with the new version I haven't been seeing this error again.


Add files via upload

yes, it makes sense. If you don't mind, Ramiro, try to take the whole dataframe as an input.


Rules not generated in order

I was expecting that the rules were generated in order of the yaml file.  I noticed that if my first three rules were "for_each" rules, the fourth rule, the first non-for_each rule was first in the output followed by the for_each rules.  Is this expected behavior?

Right now there's no provision for generating rules in order of appearance. I'm going to close this as a duplicate of #14 but I'd welcome patches to make this work.


CCLabel is blur when zoom up.

It seems CCLabel is rendered by pixels.
Is there any way to look it sharp when zomming up?

I think this problem is related to size of CCLabel's pixels is affected by parent.


Normalize event view names like Laravel

EventNode::triggerLaravelEvents() triggers events with the given template name, not the normalized name. Laravel (in ManagesEvents::addViewEvent()) saves events with the normalized name.
In my project I use / delimiters everywhere, because it's much more readable than Laravel's ., and that works (in Laravel and in Twig) because names are normalized. Except in custom triggering events since the Twig 3 upgrade.
I feel like EventNode should do a
$viewName = ViewName::normalize($templateName);

like View\Factory::normalizeName() does. But maybe I'm too me-focused right now. Laravel does the same for the first/main template render. I call it with view('home/index') in the controller, and Laravel (not TwigBridge) calls events for home.index. TwigBridge should do the same?

Something weird happens with the first/main template though. If I add a composer('*') listener, and print the $view->name() in the callback, I get the FULL path, AND the normalized path home.index. Only for the first/main template, not the includes and layouts called further in. Sounds a bit like #154 but not quite, and that's super old.
Some illustration (dump($view->name) in my composer('*')'s callback):

I call view('home/index') in my controller. Laravel triggers event for home.index
Somehow (through TwigBridge, I can see that in the trace, but not really where/how) an event for the full path is triggered
TwigBridge triggers the parent view's event (unnormalized theme/page-default)
And then that view's parent view (unnormalized theme/html)

Also, why is there a sprintf in EventNode::compile()?

#422 was first LOL
I am still curious about the full path event though....

@rudiedirkx I am having a similar issue with the paths that you mentioned. Was there any fix found for this? I made an issue here for it: https://github.com/rcrowe/TwigBridge/issues/438
I'm having a problem where my view composers aren't being triggered because it's using the path instead of the template name.

I only noticed it, I didn't have a problem with it, so it didn't really look into it.

Thanks for the update, @rudiedirkx. Thats all good, thanks anyway.


Create BuildBuddy gRPC client

ref #275

Using trick from here https://github.com/bazelbuild/rules_docker/issues/2149#issuecomment-1366151597 to make the rules_docker and rules_proto_grpc play nice.

Use https://grpc.io/docs/languages/python/basics/ to properly instantiate the client

Install and require the grpcio library as mentioned here https://grpc.io/docs/languages/python/quickstart/ to get the “google.rpc” package?

Calling the script can use Bazel run —config buildbuddy_rbe … inside of GitHub? To speed up execution?


Mock method/object inside resource

Feature Request
Desired behaviour
Be able to mock certain objects or its methods inside resource methods.  For example (mock SourceCodeRepoInterface sourceCodeRepo inside WorkflowResource's manualRegister method.  Or mock the method sourceCodeRepo.getWorkflow instead.

Unclear


BUG: Change Signup Page

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?
To maintain uniformity in whole website, we need to change the design of register page and make it same as sign in page.

Record

[X] I agree to follow this project's Code of Conduct
[X] I'm a GSSOC'24 contributor
[X] I want to work on this issue

Assigned!
Make sure to attach before & after screenshots in your PR. Also, star the repo  before working on it. While creating your PR, ensure you pull all existing changes beforehand to avoid conflicts.
Happy open source contributing!

@RitiChandak I by mistake removed my assignment, Please assign again

@RitiChandak I by mistake removed my assignment, Please assign again

Assigned!
Make sure to attach before & after screenshots in your PR and include the issue number.
Also, please star the repo before working on it.
i am the project mentor reach out for any help needed!!
Happy open-source contributing!


Camera and Microphone buttons have the wrong on/off state

Repro steps

Check the buttons

Expected Result
Currently both are enabled, but the buttons show they are disabled
See for same in discord when the mic is enabled

vs disabled

Actual Result

System Info
v2.0.0

The state is ok but we will implement tooltips and snackbar messages to indicate the state (or state switch).
https://github.com/threefoldtech/freeflowconnect_frontend/issues/82


Save Output

Is your feature request related to a problem? Please describe.
The output to the terminal is nice, but I’d like to be able to save it to a file (like html or txt)
Describe the solution you'd like
Using the rich.console.save_html or rich.console.save_text to save the output to a specified file
Describe alternatives you've considered
None
Additional context

Closed with fc6b584


always get -1 when get offset from brokers

Sarama Version: 2fd980e23bdcbb8edeb78fc704de0c39a6567ffc
Kafka Version: <IP_ADDRESS>
Go Version: 1.8.1
cli, err := sarama.NewClient(strings.Split(brokers, ","), nil)
if err != nil {
	return
}

b, err := cli.Coordinator(group)
if err != nil {
	return
}

ps, err := cli.Partitions(topic)
if err != nil {
	return
}

req := &sarama.OffsetFetchRequest{
	Version:       1,
	ConsumerGroup: group,
}

for _, pid := range ps {
	req.AddPartition(topic, pid)
}

resp, err := b.FetchOffset(req)
if err != nil {
	return
}
for id, msg := range resp.Blocks[topic] {
	fmt.Println(id, *msg)
}

I get
135 {-1  kafka server: Not an error, why are you printing me?}
150 {-1  kafka server: Not an error, why are you printing me?}
17 {-1  kafka server: Not an error, why are you printing me?}
62 {-1  kafka server: Not an error, why are you printing me?}
82 {-1  kafka server: Not an error, why are you printing me?}
113 {-1  kafka server: Not an error, why are you printing me?}
......

Presumably that means the brokers aren't storing offsets for those topic/partition/group combinations? Has something actually set the offsets in kafka? Or perhaps the offsets are still being stored in ZooKeeper and you need to use Version 0 of the OffsetFetchRequest?

Please feel free to reopen if you have more information showing this is a Sarama bug.


How to create imgui::Image ?

First of all, I want to thank you for your great article! It helped me to use imgui with ggez 0.4.4 and it helped me again when I migrated to ggez 0.5.1. It saved me a lot of time for my game ! So maybe you could help me again.
Do you know how to create an "imgui::Image" ? I need this because I want to popup a window showing an Image from a file.
It would be great if we were able to display a "ggez::graphics::Image" inside an imgui Window!
Optional question : Would it be doable to display an animated GIF ?
Thanks.

Hey! really glad it helped! I just added an example on master with loading an image and rendering it in the UI, let me know if that helps 😄

Thank you very much! This is what I need! I'll give you a link to my "little" game when it will be ready to be published. :)
When I try with an animated GIF, only a frozen image is displayed. No sure if it is easily doable. I can live whitout it. ;)

Yeah please send me a link, would love to have a look! Hmm I guess the gif behaviour makes sense, thinking about it now gifs probably need to be handled more like videos than images. Looks like imgui doesn't have a built-in way to play video https://github.com/ocornut/imgui/issues/2422. But I wonder if you can hack it yourself by displaying sequential images in your render loop. For example, break the gif into x frames, load those x textures like the code above, and then in the render loop choose which texture to display based on the time. Let's say you had 10 frames per second, between 0-100ms you show frame, then 100-200ms frame2, etc.

Thanks, I integrated your code in my game and I have to say it helped me a lot. About 10 steps is needed in order to display a simple image in an ImGui Window ! I wouldn't find myself all these steps.
Yes, an animated GIF is like a video, but I won't put effort on this, because you know, ImGui is a Gui for debugging games.
If my game has some success and I can find someone to create assets, I will take a look about playing video in a game.
Once again, thank you very much !

really glad it helped! 🎉 best of luck with your game!

I've published my game : Le Train Dispatcher !
http://athorus.itch.io/ltd
Tell me what you think about this game, even though your comments are not so nice. 😉 (I need useful comments to evolve the game concepts)
Go to the menu "Help" to see where I needed to load an image in an ImGui window.

ahhh that is super cool! very satisfying to see the image loading 🎉
btw you should add some instructions on the itch page for mac that you need to download and then chmod 755 before you're able to run the executable. Will have a deeper look at the game soon 😄

I'm glad to hear my OsX executable works! :D I compiled it thanks to osxcross as I do not own a Mac.
How should I package the file for OsX ? In a tar.gz as Linux system, in order to keep 755 ?
Fill free to open a post on itch, it will make the page alive.


use context to shutdown http server

Pull Request Test Coverage Report for Build 1757

1 of 3 (33.33%)  changed or added relevant lines in 2 files are covered.
8 unchanged lines in 3 files lost coverage.
Overall coverage decreased (-0.009%) to 63.264%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

chassis.go
0
1
0.0%

server/restful/restful_server.go
1
2
50.0%

Files with Coverage Reduction
New Missed Lines
%

core/rpc_invoker.go
2
93.1%

core/handler/router_handler.go
3
0.0%

core/fault/fault.go
3
57.14%

Totals

Change from base Build 1756:
-0.009%

Covered Lines:
5528

Relevant Lines:
8738

💛  - Coveralls


I want to change ui  color

I would want to work on this issue please assign me this @param-code

I would want to work on this issue please assign me this @param-code

It's already given to someone else.


Uses python -m pip style for installation

Updates the readme to use python -m pip install style for installing python packages. This is what we suggest at upstream to reduce the confusion of which Python interpreter is being used.
See more: https://snarky.ca/why-you-should-use-python-m-pip/

I wanted to say "not needed because virtualenv" but that link was pretty convincing.

@kushaldas sorry about that, but due to a bug in HF Transformers, CI is currently failling (not your fault). However, you would have to pull and merge from main and push again before we can accept.

Can you ensure you're up to date with respect to main? There were some issues with Hugging Face Transformers library that required extra dependencies that is now fixed in main. Then we'll be able to merge. Thank you!

I updated the PR last week. Commenting as just in case :)

Some (unrelated?) lint error on CI:

audiocraft/modules/conditioners.py:607: error: Argument 1 to "apply_model" has incompatible type "Union[Tensor, Module]"; expected "Union[BagOfModels, Union[Demucs, HDemucs, HTDemucs]]"  [arg-type]

Can anyone please tell me what needs to be done here?

Woo hoo, can anyone please rerun the CI job and then accept the PR?

Will these CI jobs ever finish?

@adefossez @JadeCopet can anyone of you please run/kick the CI as required?

Since @adefossez has left Meta for a new industry lab, I imagine we won't see much reviewing here. Would be nice to know the future of AudioCraft as a package!

Who can merge?!!! :)

Thank you again!

Yay!!! Thank you @JadeCopet :)


Need your suggestions, Thanks

Hi, I have read the paper (A Transformer-based Framework for Multivariate Time Series Representation Learning), it is a very meaningful work. I want to use this project to finish a prediction work. My taks is a regression problem, and my dataset can be described as follows:
X =
[
[[time series sequence 1]],
[[time series sequence 2]],
***
[[time series sequence s]],
]
and y = [
[[label 1]],
[[label 2]],
***
[[label s]],
]
where sequences are not the same length.
So I want to just use your model definition (in ts_transformer.py](https://github.com/gzerveas/mvts_transformer/blob/master/src/models/ts_transformer.py)) and paddding all sequences to the same length in my dataset before input the TST model.
Is there any thing I need to pay attention to in this job ? Or do you have other suggestions ?
Thansk.

Hi, if I understand correctly, your concern is whether you need to do the padding yourself. The answer is no, the code will handle that for you, you only need to provide a maximum sequence length parameter ( max_seq_len ) that makes sense for your data, for example, it could be set to the length of your longest sequence. What you should probably do first is look at the statistics of your sequence lengths. What is the variance of lengths? The computational cost of the transformer is O(N^2) with respect to the input sequence length, so if you have only a couple of very long sequences, but most of them are quite short, then it may be worth setting max_seq_len to a smaller value that will accommodate most (but not all) examples - the longer ones will be trimmed down to max_seq_len.

Great, thank you for your reply. First, I will do the statistics of sequence length, and then decide the max_seq_len. In my memory, the min/max sequence length are 50+, and 256, respectively, so I might be concer that such a large gap will affect the prediction performance if I use the max sequence length or cut of it. The sequence to be cut may contains very volunable information.

In your case it's absolutely fine to use max_seq_len = 256. Problems start when the length is in the order of N > 4000. In those cases, if longer sequences cannot be trimmed, then one should consider downsampling, or extracting features with a 1D convolutional layer first.

Think you for your reply, I have look the statistics of sequence lengths in my dataset, and it described as follows:

In my opinion, I will use the max sequence length to pad shorter sequence and then input Transformer model because the max sequence length is not very long.
In addition, I'm  thinking about whether padding operation has an effect on prediction error because the padding value may bring harmful information for prediction tasks.
I'm very interested in the input and outpu of your pre-trained model with unlabeled data. I understand that input and output of Autocoder are the same features and try to min error between input and output in order to learnt latent features in middle layers in net. However, the pre-trained model proposed in you paper how to learn latent features with unlabeled data.

The input values for time steps corresponding to padding (and all respective representations in deeper layers) are completely ignored throughout the computation of all intermediate representations. Thus, during unsupervised pre-training, they play no role whatsoever. For (global, example-level) regression and classification, they are additionally set to 0 at the final layer, before concatenating representations and making a prediction. As I wrote in my previous comment, in case of a large variance in lengths, this might affect performance (you would have to check), in which case the remedy would be to aggregate all final representations per time step using mean-pooling (i.e. Z = mean(z1, ..., zN)), instead of concatenating all representations, as currently done (i.e. Z = [z1, ..., zN]).
Regarding the unsupervised objective, you can consider this paper's masking scheme as a very specific way of doing autoencoding, with a special noise distribution designed to extract meaningful representations from most common types of multivariate time series data.

You have to first define a dataset class in datasets/data.py and load your data in a pandas dataframe. The datasets/dataset.py will take care of the rest. I have added more explanations in the README file.


Support EBS CSI driver and fix ingress

I used this chart install flink but got error

Error: render error in "flink/templates/ingress.yaml": template: flink/templates/ingress.yaml:26:15: executing "flink/templates/ingress.yaml" at <.host>: can't evaluate field host in type interface {}

and i use eks with EBS CSI driver, chart do not support. And several warnings

2020/05/28 15:51:38 Warning: Merging destination map for chart 'flink'. Cannot overwrite table item 'tolerations', with non table value: map[]

2020/05/28 15:51:38 Warning: Merging destination map for chart 'flink'. Cannot overwrite table item 'tolerations', with non table value: map[]

I submit PR #10

Hello @cxnam!
Thank you for your contribution.
Could you please update your PR according to my comments?

#29


Make this addon inert on Ember versions with native support for the feature

Once this feature is enabled on the Ember side, this addon should be updated to be completely inert (i.e. have no effect in the consuming app's build output). There is a recent example of that here. I'm not sure if there is a better way to do it that is more Embroider-friendly, but at the end of the day, I am not sure it really matters. If the reason this is using v1 build hooks is to disable itself on modern version of Ember anyway, it seems ok? cc @ef4

Yes, the example you linked should be embroider-compatible, and since this package is a v1 addon it's a good pattern for it to use.

excellent, I can get the auto-removal stuff setup today


Update README.md with status and create followup issue

Work for this issue will begin begin on branch work-liquid-labs/s3-empty-bucket/20.


Add Test Cases for 'API Fuzz Testing'

Add Test Cases for 'API Fuzz Testing'

Hey @Chhavig28 , I am new to testing(do not have much knowledge) . I want to contribute to solve this issue. Can I get some guidance? Like Some examples of what should I add or some resources. As far as I know, API testing involves checking the result of the API at unexpected values(like edge cases)

Closing issue.


Fix ordering of metric rollups in consumption history

this is fixing sporadic failures caused by https://github.com/ManageIQ/manageiq/issues/17592
the issue is that we got improper count of hours of Consumption#consumed_hours_in_interval
it was caused that set of metric rollups (after pluck) is not ordered according to timestamp.
Then we could not get proper first metric rollup according to timestamp  and then we were badly calculating consumed_hours_in_interval here.
Links
https://travis-ci.org/ManageIQ/manageiq/jobs/393976777
https://bugzilla.redhat.com/show_bug.cgi?id=1601590
@miq-bot assign @gtanzillo
@miq-bot add_label gaprindashvili/yes, sporadic failure/test

@miq-bot add_label blocker

As per BZ Target Release, removing gaprindashvili/yes


[MenuItem] SubItem Collapsing when setState is called in container

Problem description
I'm constrained to build a nested popover menu in my render method.  And every time I setState ( and render a new menu) The popover behaves differently based on which nested MenuItem I have open.
Each  has a nested  array of Checkbox inputs.  When a checkbox is clicked, it sets state in the container (required In my app in order to control checkboxes).  The first group of subItems (Menu Item 1) will remain open when checking the checkbox, any additional groups (Menu Item 2-4 +)  automatically close on checking the checkbox.
Link to minimally-working code that reproduces the issue
I've linked a code example.  Please click the button labeled "Click Me", then open up each menuItem's subMenus, and try and click (directly) on the input/checkbox.   The first submenu will remain open and the rest of them will close onTouchTap event.  I've tried including several other event handlers to prevent default on and none work.
WebpackBin:
http://www.webpackbin.com/4kX-28iDz
Versions

Material-UI:   "0.16.7",
React: "15.4.2"
Browser: 55.0.2883.95 (64-bit)

I spent a decent amount of time trying to figure out what is causing this, but could not find an answer. I believe it's specifically related to the interaction between using a Checkbox component for the primaryText attribute.
Ultimately, the currently implemented API is to close on click (see this line in MenuItem). I checked this is consistent across menu items and it is (see this WebpackBin)

Do you know how I could keep the menu open through click events?  I need
them to stay open so I can click multiple checkboxes.
On Sun, Feb 5, 2017 at 11:14 PM, Matt Stewart<EMAIL_ADDRESS>wrote:

I spent a decent amount of time trying to figure out what is causing this,
but could not find an answer. I believe it's specifically related to the
interaction between using a Checkbox component for the primaryText
attribute.
Ultimately, the currently implemented API is to close on click (see this
line
https://github.com/callemall/material-ui/blob/master/src/MenuItem/MenuItem.js#L190
in MenuItem). I checked this is consistent across menu items and it is (see
this http://www.webpackbin.com/EkCcfW-df WebpackBin)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/callemall/material-ui/issues/6051#issuecomment-277594667,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AOf6j2KCYo3taNMJVInKobttGG5a6cfvks5rZro7gaJpZM4L0hyv
.

@KevinMongiello event.stopPropagation?

@mbrookes Doesn't work.  I included that in the onTouchTap handler for the checkboxes in the WebpackBin with no avail.

@KevinMongiello This may be related to #5104

@mattste Yeah It very well might be, I remember reading that thread but it doesn't look like there was any real resolution since under certain circumstances the checkboxes appear to respond to preventDefault / stopPropagation

We have been porting the component on the v1-beta branch. We reimplemented it from the ground-up. While we haven't tested it, I think that the issue is most likely fixed on that branch. Hence, I'm closing it.
Still, we will accept PR fixes until v1-beta takes over the master branch.


Add logging during --finish and --analyze step

These steps do take some time. Especially --finish one.
It will be useful to see which file takes a lot of time, which file is getting parsed right now and so on.

Is this still needed with #37?

Actually, right now I have 3 warnings like this:
WARN: JSON parse error Something went wrong while writing to the tape.
So, it may be not a bad idea to add some logging.


feat(components): add autoclear prop to search-autocomplete

This feature change the behavior of the SearchAutocomplete component.
It adds a autoclear prop that is false by default and is used to explicitly enabled the autoclear behavior (search text is cleared when an item is clicked/selected).
This PR originates from this one by @bbrangeo .
Thanks to him for the proposal. :)

Would it be possible to have the opposite behavior by default ?

OK, I'll change the props default value.

:tada: This PR is included in version 1.4.0-rc.31 :tada:
The release is available on:

npm package (@next dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 2.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Additional destroy operators

This is fairly ugly, but I'm waiting for a reply by Roodbergen before I can continue with nearest_pairs.


Improve warning when an invalid foldable type is given to CodeLengthCalculator

The warning message previously offered send and csend as possible options, but they are not in fact valid, and will raise a warning if used. method_call should be offered instead.

Thanks!


Feat: Create Speech  [ tts-1, tts-1-hd ]

What
Implemented Create Speech API functionality.
Why
This new feature was requested in #113 and is a part of latest OpenAI update.
Affected Areas
Affects root OpenAI.swift, Models.swift files.
Added AudioSpeechQuery and AudioSpeechResult
More Info
If you got any suggestions to improve this code, feel free to comment ❤

Is there a way to support streaming of this audio? As documented here; https://platform.openai.com/docs/guides/text-to-speech/streaming-real-time-audio

Is there a way to support streaming of this audio? As documented here: https://platform.openai.com/docs/guides/text-to-speech/streaming-real-time-audio

Hi, yes, it seems like it is possible, because the data is actually being streamed every time we make a request and if you do like this:
curl https://api.openai.com/v1/audio/speech \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "tts-1",
    "input": "The quick brown fox jumped over the lazy dog.",
    "voice": "alloy"
  }' \
  --output speech.mp3

You can actually see progress how is file being fetched and written. I think for that we need to have some audio player that supports partial raw data playback, to stream the result right into it.


Support GTM server-side tagging custom domain and environments

See here.
:

:

<iframe src="<GTM_DOMAIN>/ns.html?id=TAG_ID"
height="0" width="0" style="display:none;visibility:hidden">

Also, support additional params, than can be atob(container_configuration) from Container Configuration below:
https://tagmanager.google.com/#/admin/accounts/<ACCOUNT_ID>/containers/<CONTAINER_ID>

These are: id, env and auth.
An env param stands for the GTM environment and is especially useful for the staging (and others) environment. It is only applicable for the client-side tagging at the moment.
You can find and manage your custom environments here:
https://tagmanager.google.com/#/admin/accounts/<ACCOUNT_ID>/containers/<CONTAINER_ID>/environments
Code snippet:

@wscourge Thanks for reporting! I just created a pull request (https://github.com/XD2Sketch/next-google-tag-manager/pull/8). Can you maybe take a look and check if this pull requests adds the features you need to correctly set up your google tag manager configuration?
Also, I couldn't find good documentation for what gtm_auth does, can you elaborate?

Nope, didn't find any. I just posted the decoded Container Configuration value. Anyway, I don't think it is relevant, as long as you somehow document where to get it.


Has GoodItemScan fixed this issue？I'm not entirely sure because this issue occasionally happen and occasionally doesn't.
This mod👇 fixes this issue, perhaps you can refer to it?
https://thunderstore.io/c/lethal-company/p/TheSecretDev/Unlimited_Scan/

My config settings

It should not be possible to scan any items behind you

Reason for this is the fact that the calculated distance would be in the negatives otherwise

Sorry, I edited the vague issue description.
The correct description can be found in another issue I previously initiated👇
https://github.com/ButteryStancakes/ButteryFixes/issues/7

When this issue occurred, I did not use Unlimited_Scan and  GeneralImprovements's FixPersonalScanner.
I will retest this issue tomorrow.


Add publish to medium button for a file

Add medium authentication to publish the file directly to medium's blog

Hi @hypermystic , I would like to help with this.

yeah sure...assigned


Writing out to genbank file format.

Is your feature request related to a problem? Please describe.
Currently poly can read genbank (.gb, .gbk files) but it can't write out .gb, .gbk files.
Describe the solution you'd like.
What I'd like is a gbk string builder similar in interface to the GffBuilder function in io.go.

Things that need to happen for genbank write. @Koeng101

Introduce new json struct tags in AnnotatedSequence struct.
create other unstructured map object for meta to hold meta data that isn't standard.
Actually write buildGbk and writeGbk functions
Edit docs to reflect new changes.

Fixed in pr #33.


Update neofetch for Orchid Linux

Description
Only fill in the fields below if relevant.
Features
Added Orchid Linux
Issues
Not present
TODO

Thank you for your contribution!
This PR is merged into hyfetch since this repo (dylanaraps/neofetch) seems no longer maintained.
HyFetch is a fork of neofetch with LGBTQ pride flags, but the repo also serves as an updated version of neofetch, addressing many pull requests that are not merged in the original repo.


Support for Sveltekit

Is your feature request related to a problem? Please describe.
Port of the React Library in Svelte
Describe the solution you'd like
Writing a wrapper or a library for sveltekit
Describe alternatives you've considered
Exposing the apis with vanilla Js , So it can be framework agnostic
Additional context
Any Contributions are appreciated

Any update?


An error occurred while running worker[SGX_ERROR_UNRECOGNIZED_PLATFORM]

Hello, I encountered an error when trying to run the worker.
The content of worker1.log is as follows:
[2023-12-17T13:53:08Z ERROR integritee_service::ocall_bridge::ffi::init_quote] [-]  Failed to init quote: InitQuote(SGX_ERROR_UNRECOGNIZED_PLATFORM)
thread 'main' panicked at 'called Result::unwrap() on an Err value: Sgx(SGX_ERROR_UNRECOGNIZED_PLATFORM)', service/src/utils.rs:38:59
I checked the log of aesmd and part of the source code of the worker project, and found that the extract_shard method of the integritee-service tries to use Intel's EPID-based init_quote method. However, according to my review of Intel official support documents, "All consumer CPUs and all Xeon E-xxxx CPUs that support Intel® SGX support EPID. However, the newest 3rd generation Intel® Xeon® Scalable Processors (codename Ice Lake Server) and all future Scalables only support DCAP (no EPID support)." Unfortunately, the CPU platform I am using is Intel(R) Xeon(R) Platinum 8369B. How can I solve this problem? Thank you.
By the way, I am using the branch sdk-v0.12.0-polkadot-v0.9.42.

If you're just developing, ou can build the worker in SW mode and use the --skip-ra flag
If you need SGX guarantees already, you need a supported machine.
We know it can be tricky to predict if a certain HW will work. I'ts not only the CPU which matters, it's the chipset/mainboard as well. Unfortunately, you'll have to try to know


No warning when returning undefined variable

I don't appear to be getting a warning in this circumstance:
(function () {
  return $trolol;
})();

This plugin is for catching unused variables, not for catching undefined variables.
Catching undeclared variables is already done by Phan itself.
src/undef.php:3 PhanUndeclaredVariable Variable $trolol is undeclared


New features and bug fixes

(Note:  I've decided not to commit *.lua and *.js files into pull requests anymore.  This is to make it easier for you to read the diffs. Remember to recompile Lumen after merging any of the commits.)

Added bin/make-lumen

It's a python script which fully recompiles Lumen twice by default.  Also prints out timing statistics.

This will avoid scenarios where make clean; make fails to incorporate changes to the compiler, since it rebuilds Lumen only once.
(A change to the compiler means that Lumen must be rebuilt twice in order for the changes to be correctly integrated.)

runtime.l:

New global definition for Lua hosts:  (define-global global _G)

Analogous to Node's global

New global function: list?

(list? ()) => true
(list? (obj)) => true
(list? 42) => false

string:

now detects circular descendants.
(let l '(1 2 3)  (set (at l 1) l)  l) => (1 circular 3)
(string (fn (x) x)) now gives "fn" rather than "function"

This makes LUMEN_HOST=node bin/lumen -e 'global' far more readable, for example.

Now handles unexpected types, such as cdata and userdata

Previously, LUMEN_HOST=luajit bin/lumen -e "(require 'ffi)" resulted in an error

compiler.l:

Added support for fn objects appearing in expressions passed to eval

(eval (fn (x) x)) => fn
(eval `(quote ,(fn (x) x))) => fn
(let x 42 ((eval `(quote ,(fn () x))))) => 42

Verified that the slowdown introduced by this change is negligible.

macros.l:

New macro:  (define-macro get-global (name) (get global (compile ,name)))

Necessary for e.g. (do (set (get-global '%abc) 42) %abc) => 42
(Doing (set (get global '%abc) 42) won't work because %abc's actual variable name is _37abc)

test.l:

Added tests for the new functionality listed above.


[Vulkan][Codegen] Fixed SPIR-V scoping bug with threadIdx

Change is split into two commits, one which implements the fix, and one which refactors the result into a more general use.

[Vulkan][Codegen] Fixed SPIR-V scoping bug with threadIdx
Unlike in Cuda, where the threadIdx.x/y/z are separate built-in variables, in vulkan the built-in thread index consists of an array
that must be dereferenced.  If te.thread_axis("threadIdx.x") is first declared inside a scope, then that same python object is passed as the IterVar of a separate scope, this breaks the spirv scoping rules, and may result in an undefined variable.  This only applies to threadIdx/blockIdx variables, as all other variable declarations obey tir's scoping rules.
To resolve, the threadIdx and blockIdx variables are declared at the top of the function declaration.  This makes the generated spirv code follow the same semantics as tir/cuda.

[Vulkan][Codegen] Refactor of GetLocalID and GetWorkgroupID
Shared behavior separated out into GetBuiltInValue.

Potential reviewer: @masahi

thanks @Lunderberg


Tokenize and stem bug

Currently, if the text that is passed into the tokenizeAndStem method contains non-lowercase stopwords, such as 'My', when we tokenize the string and check if the individual tokens exist in the stopwords array this check will evaluate to false, and the stopwords will be included in the output.
This change takes the input text and lowercases it prior to tokenizing, so all of the tokens can be tested directly against the stopwords and not included if we don't want to keepStops. Applicable specs included.

good catch, thanks!

Thank you! Do you expect a new release to go out soon that will include this version?

gotta make my way through the other PRs and then ill try to get one out asap

Awesome, thanks again!


Button label locked when action isn't "toggle function"

Describe the bug
In the Button Properties modal, changing the "on button press" radio list to any value other than "toggle function on/off" will lock the "General" panel at the top of the modal. The intention is clearly to lock the "Function" field, but it also stops the user editing the "Label" field. So the button's name is frozen until you change its type again.
I've tested this only on version 4.12.7.
To Reproduce
Steps to reproduce the behavior:

Go to Virtual Console, create a button, open the "Button Properties"
Focus the "Button Label" field, to see that it's editable
Change the "On button press..." radio to anything other than the default of "Toggle function on/off"
the "General" panel at the top becomes locked - if you attempt to focus "Button Label", it won't work

Expected behavior
Lock only the "Function" field and its associated buttons.
Desktop (please complete the following information):

OS: Linux
Q Light Controller Plus version 4.12.7
(no hardware involved)

Hi there,
Thank you for making a bug report and providing detail so I could investigate the issue.
I've been able to reproduce your issue in Windows 10, but only for "Toggle Scene" and "Stop All Functions"
Can you confirm if you are seeing this for the "Flash Function" radio button as well?

No laptop right now, but I only tested stop and blackout, so it's likely that the behaviour is the same as you're seeing. Can confirm later.

I confirm label is locked only on "Toggle blackout" and "Stop all functions".
It's basically because an icon is displayed so no label is needed.
I think the proper behavior (at least in v4) would be to blank the label when selecting those 2 modes. This would make even clearer that label is irrelevant
Is there any need for a label on blackout/stop all?

Ah, I see! I'm happy for you to close this issue, or keep it as a reminder to make the interface clearer, whatever works.
Personally, I'm only using this Virtual Console to assign OSC paths to functions, and I'm labelling them all after the OSC path. I suppose my preference would be that all widgets have a default name, like "[Toggle|Flash|] FunctionName", so "Toggle Blackout" and "Stop All" could have default names (instead of images) and not be special cases. But it's fine how it is, and I'm not really a typical user.


[SPARK-34111][BUILD][3.1] Deconflict the jars jakarta.servlet-api-4.0.3.jar and javax.servlet-api-3.1.0.jar

What changes were proposed in this pull request?

Use the right name for the artifact javax.servlet-api in hadoop 3.2 profile
Why are the changes needed?

fix deps issue
Does this PR introduce any user-facing change?

no
How was this patch tested?

should pass existing jenkins
I have manually tested a spark application with the javax.servlet-api-3.1.0.jar deleted and it worked fine

cc @HyukjinKwon @dongjoon-hyun

cc @HyukjinKwon @dongjoon-hyun

Test build #134156 has started for PR 31214 at commit 3fe82f6.

Test build #134156 has started for PR 31214 at commit 3fe82f6.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38739/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38739/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38739/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38739/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38739/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38739/

Test build #134156 has finished for PR 31214 at commit 3fe82f6.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test build #134156 has finished for PR 31214 at commit 3fe82f6.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134156/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134156/

I think this causes the test being hanging:

I will lower the priority of this ticket from Blocker to Critical since we haven't found an explicitly known issue in Apache Spark yet by this, and this is at least fixed in master by shading the Hadoop client by SPARK-33212, which I think is a good step forward. cc @gatorsmile FYI.
I will cut the RC Monday PST. It will be good to have this fix, though.

I think this causes the test being hanging:

I will lower the priority of this ticket from Blocker to Critical since we haven't found an explicitly known issue in Apache Spark yet by this, and this is at least fixed in master by shading the Hadoop client by SPARK-33212, which I think is a good step forward. cc @gatorsmile FYI.
I will cut the RC Monday PST. It will be good to have this fix, though.

I have tested andpassed  the hanging ut locally. seems a github action issue

I have tested andpassed  the hanging ut locally. seems a github action issue

GA Looks OK now? @HyukjinKwon @dongjoon-hyun

GA Looks OK now? @HyukjinKwon @dongjoon-hyun

let me retrigger few more times.

let me retrigger few more times.

+1 for @HyukjinKwon 's decision.

+1 for @HyukjinKwon 's decision.

@yaooqinn would you mind pushing 3 empty commits individually so we can run the builds more in parallel?

@yaooqinn would you mind pushing 3 empty commits individually so we can run the builds more in parallel?

Okay, I found that hanging happens in another PR too. Let me merge this in.

Okay, I found that hanging happens in another PR too. Let me merge this in.

thanks~ @HyukjinKwon. And sorry for late reply

thanks~ @HyukjinKwon. And sorry for late reply

Merged to branch-3.1.
It's okay. thanks for fixing this.

Merged to branch-3.1.
It's okay. thanks for fixing this.


Added option for anonymous SMTP

I have encountered a use case in which I need to use a SMTP server which does not support AUTH. I have added new option to disable SMTP AUTH.

Thank you @lhotamir for pull request. Please use following command to install latest changes until new version is released.
pip install git+https://github.com/adiralashiva8/pytest-email

Will update documentation ASAP


Failed to merge error

Describe the bug
Failed to merge error. Tried on 2 users. Tried on Classic and LEX.
To Reproduce
Steps to reproduce the behavior:

Go to user
Click on "Open permission report" button
Wait
See error "Failed to merge, see console for details."

Expected behavior
First time I use the extension. Extension did sound promising but this was disappointing.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

Browser type and version [e.g. Chrome 88.0.4324.150]
Extension version [e.g. 1.0.0]

Does anything come up in the console regarding this error?
Is there anything special about your users or their profile, permission sets or permission set groups?

I haven't checked the console (I don't know how it works or what to look
for).
Other than being complex permissions (which is the reason to use the
extension), nothing strikes to me as "special"
On Wed, 31 Aug 2022 at 17:44, Aurel Hudec @.***> wrote:

Does anything come up in the console regarding this error?
Is there anything special about your users or their profile, permission
sets or permission set groups?
—
Reply to this email directly, view it on GitHub
https://github.com/hudec117/sf-user-perm-report/issues/45#issuecomment-1233110802,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AGHV75DMN3EP7ILARBSWF73V354WTANCNFSM57WKNGAQ
.
You are receiving this because you authored the thread.Message ID:
@.***>

@hudec117 I am running into this same issue. I don't know how to access the console to see error details. It seems to be happening on any user in some of my orgs that have complex permissions. The report works in the same browser in a different org with simple permissions.

@aknight1157 could you try removing all the permission sets from the user and running the report? If it works, that would indicate the issue is with one of the permission sets. If not, then it means it may be the profile.

I have been able to replicate this issue by putting a managed Permission Set into a permission set group and then trying to run the report. Is this the case in your scenario?

This should be fixed as of 1.2.4, in the process of being published to the Chrome Web Store. If you find it still doesn't work with the update, feel free to re-open!


update ruby gem version to 1.0.2

for releasing a new version by ruby gem because of this bug

Thanks @deme0607! Will merge it shortly.

Thanks!

Thanks!


Java 8 Dates support

Since I'm getting the following error
Caused by: java.lang.NoSuchMethodException: java.time.ZonedDateTime.()
at java.lang.Class.getConstructor0(Class.java:3082)
at java.lang.Class.newInstance(Class.java:412)
... 10 more
I assume that Java 8 Dates types are not supported (the same applies for LocalDateTime and OffsetDateTime).
I also saw that for deprecated Date type the library works fine.
Didn't have time to look at the code - but if this is the case than I advice you to add it to future release since this is the standard for Java 8.

Java 8 Date types are not supported yet.
I'll add support for them in a future release.
Thank you for pointing out this issue.
Regards
Mahmoud

Hi @danielraczynski ,
I added support for Java 8 Date and Time types in the 2.0.0-SNAPSHOT version (which I deployed to maven central repo). Use the following repo to get it:
<repository>
    <id>ossrh</id>
    <url>https://oss.sonatype.org/content/repositories/snapshots</url>
</repository>

It would be great if you could give it a try and tell me if it is ok for you.
Thank you.
Kind regards
Mahmoud

Hi, i dont unferstand because in #28,  you said that be carefull to use
Java8 classes as Supplier class.
JMPrieto.
El lun., 21 de diciembre de 2015 22:35, Mahmoud Ben Hassine <
<EMAIL_ADDRESS>escribió:

Hi @danielraczynski https://github.com/danielraczynski ,
I added support for Java 8 Date and Time types in the 2.0.0-SNAPSHOT
version (which I deployed to maven central repo). Use the following repo to
get it:

ossrh
https://oss.sonatype.org/content/repositories/snapshots

It would be great if you could give it a try and tell me if it is ok for
you.
Thank you.
Kind regards
Mahmoud
—
Reply to this email directly or view it on GitHub
https://github.com/benas/random-beans/issues/41#issuecomment-166431058.

Hello @prietopa
Only module random-beans-java8 requires Java 8, all other modules are Java 6 compatible.
In #28, the goal is to add a method in the PopulatorBuilder that accepts a Supplier an adapt it to a Randomizer (which is a great idea), but this requires the random-beans module to use Java 8 which makes the library incompatible with Java 6..
The goal is to use Java 8 APIs only in the random-beans-java8 and keep using Java 6 compatible APIs in other modules.
Java is backward compatible, unless you use APIs introduced in a newer version..
I've tried to set compiler settings (-source 1.8, -target 1.6) to compile in version 8 and produce version 6 byte code, but this is not possible.

understood!
JMPrieto
2015-12-22 11:01 GMT+01:00 Mahmoud Ben Hassine<EMAIL_ADDRESS>
Hello @prietopa https://github.com/prietopa
Only module random-beans-java8 requires Java 8, all other modules are
Java 6 compatible.
In #28 https://github.com/benas/random-beans/issues/28, the goal is to
add a method in the PopulatorBuilder that accepts a Supplier an adapt it
to a Randomizer (which is a great idea), but this requires the
random-beans module to use Java 8 which makes the library incompatible
with Java 6..
The goal is to use Java 8 APIs only in the random-beans-java8 and keep
using Java 6 compatible APIs in other modules.
Java is backward compatible, unless you use APIs introduced in a newer
version..
I've tried to set compiler settings (-source 1.8, -target 1.6) to compile
in version 8 and produce version 6 byte code, but this is not possible.
—
Reply to this email directly or view it on GitHub
https://github.com/benas/random-beans/issues/41#issuecomment-166571372.

@benas I cloned the repo and noticed java8 module commented out.
Uncommented, and it fails to build.
What's missing? Perhaps I could help...

I decided to postpone Java 8 support to v3. See here: https://github.com/benas/random-beans/issues/49#issuecomment-174336657
This is because I was not able to make the code base compile with -source=1.8 and -target=1.7

Hi @danielraczynski
As discussed, Java 8 support has been postponed to v3 (the upcoming release). I have just published the 3.0.0-SNAPSHOT to maven central. This version includes support for JSR 310 types.
Here is a sample bean and a passing test. It would be great if you could give it a try and tell me if it is ok for you. Thank you.
Kind regards
Mahmoud

hello @danielraczynski
If you have any problem with JSR 310 types support, please don't hesitate to file an issue.
Kind regards
Mahmoud


[21.11] ptcollab: Unbreak on Darwin, <IP_ADDRESS> -> <IP_ADDRESS>

Motivation for this change
Backports of #151926 & #152274.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

@ofborg build ptcollab


[FSDE-5953] Migrate Brand component to new Vue documentation

The CI pipeline did not run successfully in https://jenkinsprod.corp.intranet:8443/job/UX-CHI/job/Productive/job/Chi/job/PR-222/1/. :x:

The CI pipeline run successfully in https://jenkinsprod.corp.intranet:8443/job/UX-CHI/job/Productive/job/Chi/job/PR-222/2/. :white_check_mark:

You can check this PRs instance in https://54cffb020ef9d22894e34c6818508cfe650fe68a-chi-pr.enterprise.ctl.io (external) and in https://nginx-pr-222-ux-chi.kubeodc-test.corp.intranet/ (internal)

The CI pipeline run successfully in https://jenkinsprod.corp.intranet:8443/job/UX-CHI/job/Productive/job/Chi/job/PR-222/3/. :white_check_mark:

You can check this PRs instance in https://ebb301bba04174f6f6f8a95ba4e6060b21e76125-chi-pr.enterprise.ctl.io (external) and in https://nginx-pr-222-ux-chi.kubeodc-test.corp.intranet/ (internal)

The CI pipeline did not run successfully in https://jenkinsprod.corp.intranet:8443/job/UX-CHI/job/Productive/job/Chi/job/PR-222/4/. :x:

The CI pipeline run successfully in https://jenkinsprod.corp.intranet:8443/job/UX-CHI/job/Productive/job/Chi/job/PR-222/5/. :white_check_mark:

You can check this PRs instance in https://2a7315e99913711d72688c25465a87e932ff3642-chi-pr.enterprise.ctl.io (external) and in https://nginx-pr-222-ux-chi.kubeodc-test.corp.intranet/ (internal)

The CI pipeline run successfully in https://jenkinsprod.corp.intranet:8443/job/UX-CHI/job/Productive/job/Chi/job/PR-222/6/. :white_check_mark:

You can check this PRs instance in https://23255f4d3e874d2c64c8e2708ba4a8dd74624fc1-chi-pr.enterprise.ctl.io (external) and in https://nginx-pr-222-ux-chi.kubeodc-test.corp.intranet/ (internal)

The CI pipeline run successfully in https://jenkinsprod.corp.intranet:8443/job/UX-CHI/job/Productive/job/Chi/job/PR-222/7/. :white_check_mark:

You can check this PRs instance in https://cfb48087f163568bdc6779154432279076d4e575-chi-pr.enterprise.ctl.io (external) and in https://nginx-pr-222-ux-chi.kubeodc-test.corp.intranet/ (internal)

The CI pipeline run successfully in https://jenkinsprod.corp.intranet:8443/job/UX-CHI/job/Productive/job/Chi/job/PR-222/9/. :white_check_mark:

You can check this PRs instance in https://7ea97799a70075d71ed631fdf69b5b8b4ebcecd2-chi-pr.enterprise.ctl.io (external) and in https://nginx-pr-222-ux-chi.kubeodc-test.corp.intranet/ (internal)

The CI pipeline run successfully in https://jenkinsprod.corp.intranet:8443/job/UX-CHI/job/Productive/job/Chi/job/PR-222/10/. :white_check_mark:

You can check this PRs instance in https://32bd1bad6337d16a19c86a2f6c7c1bd3758b75fb-chi-pr.enterprise.ctl.io (external) and in https://nginx-pr-222-ux-chi.kubeodc-test.corp.intranet/ (internal)

The CI pipeline run successfully in https://jenkinsprod.corp.intranet:8443/job/UX-CHI/job/Productive/job/Chi/job/PR-222/11/. :white_check_mark:

You can check this PRs instance in https://8f54572a9c171dfaf69bfda9511153ce9a089fc9-chi-pr.enterprise.ctl.io (external) and in https://nginx-pr-222-ux-chi.kubeodc-test.corp.intranet/ (internal)

The CI pipeline run successfully in https://jenkinsprod.corp.intranet:8443/job/UX-CHI/job/Productive/job/Chi/job/PR-222/12/. :white_check_mark:

You can check this PRs instance in https://7b9744853c5390e07714daea2739ed397b0b4bcb-chi-pr.enterprise.ctl.io (external) and in https://nginx-pr-222-ux-chi.kubeodc-test.corp.intranet/ (internal)

The CI pipeline run successfully in https://jenkinsprod.corp.intranet:8443/job/UX-CHI/job/Productive/job/Chi/job/PR-222/13/. :white_check_mark:

You can check this PRs instance in https://67e0ad693c690e63790a451b56a6aff674179fa6-chi-pr.enterprise.ctl.io (external) and in https://nginx-pr-222-ux-chi.kubeodc-test.corp.intranet/ (internal)

The CI pipeline run successfully in https://jenkinsprod.corp.intranet:8443/job/UX-CHI/job/Productive/job/Chi/job/PR-222/14/. :white_check_mark:

You can check this PRs instance in https://12d5c69f5f1a714851b81f28af2ec284ecf398ef-chi-pr.enterprise.ctl.io (external) and in https://nginx-pr-222-ux-chi.kubeodc-test.corp.intranet/ (internal)

The CI pipeline run successfully in https://jenkinsprod.corp.intranet:8443/job/UX-CHI/job/Productive/job/Chi/job/PR-222/15/. :white_check_mark:

You can check this PRs instance in https://74940fe788510ad2ac5c22bad0c15a5f575235b3-chi-pr.enterprise.ctl.io (external) and in https://nginx-pr-222-ux-chi.kubeodc-test.corp.intranet/ (internal)

The CI pipeline run successfully in https://jenkinsprod.corp.intranet:8443/job/UX-CHI/job/Productive/job/Chi/job/PR-222/16/. :white_check_mark:

You can check this PRs instance in https://7edb60378fd31f80d077747da124454cad939cb7-chi-pr.enterprise.ctl.io (external) and in https://nginx-pr-222-ux-chi.kubeodc-test.corp.intranet/ (internal)


⚠ SceneTime has degraded performance

In e6c4d5d, SceneTime ($ST) experienced degraded performance:

HTTP code: 200
Response time: 8643 ms

Resolved: SceneTime performance has improved in 99dc3f7 after 12 minutes.


rplidar_ros: 1.5.2-0 in 'indigo/distribution.yaml' [bloom]

Increasing version of package(s) in repository rplidar_ros to 1.5.2-0:

upstream repository: https://github.com/robopeak/rplidar_ros.git
release repository: https://github.com/kintzhao/rplidar_ros-release.git
distro file: indigo/distribution.yaml
bloom version: 0.5.21
previous version for package: null

rplidar_ros
* Release 1.5.2.
* Update RPLIDAR SDK to 1.5.2
* Support RPLIDAR A2
* Contributors: kint

This version looks like it has valid control files! Lets see how it does on the buildfarm.


Support latest version of compreface

https://github.com/exadel-inc/CompreFace/blob/master/.env
Note that although the env has version 1.1.0, the actual images in docker hub are 1.0.1

I bumped the addon to CompreFace v1.1.0, let me know if you run into any issues.


Update v4 release to 4.67

Move 4.67.0 from pre-release to GA.

Blocking this until VS Code issues are resolved

VS Code PR to verify new templates has been merged here.


Rename to spetlr

change name to spetlr

Let's get the PR approved.
It blocks the integration tests of the rest of the PRs (due to renaming of the cosmos db - free tier issue)


Cross-Compile to musl example not working

I tried to build for musl but there seems to be two issue with the example given here: https://crane.dev/examples/cross-musl.html
The first issue is in
rust-overlay = {
      url = "github:oxalica/rust-overlay";
      inputs = {
        nixpkgs.follows = "nixpkgs";
        flake-utils.follows = "flake-utils"; # <-- Does not exist
      };
    };

The input flake-utils does not exist for rust-overlay and an error is thrown.
If I remove the line from the template, my flake looks like this:
{
  description = "Building static binaries with musl";

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";

    crane = {
      url = "github:ipetkov/crane";
      inputs.nixpkgs.follows = "nixpkgs";
    };

    flake-utils.url = "github:numtide/flake-utils";

    rust-overlay = {
      url = "github:oxalica/rust-overlay";
      inputs = {
        nixpkgs.follows = "nixpkgs";
      };
    };
  };

  outputs = { nixpkgs, crane, flake-utils, rust-overlay, ... }:
    flake-utils.lib.eachSystem [ "aarch64-linux" ] (system:
      let
        pkgs = import nixpkgs {
          inherit system;
          overlays = [ (import rust-overlay) ];
        };

        craneLib = (crane.mkLib pkgs).overrideToolchain (p: p.rust-bin.stable.latest.default.override {
          targets = [ "aarch64-unknown-linux-musl" ];
        });

        my-crate = craneLib.buildPackage {
          pname = "kellnr";
          src = craneLib.cleanCargoSource ./.;
          strictDeps = true;

          CARGO_BUILD_TARGET = "aarch64-unknown-linux-musl";
          CARGO_BUILD_RUSTFLAGS = "-C target-feature=+crt-static";
        };
      in
      {
        checks = {
          inherit my-crate;
        };

        packages.default = my-crate;
      });
}

A nix build throws the error:
error:
       … while calling the 'derivationStrict' builtin

         at /builtin/derivation.nix:9:12: (source not available)

       … while evaluating derivation 'kellnr-0.1.0'
         whose name attribute is located at /nix/store/z49izx4bdhyi1vd1qjlgdsg9vqirrlbd-source/pkgs/stdenv/generic/make-derivation.nix:334:7

       … while evaluating attribute 'cargoArtifacts' of derivation 'kellnr-0.1.0'

         at /nix/store/6p9yzbswxw5mqs339l3ib36pdxfazdk7-source/lib/mkCargoDerivation.nix:59:10:

           58| } // {
           59|   inherit cargoArtifacts;
             |          ^
           60|

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: Dependency is not of a valid type: element 1 of nativeBuildInputs for kellnr-deps

Any ideas how to get is working?

Hi @secana thanks for the report!

The input flake-utils does not exist for rust-overlay and an error is thrown.

Thanks for the note, should be fixed in #663 !

A nix build throws the error:

What version of Nix are you using? (On my NixOS machine I currently have nix (Nix) 2.18.4
Do you have a flake or branch I can check out that reproduces the issue?

I tried replacing the flake.nix of the cross-musl example with the one in your original issue but it built fine on my machine :thinking:

Thanks for the fast response. I updated the flake and the second issue disappeared.
Btw. crane is an awesome project. Makes rust on Nix great experience and it's one of the few projects with good documentation.


asynctest missing main.py

this would allow it to be run from the command-line like unittest:
python3 -m asynctest discover ...

Hi,
Currently, you can use
python3 -m unittest
Do you want to open a pull request?

ya found out it works, eh, going to close this.


Update to latest superagent with support for nested directory

So I have an alternative to trying to patch the form-data package used in superagent (#71). Let's just upgrade superagent to the latest version with support for passing the filename.
The advantage are:

No separate fork of form-data
form-data handles path normalization
superagent reads files in node (fixes #45)

Manual integration test in browser:
rm -rf node_modules
npm install
npm run compile && npm run build

Open this file in browser
<html>
  <head>
    <script src="dist/particle.min.js"></script>
    <script>
var particle = new Particle();
var TOK = 'your_token';
var promise = particle.flashDevice({ deviceId: 'your_device', files: {
  'application.ino': new Blob(['#include "subdir/status.h"\nextern int foobar; void setup() { Particle.variable("foobar", foobar); }'], { type: "text/plain" }),
  'subdir/status.h': new Blob(['int foobar;'], { type: "text/plain" }),
}, auth: TOK
});

promise.then(function(response) {
  console.log(JSON.stringify(response.body));
}).catch(function(err) {
  console.error(JSON.stringify(err));
});
    </script>
  </head>
</html>

Coverage increased (+0.004%) to 74.414% when pulling 2984291d23b3bdcd1f9377e7712850b9ce894e68 on feature/upgrade-superagent into 525bea3549344672a1889182397719ca17310549 on master.

Coverage decreased (-0.04%) to 74.368% when pulling e5c5d436b6627b399e81173187e1c9132e1cea03 on feature/upgrade-superagent into 525bea3549344672a1889182397719ca17310549 on master.

Coverage decreased (-0.04%) to 74.368% when pulling 85680361e3a1ca6d78eb2ca4e406f932aa894de6 on feature/upgrade-superagent into 525bea3549344672a1889182397719ca17310549 on master.

Build is green!!!!!!!!!!!!!!!!11!11111111!!!!
How about releasing as 7.0.0 since superagent 2 -> 3 is a potentially breaking change?

Agreed!


⚠ File access (beta) has degraded performance

In d3cc55e, File access (beta) (https://beta.fyle.uk/file/fish) experienced degraded performance:

HTTP code: 200
Response time: 7187 ms

Resolved: File access (beta) performance has improved in acf8bc6.


Scope Calls not working

Hi,
I want to develop an app for doing calls in Teams. When I want to get the login token, the popup says the Calls scopes don't exist. Is it supported yet ?
I tried typing "call", "calls" or "Calls.Initiate.All"

@johanjudai You can find more information about the Calls-related scopes here: https://docs.microsoft.com/en-us/graph/permissions-reference#calls-permissions
Please note, these scopes require Application Permissions, which do not work with public client applications such as SPAs, as public client applications work with Delegated Permissions (which means these scopes will not work with the Implicit Flow, which MSAL.js uses).
You can find more information about Application Permissions here, and you find can an example of setting up a Confidential Client with MSAL.net here.
Also, in the future, please use one of the predefined issue templates, as listed here, thanks!

@hamiltonha  We have many questions on scopes from folks using this library. Can we note this somewhere and have a FAQ which we can refer to? Tagging this as documentation issue assigned to you and me, so we can track this.


Auth Check Rename

Good Afternoon,
The Bot has roles and permissions, the auth sql is set up and bot is setting roles of the members.
The Bot does not seem to be renaming the accounts as per TICKER | Ingame name.
Log shows..
Dramiel.INFO: auth: scopehone authed [] []
Dramiel.INFO: auth: Nhilas Adaar authed [] []
Permissions as follows:
http://prntscr.com/drt2c2
The Bots role 'Bot' is above the roles that it is managing.
Config:
http://prntscr.com/drt2oh
Also, i see there is an 'onTick' to check users per the sql, it does not seem to be removing users roles that are not authed.
Thoughts?
Appreciate the hard work that you have done with this bot shib!

Could this be the problem? As zkill does not seem to be reporting either.
scopeh@localhost:~/marketoracle/log$ php -v
PHP 7.0.14-2+deb.sury.org~trusty+1 (cli) ( NTS )

Thats the thing the Error log is empty.

So killmails was my fault by the looks of it:
[2017-01-05 14:14:32] Dramiel.INFO: Discord WebSocket is ready!  [] []
[2017-01-05 14:14:35] Dramiel.INFO: AuthCheck: Checking for users who have left corp/alliance.... [] []
[2017-01-05 14:14:45] Dramiel.INFO: AuthCheck: Corp/alliance check complete. [] []
[2017-01-05 14:14:45] Dramiel.INFO: AuthCheck: Resetting player names.... [] []
[2017-01-05 14:14:46] Dramiel.INFO: AuthCheck: Names reset. [] []
[2017-01-05 14:14:46] Dramiel.INFO: Killmails: Checking for new killmails. [] []
[2017-01-05 14:14:48] Dramiel.INFO: Killmails: Mail 58898381 queued. [] []
[2017-01-05 14:14:48] Dramiel.INFO: Killmails: Mail 58898391 queued. [] []
[2017-01-05 14:14:48] Dramiel.INFO: Killmails: All kills posted, newest kill id for ICON is 58898391 [] []
[2017-01-05 14:14:48] Dramiel.INFO: Killmails: All kills posted, newest kill id for FCON is 58899412 [] []
[2017-01-05 14:14:48] Dramiel.INFO: Help: Sending help info to scopeh [] []
[2017-01-05 14:14:48] Dramiel.INFO: QueueProcessing - Completing queued item #1 [] []
[2017-01-05 14:14:48] Dramiel.INFO: QueueProcessing - Completing queued item #2 [] []
[2017-01-05 14:19:26] Dramiel.INFO: auth: Katusa authed [] []
[2017-01-05 14:24:48] Dramiel.INFO: Killmails: Checking for new killmails. [] []
[2017-01-05 14:24:49] Dramiel.INFO: Killmails: All kills posted, newest kill id for ICON is 58898391 [] []
[2017-01-05 14:24:49] Dramiel.INFO: Killmails: All kills posted, newest kill id for FCON is 58899412 [] []
[2017-01-05 14:34:49] Dramiel.INFO: Killmails: Checking for new killmails. [] []
[2017-01-05 14:34:50] Dramiel.INFO: Killmails: All kills posted, newest kill id for ICON is 58898391 [] []
[2017-01-05 14:34:50] Dramiel.INFO: Killmails: All kills posted, newest kill id for FCON is 58899412 [] []

Its not removing roles of users that have not authed & not renaming on discord as per the users registered & ticker.
https://cdn.discordapp.com/attachments/224628997441454081/266574889232629760/unknown.png

Close this ticket, looks like the bot was just warming up. haha.

Bot works with php7 fyi

Thanks Shib!
Just to clarify, if i give someone roles within discord and they are not authed, should the bot remove those roles?
Also, if that user was to leave the alliance or standings or reset, should the bot adjust the roles accordingly?
Many Thanks
Nice work!

@scopehs exactly. that's what he does

Yes unless the user the role is set as exempt under the auth settings.

Yes

Perfect and does that cjeck it as part of the 'onTick'


Fix ansi-html vulnerability

Snyk issue: https://security.snyk.io/vuln/SNYK-JS-ANSIHTML-1296849
Implements the fix listed here: https://github.com/Tjatse/ansi-html/issues/19#issue-879174977

ansi-html is unmaintained, so we force a resolution for any indirect dependencies to use ansi-html-community instead

Verified that frontend works fine.
Includes changes from #1048, so you may want to merge that first.

@0xJem wow & it's literally a one character fix https://github.com/mahdyar/ansi-html-community/commit/283cda2de47061615834316f88279292131b3e5a


implement size balanced V4 raw chunk format

Description
Implements #7616.
This PR aims to handle variable length data better than the existing raw index chunk format. It does this by allowing the user to choose a chunk size (default = 1MB) but allows a variable number of values per chunk. The file consists of:

A 16 byte header, which is big endian for backwards compatibility

the version
the target chunk size (which allows the reader to size a buffer for the common case)
the compression type
the offset to the start of the chunks

8 bytes of metadata for each chunk

the first docId in the chunk (sorted). The MSB of the docId offset is used to mark huge chunks.
the byte offset of the chunk. This is an unsigned 32 bit integer supporting up to 4GB of compressed data

The chunks. There are two kinds of chunk:

regular chunks have the same format as chunks in older versions: integer length prefix, followed by N offsets for each of the N values in the chunk
huge chunks just have a raw value, and can be identified by the MSB of the doc offset.

The docId resolution algorithm on the reader side is as follows:

Perform binary search for docId in docId offsets to locate chunk.

If docId in range of current chunk in reader context, go to 5

Check if chunk is huge or regular
If huge, just allocate space for the value, decompress the value if necessary, and return it
If regular, decompress the chunk into a preallocated buffer
If regular, find value in constant time by subtracting docIdOffset from docId and reading Nth value (the length is determined by the next offset or the limit of the buffer as usual)

The format supports entirely incremental construction (does not require segment level statistics) so can be used for real time, and requires bounded, sublinear memory for buffering.
Upgrade Notes
Does this PR prevent a zero down-time upgrade? (Assume upgrade order: Controller, Broker, Server, Minion)

[ ] Yes (Please label as backward-incompat, and complete the section below on Release Notes)

Does this PR fix a zero-downtime upgrade introduced earlier?

[ ] Yes (Please label this as backward-incompat, and complete the section below on Release Notes)

Does this PR otherwise need attention when creating release notes? Things to consider:

New configuration options
Deprecation of configurations
Signature changes to public methods/interfaces
New plugins added or old plugins removed

[ ] Yes (Please label this PR as release-notes and complete the section on Release Notes)

Release Notes

Documentation

Since the target chunk size is much larger than the header size, I think it should not add much overhead to store long offset and remove the 4G limit for single index. We can also include the uncompressed size in the header in case some compressor does not include the length info in the compressed data.

There's a couple of things here:

Compression metadata - this was the purpose of #7655 - to ensure that all formats we use have the correct metadata (3/4 already did) and enforce an upgrade path for LZ4 when using this chunk format. So there's no need for any per-chunk compression metadata, and it factors into the next point
Offset sizes - to me, 4GB of compressed chunks feels like a lot. At a compression ratio of 2x, that's 8GB raw data in a single segment, at 10x (JSON can be amazingly repetitive) it's 40GB. I am aware that in the past 32 bit offsets were shown not to be enough for some use cases, but they were signed offsets so only permitted 2GB compressed data. I am unaware of any evidence that 4GB would not have been enough. Why do I care? It's not for the sake of storage overhead, but I want to keep the metadata as small as possible in memory for the sake of searching it quickly. Can we frame the discussion in terms of why a user would want/need more than 4GB compressed data for a raw column in a single segment?

Hi @richardstartin - Just FYI, I agree that 4GB of compressed data is large. But we hit the 2GB limit, due to storing crawled web pages (many of which were > 100K/page) in a column, when we had more than 100K rows (IIRC) in our per-month segment. We wound up sub-partitioning each month, which was fine, but added complexity to our ETL workflow.

@kkrugler that's great feedback. Given that the constraint was lifted last year when version 3 of the format was implemented, it would be interesting to know whether you would keep the controls in place for your use case (because there are benefits in doing so), or where you would draw the line if adjusting the partitioning limits.

Hi @richardstartin - hard to say whether we'd go for fewer/bigger partitions now or not, as we've tuned the Flink workflows and our Pinot cluster for the current configuration (34 segments/month, and 60 months). And I wish I had a better grasp of Pinot metrics to determine whether our current segment sizes were reasonably optimal or not - it feels like a variant of the Goldilocks problem...not too big, and not too small.

@kkrugler What I'm getting at is I suspect the limit pushed you in the right direction, and a 4GB limit should be comfortable, especially if values are packed in to chunks effectively.

@richardstartin - yes, 4GB should be fine. The point I should have made better is that general rules of thumb about sizing Pinot segments are mostly based on # of queryable columns per segment (and thus the impact on query performance). But in our situation it was just a chunk of non-queryable text data.


Add OctaSine to list of users

Checklist

[x] I have read the Contributor Guide
[x] I have read and agree to the Code of Conduct
[x] I have added a description of my changes and why I'd like them included in the section below

Description of Changes

Add OctaSine to list of users in README file

Related Issues
None

Thanks!


Arc connection issue - Exception occurred while connecting to cluster - Exception calling "Add" with "2" argument(s): "An item with the same key has already been added."

I have tried Arc connecting AKS on several different servers brand new images and I receive the same error on all of them.
HAs anyone seen this error when connecting to ARC and what was your resolution?
Exception occurred while connecting to cluster - Exception calling "Add" with "2" argument(s): "An item with the same key has already been added."
My Linux node  setting below
"LinuxNode":  {
"CpuCount":  6,
"MemoryInMB":  8192,
"DataSizeInGB":  25,
"LogSizeInGB":  1,
"TimeoutSeconds":  300,
"TpmPassthrough":  false,
This is what the log shows
AksEdge - Connecting cluster to Azure Arc failed: Exception occurred while connecting to cluster - Exception calling "Add" with "2" argument(s): "An item with the same key has already been added."_x000A_at Test-ClusterConnectivity, C:\Program Files\WindowsPowerShell\Modules\AksEdge\1.6.384.0\AksEdge.psm1: line 3326_x000D__x000A_at Set-ArcConnectionInternal, C:\Program Files\WindowsPowerShell\Modules\AksEdge\1.6.384.0\AksEdge.psm1: line 3212_x000D__x000A_at Connect-AksEdgeArc, C:\Program Files\WindowsPowerShell\Modules\AksEdge\1.6.384.0\AksEdge.psm1: line 2662_x000D__x000A_at , : line 1

I have the same issue.

Facing the same issue.

To resolve this issue, redeploy using the July 2024 update, or try the following work around:
Confirm the permissions assigned for the ServicePrincipal - Ensure the ServicePrincipal has "Kubernetes Cluster - Azure Arc Onboarding" & "Contributor" roles at ResourceGroup (or Subscription) level.
If the above is confirmed, then the following workaround:
#Replace "xyz" in the instructions below with corresponding values
cp 'C:\Program Files\WindowsPowerShell\Modules\AksEdge\1.6.384.0\AksEdge*' C:\Users\xyz\desktop\AksEdge\
Open C:\Users\USERNAME\desktop\AksEdge\AksEdge.psm1 and replace Test-ClusterConnectivity with below implementation:
function Test-ClusterConnectivity
{
param (
[Parameter(Mandatory)]
[System.Object]$ConnectClusterArgs
)
try
{
#increasing wait time for cluster connectivity check to an hour
$res = $false
$clusterArgs = @{
ClusterName = $ConnectClusterArgs['ClusterName']
ResourceGroupName = $ConnectClusterArgs['ResourceGroupName']
}
Write-SubStatus "Connecting cluster to Azure, try count : $retries"
# [Bug]: [ArcK8s] Arc fails to connect even after a couple of retires (https://dev.azure.com/msazure/msk8s/_workitems/edit/25630601/)
# Sometimes the connectivity status from the New-AzConnectedKubernetes indicate Connecting which could result in incorrect/unnecessary retry.
# Additionally, higher back off interval of 5 min and lack of logs between reties result in a behavior which resembles a hang up screen.
# Fixing this by using onboarding timeout parameter supported by New-AzConnectedKubernetes command to 10 min for cluster to connect(typically the connect commands succeeds under 2 min).
# Log between the retry attempts to user, retry in a determinitic way and exit the connect command if we are unable to connect to Azure under an hour ([ 10 min (onboarding timeout) + 1 min (backoff timeout)] * 5 (retries) = 55 min)
New-AzConnectedKubernetes -OnboardingTimeout 600 -ProvisioningState 'Succeeded' @ConnectClusterArgs 2>&1 | Out-File -FilePath $ArcConnectionLocation -Append
        for($retries = 1; $retries -le 20; $retries++)
        {
            $arcConnected = Get-AzConnectedKubernetes @clusterArgs -ErrorAction SilentlyContinue 2>&1

            if($arcConnected.ConnectivityStatus -eq "Connected")
            {
                Write-Substatus "Cluster reached connected status"
                $res = $true
                break
            }

            Write-SubStatus "Retrying in 1 minute.."
            Start-Sleep -Seconds 15
        }
    return $res
}
catch
{
    $err = $_.Exception.Message.ToString()
    $msg = "Exception occurred while connecting to cluster - $err"
    Write-Status $msg -color Red
    throw $msg
}

}
Save file
Remove-Module AksEdge; Import-Module "C:\users\xyz\Desktop\AksEdge\AksEdge.psm1" -Force
Suggest try a new deployment with a different cluster name:
Remove-AksEdgeDeployment -Force
New-AksEdgeDeployment...


Rename bitfield to bitfield_width

Its more clear what the methods/fields are returning/storing when we add "width"
to the name.
r? @pepyakin

Yeah, thats very helpful!
@bors-servo r+

:pushpin: Commit 1f15e60 has been approved by pepyakin

:hourglass: Testing commit 1f15e605137edf34ec4ee239cbfc1bdee9fd0279 with merge bcf0832760328afbc417784fb95a9c499ade3c7a...

:sunny: Test successful - status-travis
Approved by: pepyakin
Pushing bcf0832760328afbc417784fb95a9c499ade3c7a to master...


Add conditional requirement

Supersede https://github.com/MobilityData/gbfs-validator/pull/59 at the request of @PierrickP
This PR adds a first batch of conditional checks such as:

Files required depending on the content of other files
Fields required depending on the content of other files
Value of fields depending on the content of other files (e.g. the validity of ids referring to an object in another file)

From @isabelle-dr https://github.com/MobilityData/gbfs-validator/pull/59#issuecomment-992410246 :

default_pricing_plan_id in vehicle_types.json is checked (requirement and value)
vehicle_types_available in station_status.json requirement is checked
num_docks_available in station_status.json can't be automatically checked because it depends on something not defined in the files: REQUIRED except for stations that have unlimited docking capacity. It could be done in a separate PR as a checkbox option in the validator UI such as "[  ] All stations limited docking capacity".
vehicle_docks_available in station_status.json can't be automatically checked because it depends on something not defined in the files: REQUIRED in feeds where [...] certain docks are only able to accept certain vehicle types. It could be done in a separate PR as a checkbox option in the validator UI such as "[  ] All stations accepts all vehicle types".
system_id in free_bike_status.json is not touched by that PR.
lat and lon in free_bike_status.json : As the conditional is between two fields of the same object, it has been fixed in the schemas: https://github.com/MobilityData/gbfs-json-schema/pull/66
vehicle_type_id in free_bike_status.json is checked (requirement and value)
current_range_meters in free_bike_status.json requirement is checked
the file vehicle_types.json conditional requirement is checked

Also, this PR adds the following checks:

The conditional requirement of system_pricing_plans.json
The conditional requirement and value of default_pricing_plan_id in vehicle_types.json
The conditional requirement of rental_apps in system_information.json
The conditional requirement of rental_apps in system_information.json
The conditional requirement of rental_apps.ios.store_uri and rental_apps.android.store_uri in system_information.json

Other changes on this PR:

Update of https://github.com/MobilityData/gbfs-json-schema (it was required as tests were falling due to invalid json syntax)
Adds an option to download the generated schema used for the validation

Thank you for working on this! 🙏

Hello,
This contribution is amazing 🔥
I opened #64 to update the documentation so it reflects all this work. I know we require one review before merging - can you wait for a review by a MobilityData as well in the future?
🙏🙏🙏

Hi @PierrickP,
thanks for your work!
I suspect that this change broke https://github.com/MobilityData/gbfs-validator/pull/41. For v2.2 feeds the vehicle_types.json is thought to be required again, even if the criteria stated in GBFS.md

REQUIRED of systems that include information about vehicle types in the free_bike_status file.

is not given.
Example feed: https://gbfs-validator.netlify.app/?url=https%3A%2F%2Fgbfs.nextbike.net%2Fmaps%2Fgbfs%2Fv2%2Fnextbike_le%2Fgbfs.json
which states

Missing file and required

for the vehicle_types.json.
Should I open a new issue?
Best regards


[WIP] chore(semantic-release): #64 - Enable semantic release

Configuration based on recipe  for https://github.com/marketplace/actions/action-for-semantic-release
Currently set to :

branches - master and semantic-release (for testing)

Job output https://github.com/grzesuav/metac/actions/runs/57755684

click on Release on the left
you can expand to see Release in logs

I have triggered a release on my fork :
https://github.com/grzesuav/metac/releases
PLEASE DO NOT MERGE

PLEASE DO NOT MERGE YET

Addressing also https://github.com/AmitKumarDas/metac/issues/87

Thank you @grzesuav for your contribution. It surely helps us in enabling smoother release process


Registration

How would Registering a new user for  Developer Authenticated look like?
Been trying to figure that out.

I would begin by looking at the Cognito docs (http://docs.aws.amazon.com/cognito/devguide/identity/developer-authenticated-identities/ make sure to select iOS as the platform in the upper right) and the blog post https://mobile.awsblog.com/post/Tx3E3NJURV1LNV1/Integrating-Amazon-Cognito-using-developer-authenticated-identities-An-end-to-en

Thanks. I was able to register inside the ios cognitosyncdemo app. But it isn't secure because i am solely passing a url unencrypted to /registeruser. If anyone wants to see my implementation let me know.