PowerShell Core test result data should be available via public dashboard

testability
this should include results from daily run as well as CI
see https://github.com/PowerShell/PowerShell/blob/master/docs/testing-guidelines/TestRoadmap.md

@JamesWTruher isn't this technically done? Or are we leaving it open until we have good coverage numbers again?

We should do this, but this doesn't block 6.0.0 release

this is done


[10.x] Add getWhere To Eloquent Builder

A shortcut for ->where(...)->get().
Example:
// Before
User::where('status', 'active')->get();

// After
User::getWhere('status', 'active'');

If something was missing, please tell me so that it can be improved.

Thanks for your pull request to Laravel!
Unfortunately, I'm going to delay merging this code for now. To preserve our ability to adequately maintain the framework, we need to be very careful regarding the amount of code we include.
If possible, please consider releasing your code as a package so that the community can still take advantage of your contributions!
If you feel absolutely certain that this code corrects a bug in the framework, please "@" mention me in a follow-up comment with further explanation so that GitHub will send me a notification of your response.

getWhere can be good feature, it will be good if we can add this in Laravel Direct @taylorotwell
Nice work @milwad-dev


Fix/tasks scroll behavior

Description
Fix: tasks doesn't scroll automatically to make visible the focused new task when is necessary
Type of Change

[ ] ✨ New feature (non-breaking change which adds functionality)
[x] 🛠 Bug fix (non-breaking change which fixes an issue)
[ ] ❌ Breaking change (fix or feature that would cause existing functionality to change)
[ ] 🧹 Code refactor
[ ] ✅ Build configuration change
[ ] 📝 Documentation
[ ] 🗑 Chore

Codecov Report

Merging #18 (d515a14) into main (00bdf2e) will decrease coverage by 0.07%.
The diff coverage is 96.82%.

@@            Coverage Diff             @@
##             main      #18      +/-   ##
==========================================
- Coverage   98.42%   98.34%   -0.08%
==========================================
  Files         102      102
  Lines        2220     2235      +15
==========================================
+ Hits         2185     2198      +13
- Misses         35       37       +2

Impacted Files
Coverage Δ

lib/planner/layout/planner_layout_builder.dart
96.61% <96.00%> (-3.39%)
:arrow_down:

lib/planner/widgets/planner_tasks.dart
100.00% <100.00%> (ø)

lib/task/view/task_page.dart
95.23% <100.00%> (+0.23%)
:arrow_up:

lib/task/widgets/task_text_field.dart
97.56% <100.00%> (+0.12%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 00bdf2e...d515a14. Read the comment docs.


envoy: Update to Envoy version to 1.20.0

Updates Envoy version to 1.20 and validated
that there are no warnings with the existing config.
Should also fix #4141
Signed-off-by: Sotiris Nanopoulos<EMAIL_ADDRESS>
Description:

Testing done:

Affected area:

Functional Area

New Functionality
[ ]

CI System
[ ]

CLI Tool
[ ]

Certificate Management
[ ]

Control Plane
[ ]

Demo
[ ]

Documentation
[ ]

Egress
[ ]

Ingress
[ ]

Install
[ ]

Networking
[ ]

Observability
[ ]

Performance
[ ]

SMI Policy
[ ]

Security
[ ]

Sidecar Injection
[ ]

Tests
[ ]

Upgrade
[ ]

Other
[ ]

Please answer the following questions with yes/no.

Does this change contain code from or inspired by another project?

Did you notify the maintainers and provide attribution?

Is this a breaking change?

@anrossi

Don't close this issue yet. The work is ongoing.

@rectified95 FYI

Thank you @anrossi and @rectified95 for working on this!


gertty: init at 1.6.0

Motivation for this change
Add gertty package to nixpkgs.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

Notify maintainers
cc @

Hey @Dav1d23 do you still use Nix and care to get this packaged?
We can help you make the necessary changes.

Hello,
yes, I still use Nix, but I don't really use this package anymore.
As far as I understand, I only put the derivation in the wrong place, but
now something else might be missing or it may just not work anymore!
So, please go ahead with the needed changes if that's easy for you, I doubt
I will have time to put into this anytime soon, sorry :(
On Sun, 5 Jun 2022, 05:52 Daniel Schaefer, @.***> wrote:

Hey @Dav1d23 https://github.com/Dav1d23 do you still use Nix and care
to get this packaged?
We can help you make the necessary changes.
—
Reply to this email directly, view it on GitHub
https://github.com/NixOS/nixpkgs/pull/74998#issuecomment-1146736189, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ACMQLP6ZQKJ4CWKPKUU7UULVNQQBHANCNFSM4JVOK4LA
.
You are receiving this because you were mentioned.Message ID:
@.***>


How to create sequence flow between 2 tasks

Hi,
I'm working on a BPMN file generator on the back-end side based on customer data.
I managed to create start event, tasks and end event. And now I'm having trouble connecting them with a sequence flow.
Here's a POC code:
`const BpmnModdle = require('bpmn-moddle');
const fs = require('fs');
exports.buildSimpleBpmnDiagram = async (processName, tasks) => {
const moddle = new BpmnModdle();
let {rootElement: processElement} = await fromFile(moddle, `${__dirname}\\templates\\defaultTemplate.bpmn`);

let model = processElement.$model;

let flowElements = processElement.rootElements[0].flowElements;

let taskElement = model.create('bpmn:Task', { id: 'Task_1'});
let taskElement2 = model.create('bpmn:Task', { id: 'Task_2'});
let sequenceElement = model.create('bpmn:SequenceFlow', { id: 'Sequence_1', sourceRef: taskElement, targetRef: taskElement2});

flowElements.push(taskElement, sequenceElement, taskElement2);

let planeElements = processElement.diagrams[0].plane.get('planeElement');

planeElements.push(
    model.create('bpmndi:BPMNShape', {
        id: 'TaskShape_1',
        bpmnElement: taskElement,
        bounds: model.create('dc:Bounds', {x: 200, y: 100, width: 100, height: 100})
    }),
    model.create('bpmndi:BPMNShape', {
        id: 'SequenceShape_1',
        bpmnElement: sequenceElement
    }),
    model.create('bpmndi:BPMNShape', {
        id: 'TaskShape_2',
        bpmnElement: taskElement2,
        bounds: model.create('dc:Bounds', {x: 400, y: 100, width: 100, height: 100})
    })
);

const result = await moddle.toXML(processElement, {preamble: false});
return result.xml;

};
function fromFile(moddle, file) {
return fromFilePart(moddle, file, 'bpmn:Definitions');
}
function fromFilePart(moddle, file, type) {
const fileContents = readFile(file);
return moddle.fromXML(fileContents, type);
}
function readFile(filename) {
return fs.readFileSync(filename, {encoding: 'UTF-8'});
}
`
I did not manage to add the "outgoing"/"ingoming" properties to the tasks
And I did not manage to add the "waypoint" to the sequence shape.
Please assist,
Thanks

Thanks for reporting. Mind you sharing your code inside CodeSandbox  in a way that we can inspect it? Btw. the forum would be an even better place for questions like this.

I'm closing this due to inactivity. Please feel free to re-open once you have a specific feature request or bug report.


puzzle mode breaking due to garbage weirdness

puzzle mode is working again, i suppose someone has fixed it?


Not able to add DO node templates from "Add Template" option.

Rancher server version - Build from master
Steps to reproduce the problem:
Create a DO template using "Add Template" option.
DO template creation fails with following error:

Version - 2.0 master 3/1


Television

Hello,
I am investigating your television data.
It appears that there is a significant amount of time when the television is not being used.
Is this a mistake? How could you explain this?
Thanks

The sensor used to capture this was an infrared receiver. So either the tv has been turned on using its dedicated switch on button, the receiver didn't capture the signals or the television was really not used which you can simply cross check with the power meter data.


Print jobs hang with custom FoomaticRIPCommandLine and glibc 2.28

*FoomaticRIPCommandLine: "(printf '\033%%-12345X@PJL\n@PJL JOB\n@PJL SET COPIES=&copies;\n'%G|perl -p -e "s/\x26copies\x3b/1/");
(gs -q -dBATCH -dPARANOIDSAFER -dNOPAUSE -dNOMEDIAATTRS -dNOINTERPOLATE %B%A%C %D%E | perl -p -e "s/^\x1b\x25-12345X//" | perl -p -e "s/\xc1\x01\x00\xf8\x31\x44/\x44/g");
(printf '@PJL\n@PJL EOJ\n\033%%-12345X')"
*End

Ricoh and Savin pxlmono printers have custom shell scripting in FoomaticRIPCommandLine. These printers hang at Processing with 0% CPU. As I reported for Ricoh:

Bisecting with the ALA shows that 2018/08/05 prints to the MP 2352 and 08/06 hangs. Partial upgrading a few packages at a time listed with pacman -Qqu from 8/5 to 8/6 shows that the printing problem is an incompatibility with glibc 2.28. Unfortunately downgrading glibc is not an option because many system components require glibc>=2.28 including Chrome and Firefox.

Test printers:
Ricoh Aficio MP 2352 , pxlmono-Ricoh 20161206 (OpenPrinting LSB 3.2)
Savin 9060 , pxlmono-Savin 20161206 (OpenPrinting LSB 3.2)
I put the lines into a shell script and added, changed, and removed lines. Nothing would print. The only way I found to get these printers to print is to remove all the extra shell scripting and leave only the gs command similar to the one in the  HP LaserJet 4100 Foomatic/pxlmono which prints without hang and as described here.
Printing [Foomatic] FoomaticRIPCommandLine for PDF as Standard Print Job Format?
This states that cups changes behavior between a FoomaticRIPCommandLine that starts with "gs " and one that starts with anything else.

I have tried to modify the ppd and force the FoomaticRIPCommandLine to "gs -q -dBATCH -dPARANOIDSAFER -dNOPAUSE -dNOINTERPOLATE %B%A%C %D%E". In this case no pdftops conversion is performed, but I'm not sure what I am losing. Why are the extra printf/perl commands needed? Am I losing some printing options in this way? If this is true, couldn't this options be added in another way?
aitziber

In my case preventing the extra step stops the hanging.
HTop:
11721 root       20   0 82264 14064  7280 S  0.0  0.0  0:00.46 ├─ /usr/bin/cupsd -l
16116 cups       20   0 82676 15880  6964 S  0.0  0.0  0:00.01 │  ├─ TestSavin 333 root Test Page 1 job-uuid=urn:uuid:9450815b-db4d-30bd-6873-b9c495e33601 job-originating-host-name=localhost date-time-at-creation= date-time-at-processing= time-at-creation=1536369858 time-at-processing=1536369858
16117 cups       20   0 58256  6232  4880 S  0.0  0.0  0:00.01 │  │  └─ pdftops 333 root Test Page 1  job-uuid=urn:uuid:9450815b-db4d-30bd-6873-b9c495e33601 job-originating-host-name=localhost date-time-at-creation= date-time-at-processing= time-at-creation=1536369858 time-at-processing=1536369858 /var/spool/cups/tmp/foomatic-YpTCST
16119 cups       20   0 59448  7080  5748 S  0.0  0.0  0:00.00 │  │     ├─ pdftops 333 root Test Page 1  job-uuid=urn:uuid:9450815b-db4d-30bd-6873-b9c495e33601 job-originating-host-name=localhost date-time-at-creation= date-time-at-processing= time-at-creation=1536369858 time-at-processing=1536369858
16118 cups       20   0  196M 27632 13092 S  0.0  0.1  0:01.74 │  │     └─ gs -q -dNOPAUSE -dBATCH -dSAFER -dNOMEDIAATTRS -sDEVICE=ps2write -dShowAcroForm -sOUTPUTFILE=%stdout -dLanguageLevel=3 -r600 -dCompressFonts=false -dNoT3CCITT -dNOINTERPOLATE -c save pop -f /var/spool/cups/tmp/foomatic-YpTCST

cupsctl --debug-logging, print, cancel job
job334.log.zip

Please file this bug against the cups-filters project. We have nothing to do with foomatic in this project...


#6356: Remove half-half grid logic for bmms

Reader kernels are using passed in grid while compute uses grid from split_work_to_cores
This can cause hangs unless we rigorously check passed in compute grid

Passing post-commit: https://github.com/tenstorrent-metal/tt-metal/actions/runs/8510565047


M64 release notes

M64 release notes blog post!

Any community spotlight? (I'd love us to make this a standard thing for all release notes!)


How to get specific SHAs and releases

The title basically says it all. How do I get another sha then master and how do I get a specific release? Couldn't find info about that in the readme.

Hi, thank you for reporting.
I just run git checkout in submodule, but you gave me good idea, I implemented this feature, now you can specify commit-ish using = delimiter, like manul -I golang.org/x/blah=tag-1 or manul -U golang.org/x/blah=old-blah
Implemented in https://github.com/kovetskiy/manul/commit/9780bece0ac3b054d830f1a91222bd645ce4b11e
Feel free to close the issue if it works for you.

Wow, that was ⚡ fast. I am gonna try it on a couple of repos and report back later. Thank you 👍

Trying to go get the current master gives me this error:
# cd /home/klautcomputing/go/src/github.com/kovetskiy/manul; git submodule update --init --recursive
Submodule 'vendor/github.com/PuerkitoBio/goquery' (https://github.com/PuerkitoBio/goquery) registered for path 'vendor/github.com/PuerkitoBio/goquery'
Submodule 'vendor/github.com/kovetskiy/godocs' (https://github.com/kovetskiy/godocs) registered for path 'vendor/github.com/kovetskiy/godocs'
Submodule 'vendor/github.com/kovetskiy/lorg' (https://github.com/kovetskiy/lorg) registered for path 'vendor/github.com/kovetskiy/lorg'
fatal: No url found for submodule path 'vendor/github.com/reconquest/coproc.bash' in .gitmodules
package github.com/kovetskiy/manul: exit status 128```

I just ran it again on a different machine and it worked, came back to the first one and it worked there as well 👍  Tested it and it works.
Thank you again for building this!


What a cool module!

Sorry it's not really an issue I'm addressing about.
When using the googlesearch module's search  function,  can an additional parameter such as specifying a site: www.wikipedia.com to Google Index be accepted? I want to limit the results so that when I execute the search function, all related URL links, based on the given query, gets retrieved from sources in the Wikipedia website.

Do you mean this?
>>> for i in search("Google site:wikipedia.com"):
...     print(i)
...
https://www.wikipedia.com/wiki/Topic-Sensitive_PageRank
https://wikipedia.com/wiki/Google_(verb)
https://en.wikipedia.com/wiki/History_of_Google
https://www.wikipedia.com/wiki/Alphabet_Inc.
https://en.wikipedia.com/wiki/Google_Earth
https://www.wikipedia.com/wiki/Category:Google
https://www.wikipedia.com/wiki/Google_Video
https://www.wikipedia.com/wiki/Waymo
https://en.wikipedia.com/wiki/Googol
https://wikipedia.com/wiki/Google_logo
https://en.wikipedia.com/wiki/Knowledge_Graph
https://en.m.wikipedia.com/wiki/Android_(Operating_System)
https://www.wikipedia.com/wiki/Google_Hangouts

Oooh! So you can add a "site" argument as a parameter to the googlesearch function? That's cool ! The documentation should include that.. correct me if I'm wrong, is it stated?

Well, that is standard Google. Not module stuff.
See https://support.google.com/websearch/answer/2466433 and https://www.spyfu.com/blog/google-search-operators/


Update NFR and glossary

Codecov Report

Merging #302 (8c3f3fd) into master (ec84926) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #302   +/-   ##
=========================================
  Coverage     75.02%   75.02%
  Complexity      696      696
=========================================
  Files           110      110
  Lines          2102     2102
  Branches        250      250
=========================================
  Hits           1577     1577
  Misses          435      435
  Partials         90       90

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ec84926...8c3f3fd. Read the comment docs.


fix: get tags one per line

Fixes https://github.com/GothenburgBitFactory/timewarrior/issues/581 for me, not sure if this is the best solution, though.

Nope, this is wrong, breaks up tags that have spaces in them..


Drag always triggers longpress...

Hi!
I'm running into an issue where drags are consistently triggering the long-press callout. I’m not overriding the default long-press implementation, yet the callout still appears even when my finger movement clearly exceeds the supposed 10-point threshold. Is there something I might be missing in the configuration?
I also noticed https://github.com/KeyboardKit/KeyboardKit/issues/469 was mentioned as addressing related gesture behavior, but I couldn’t find any corresponding documentation. Was this ever released?
Additionally, I’m attempting to implement swipe typing by calculating each key’s position. However, the UI still behaves like a standard drag: the key enlarges and the callout pops up. I’d like to prevent these default behaviors specifically during a swipe, while still keeping them when a user actually presses or long-presses. Is there a recommended way to achieve this?
Thanks in advance for any guidance!

realted to this, was this released? I do not see anything about this in the documentation...
https://github.com/KeyboardKit/KeyboardKit/issues/469
Thanks!

I see this was released as a milsetone but you ve mentioned in a few posts that it is a big undertaking to make a keyboard wide gesture. I am trying to incorporate swipe typing, and i am calculating the positions on each key, but it just doesnt behave like a drag on the UI in that the specific key enlarges and then the callout pops up. Can I get both of those to stop, while having them remain when you actually press or longpress, just not on drag?

Hi @ben1787
The secondary action callout will currently always show if a key with secondary actions is long pressed, but I agree that it shouldn't show when the gesture point is way off from the key's frame (e.g. when swiping). I'll adjust it to behave like that.
The single gesture configuration is a feature that I've been wanting to implement for so long, since it's also required to make swipe typing work. But while it seems feasible, it holds many hidden complexities.
For instance, the native keyboard allows you to tap the keyboard with multiple fingers, where the last finger will take over as the primary gesture. I'm not sure how to implement this in SwiftUI.
If you have any ideas or want to look at this together, just let me know. Otherwise, I'll focus on KeyboardKit all through H1 next year, with hopes to finally get some of these old features in place.

Honestly, I think I can manage with the gestures as they are by calculating the key locations.
One thing that would be really convenient is if I could simply get the button for a given CGPoint. That way, I wouldn’t have to manually calculate button positions, which can get quite complicated due to their non-uniform layout.
But i can make do at least with the simple update of don't initiate longpress on a drag, don't have callout popup on a drag. If i am dragging and not on the initial button, i would ideally not be seeing any UI anything associated with the button where the drag started since i am not touching that button anymore.
Why do you think that single gesture config is required for swiping? Definitely happy to brainstorm with you if you like at some point!


Add triage-incoming.yml

Taken from https://github.com/matrix-org/synapse/blob/develop/.github/workflows/triage-incoming.yml
Example usage in https://github.com/matrix-org/matrix-appservice-bridge/actions/runs/3274938737

The warning looks like a fault with the module (it seems to do the same thing with Synapse, and we're definitely using the intended syntax https://github.com/matrix-org/synapse/actions/runs/3274133082)

Would you mind changing Synapse to point to this?

Yeah, can do post release.

 (Merge PR, create github release with semver bump, force push the v1 tag to point at the release's tag)

Yeah, can do!

Thanks!


JOML - 'mallocInt' - Out of stack space.

Environment

LWJGL version: 3.1.6
LWJGL build #: 14
Java version: JDK 9
Platform: Windows
Module: JOML

Description
I'm trying to read the stanford dragon.obj model into my engine and I get this error: java.lang.OutOfMemoryError: Out of stack space.
And It's point exactly to my createBuffer method where I use mallocInt to store indices VBO. I'm using the recommended allocation method described in JOML wiki.
Here's the allocation code:
`
public IntBuffer createBuffer(int[] data) {
IntBuffer buffer = null;
try (MemoryStack stack = MemoryStack.stackPush()) {
    buffer = stack.mallocInt(data.length);
    buffer.put(data).flip();

    return buffer;

}

}
`
I don't know what's causing this, the stack have a data limit? I can't find any information about this.

Yes, the stack has a fixed capacity. It defaults to 64kb, can be changed programmatically with Configuration.STACK_SIZE or via the command line with -Dorg.lwjgl.system.stackSize=<value in kb>.
Note that MemoryStack is supposed to be used for local/short-lived/small allocations. Model data do not qualify (it's typically large) and heap allocation (e.g. via MemoryUtil) is more appropriate.

@AwfulMint I wonder how you got this information from JOML's wiki. There is not a single reference in JOML's wiki to LWJGL's MemoryStack or MemoryStack.stackPush() and nowhere is there a mention that this is "the recommended allocation method" (though it might be for small matrices).

Thanks @Spasi for the tips, and to @httpdigest to tell me this important info about small matrices, I fixed the "bug" :D

if you use a stored instance of MemoryStack clStack = MemoryStack.stackPush(); then you continue to push on the same stack without anything getting freed. so instead of using a stored stack, get a new stack local instance for each class method that needs to temporarily store stack variables.


Q: can get vertical scroll position (window.scrollY) thru this library?

I liked to have an icon to scroll-to-top when my page was scrolled down and with this helpful library, that was nice and easy.. but with a small side effect:
I used to monitor window.scrollY value to show the icon and after adding this library, its value doesn't seem to be as responsive as before (slow to increase, not decreasing, not resetting after scrollToTop, etc.).

Currently no support for vertical scroll/position. Couldn't you reset window.scrollY by yourself when calling scrollToTop?

Ok, I had to addEventListener for scroll and get window.scrollY inside it. Now it responds well as expected. Closing this issue.


[HELM] Kibana extra volumes

Description of changes:
User can define extra volumes for Kibana
Test Results:

Deploy using the following chart values :

kibana:
  extraVolumes:
    - name: my-ca-trust
      hostPath:
        path: "/etc/ssl/certs"

  extraVolumeMounts:
    - name: my-ca-trust
      mountPath: /usr/share/kibana/config/certs
      readOnly: true

Check the volumes on the Kibana pod

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Hi @rexbut thanks for this PR about kibana containers.
We will review it within our team
Thanks.

PR merged, thanks @rexbut for the contribution.


Trunk check failing in Github action even though it shows no issue

This is our PR workflow:
# pr.yml
name: PR workflow
on:
    pull_request:
        types: [opened, reopened, synchronize]
        branches:
            - main
env:
    NODE_AUTH_TOKEN: ${{ secrets.GP_REPO_PACKAGE_R }}
jobs:
    pr:
        name: PR Checks
        runs-on: ubuntu-latest
        timeout-minutes: 15
        steps:
            - name: Checkout
              uses: actions/checkout@v2

            - name: Cache Linters/Formatters
              uses: actions/cache@v2
              with:
                  path: ~/.cache/trunk
                  key: trunk-${{ runner.os }}

            - name: Setup Node
              uses: actions/setup-node@v2
              with:
                  node-version: 'lts/*'

            - name: Install dependencies
              run: yarn --frozen-lockfile

            - name: Trunk Check
              uses<EMAIL_ADDRESS>
Trunk check shows no issue:

But in the Github action we're getting below:

Can you help us understand what's going on?

Discussed on slack, but closing the loop here: This was caused by not having committed .trunk/trunk.yaml into the repo yet (produced by running trunk init), we'll make sure it prints a good error for this in the future!


KAFKA-3163: Add time based index to Kafka.

This patch is for KIP-33.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-33+-+Add+a+time+based+log+index

It seems this conflicts with KAFKA-3510. I'll do the rebase.

cc @junrao. Will you be able to help review this patch? Thanks!

Also, could you patch the DumpLogSegment tool to support the time index?

@becketqin, it would be nice to have some numbers for how long it takes to do search by timestamp before and after the change on a cluster with a reasonable amount of data.

@ijuma Is there a specific use case where the performance of searching by timestamp matters much? Supposedly searching by timestamp should be a relatively rare operation. With LogAppendTime, the time will be similar to offset search. With CreateTime, the time cost largely depends on how the data look like. i.e. if we have to scan tons of logs, it is going to be slow, otherwise it is fast. One thing for sure is that it is going to be slower than before which did not do much except for looking at the last modification time of segment files.

@becketqin As a general rule, we should be very thorough with regards to performance testing whenever we introduce a new disk structure (like an index). I gave one example, but we should really also verify the impact on normal operations as well. Even if we believe that this should be minimal, we cannot be sure until we have tested it.

@Ishiihara Thanks for the review.
@ijuma I see, yes, I agree that we should understand the performance impact of the changes we make in general. I was just curious if you care about the search by timestamp specifically.
@junrao I updated the patch to let the index point to the offsets of shallow messages instead of the next offsets. Would you be able to take another look? I am not sure if we are still able to make it into 0.10.0 given 0.10.0 should have cut off on yesterday.

@junrao I addressed your previous comments. Would you take another look?

@becketqin : Thanks for updating the patch. As the patch is getting closer to be ready, could you run your branch on the system tests? Also, could you do some performance testing like you did in KIP-31/32 to see if there is any noticeable performance degradation?

@junrao Thanks a lot for the review. I have updated the patch. The only pending thing is whether we want to allow the empty time index or not. I replied to your comment. Please let me know if you think the benefit is good to have or not.
I have sent email to the mailing list regarding the time based log rolling behavior change to see if people have any concern.
I will run the performance test as we did for KIP-31/32 and update the result.

@junrao Thanks a lot for the review. I just updated the patch to address your comments. I will create another KIP to discuss the new search for timestamp semantic.

@becketqin : Thanks for the latest patch. I made another pass and left some comments. Most of them are minor though. Once you have addressed those, it would be good if you could run through our system tests and post some performance testing results on the jira. Ideally, we probably want to load some sizable amount of data on at least 2K - 4K partitions in a broker and see if there is any performance degradation.

@junrao Thanks a lot for the patient and careful review. I am on vacation in China from Jun 22 to Jul 11. The network becomes a little flaky. Could you help kick off a system test? I will see if I can go to our newly opened Shanghai office and use the VPN there to run the performance test on our test cluster. Thanks again.

Also, in DumpLogSegments, could we dump the timestamp in the .log file too?

@becketqin : I made another pass and left a few more comments. It would be good if you can run some performance tests to see if there is any degradation. For instance, with this PR, we will need to read the last entry of each time index during broker startup. Not sure how much impact this will have on starting time. So, it will be good to test this out.

@junrao Thanks for the review and sorry for the late response. I just returned from China yesterday. I have updated the patch to address your comments. Because we are in the middle of data center cutting over, it may take some time to setup the new dev test environment. I will do the performance test this week.

Also, I left a comment in the jira about making the output of DumpLogSegments clearer. Could you take a look?

@junrao I just updated the patch to address the latest comments. One thing worth mentioning is that dumping the active time index may be slow because time index requires looking up the offset index. The offset index of the active segment is always the max index size. This will cause the dump log segment tool to create an offset index with a lot of empty index entries in the end so the binary search in this case does not quite work. Also because the active segment is still growing, dumping it may have some race condition and result in false alarm. So I suppose it is fine to assume that people won't dump the active log segment.

@becketqin : Thanks for the latest patch. It looks good to me. Do you have any updates on performance tests and the system tests?

@junrao Thanks for the review. I have tested the produce and consume performance with the same script we used for KIP-31/32. The producer and consumer performance are the same.
Our new dev clusters are ready for use now, but I am still waiting for the mirror maker instances, which should be setup by today. I will run the performance test on the real data and test the segment loading and recovery time with more partitions after that.

For system test, I was trying to use the confluent jenkins server before. But it seems that it does not support building against an arbitrary branch anymore. I am setting up a Hudson job to run the system test internally at this point. But that may take some time as I am not familiar with that.

@becketqin, you can use https://jenkins.confluent.io/job/system-test-kafka-branch-builder/

@ijuma Got it. Thanks.

@junrao Sorry for the late response. I have run the performance test. On a broker with ~15500 log segment files. The log segments loading time is about 6 seconds without this patch. On another broker with 17700 log segments files, the log loading time was 35 seconds with this patch. The results varied a little from run to run, but it is 4x - 6x slower when we need to load the timestamp from the log segment. For a large Kafka cluster, this might be an issue. One optimization may be load the log maximum timestamp lazily. I'll update the new patch after testing that.
I did not see any throughput issue with this patch. So the impact is only at startup time.

@becketqin : Thanks for the results. So the 4x-6x overhead mostly comes from reading the last entry of the time index?
You also did producer/consumer perf test on a large number of topic/partitions and saw no degradation in throughput?

@junrao I made some further test. The previous 4x - 6x slow down may not be very accurate. I noticed a bug in the code that may cause unnecessary rebuild of the indexes if a time index is empty.
After fixing the bug, on a broker with ~20000 log segment files, I got the following results:

log loading time: ~13 seconds without time index. 27-28 seconds with time index.:
I tested a patch that skips sanity check and loads the max timestamp lazily so we don't read the last entry at starting time. The time reduced by 1-2 seconds but not much.

So it looks that the slow down was mainly caused by the system calls that create the memory mapped files. Because we are creating 2x many memory mapped files, it takes about 2x of the time.
I tested the throughput with part of the production data. The brokers handled fine. And the produce request total time and local time looks similar. I will try to push more traffic and see if that makes any difference.

@becketqin : Thanks for the update. Is memory-mapping the index really that expense? Could you do some micro-benchmark to test this out?
Also, have you run the system tests?

@junrao Yes, I have run system test and it passed. Sure, I can run the micro benchmark on mmap.

@junrao the link to system test run. https://jenkins.confluent.io/job/system-test-kafka-branch-builder/483/parameters/

@junrao I did some investigation on the segment loading and here are the findings:

mmap() and resize() takes much time during reloading.
mmap() speed dramatically slows down after 20000 files.
resize() is also much slower when timeindex files are added.

The summary can be found here:
https://docs.google.com/spreadsheets/d/14aahuei8dXALtwniyWDpNrsNtEuYc1-L5mPwRhDnWuQ/edit#gid=550436079
In the graph each bucket contains 250 mmap() or resize() call. The time is the summary of the time taken by all the calls in that bucket. For example, the first 250 mmap() calls during log loading goes to the first bucket, the next 250 mmap() calls goes to the second bucket, and so on.
I am not sure what is the right solution here. Maybe there are some OS level tweaks to avoid the deterioration of the mmap() time and resize() time, but I haven't looked into that yet.

@becketqin : Thanks for the results. The results in "log loading time distributing" shows that loading the log with timestamp index takes 15 times longer than without (399 secs vs 21 secs). Is that correct? If the overhead is in mmap(), with timestamp index should be at most twice as slow. Does this suggest there is sth in the timestamp index that is much more expensive than the offset index?

@junrao Sorry I did not explain it clearly. In the google sheet there are also two sheets showing that the time taken by mmap() is related to how many files has already been memory mapped. After we have mapped ~20000 files, the time taken for further mmap() calls increased dramatically.
The reason we see a 15 times larger loading time is the following. Currently I have about 22000 log segments in a broker. So that results in 22000 offset index files. It took 21 seconds to load. When we do mmap for those files, it is roughly at the tipping point where the mmap() becomes taking much longer.
Now we added the time index file so the number of mmap() call is doubled. Even though the first 22000 mmap() calls took the same time as before, which is 21 seconds, the next 22000 mmap() calls took much longer. It does not matter whether the mmap() calls are for offset index file or time index file. Because after adding the time index, presumably the first 22000 mmap() calls are half/half for each type of index files.
I am not sure if this is OS specific and only an issue for our environment or it is a general issue.

@becketqin : What happens if you do an experiment with 44K segments and turn off the time index? Does that also take 15 times longer? Were the mmap results based on a micro benchmark or did you extract that from the segment loading test?

@junrao I ran test with 8500 log segments with time index files. They are loaded in less than 3.5 seconds. And it took 1.5 second without time index files. Not sure if that is equivalent to testing files with 44K, but apparently the time does not grow linearly with the number of log segments.
The mmap results are extracted from the segment loading test. Micro benchmark which repeatedly maps a single file does not reveal this scalability issue.

@becketqin : Thanks for the new results. It feels a bit weird that the turning point is at exactly 22K segments. Also, in the "time to resize" sheet, the results seem to suggest that resizing at 10 batches with time index is 6 times more expensive than resizing at 10 batches w/o time index. This seems to be at the pointe where the number of loaded segments is still well below 22K?
Is it possible to do an experiment with 44K segments and turn off the time index? This will tell us more about whether the overhead comes from scalability related to mmap() or something else in the time index.

@junrao I refined the profiling code a little, I can stably reproduce the mmap() time issue. But the resize time seems to be consistent now. I'll do an experiment on a broker with ~50K segments without time index and see what happens there.

@becketqin : Any new updates on the experiment?

@becketqin : Are you still working on this? There are other KIPs such as KIP-58 that will be depending on this patch. So, we probably don't want this patch to sit for too long. We are pretty close to committing this patch.

@junrao Sorry, I just saw your ping... Yes, I am still working on this but was distracted by something else before. I will work on this this weekend and get it done.
I tried with 50K segments, and it took about 11 seconds to load all the segments without time index, and about 30 seconds with the time index.
I ran a bunch of tests on different machines. It seems that the machine I originally ran the tests on has some disk issue. A stably reproduceable result was that loading the segments took 3-4x longer with the time index. And what interesting is that for some of the logs with small number of log segments (e,g 100) I saw the loading time is significantly longer than other logs with much more segments (e.g. 3000). This happened both w/ and w/o time index. I am trying to see what caused this.
I'll update the findings this weekend.

@becketqin : Thanks for the update. Is the loading time affected by # of log segments, but not by the size of log segments? It seems that loading the time index is still more expensive than the offset index? Looking at the code, we could make a couple of simple improvements.

In the constructor of LogSegment, we call timeIndex.lastEntry twice unnecessarily.
/* The maximum timestamp we see so far */
@volatile private var maxTimestampSoFar = timeIndex.lastEntry.timestamp
@volatile private var offsetOfMaxTimestamp = timeIndex.lastEntry.offset

In TimeIndex.sanityCheck(): we also call lastEntry twice unnecessarily.

Not sure how much difference they will make. Does profiling reveal which part of loading time index is expensive? If it's in calculating maxTimestamp and offsetOfMaxTimestamp, it may be possible to do the calculation lazily. This probably will make the code more complicated. So, probably only worth doing if there is significant performance gain.

@junrao I have been running tests this afternoon.
Actually the previous results I got have included the changes you mentioned above. In addition, in the sanity check one difference for time index is that we also check that the last indexed timestamp is greater than the first indexed timestamp. The requires reading both the first and last entry. I also commented out that check.
The broker that used to have > 40000 segments has only 12000 segments now due to the retention. So I was not able to run the tests on the 40000 partitions. I started to populate data to the cluster again. Hopefully I will have >40000 segments by tomorrow.
One promising experiment I did this afternoon was changing the number of log recovery threads. This seems reduce the mmap time effectively. But since I do not have a 40000-segments broker this afternoon, it is hard to say if that is going to help in a large broker. I'll try this out tomorrow.
Back to the tests I ran earlier on a 40000 segments broker and this afternoon, the profiling I did was the following:

the time used to load each partition
the time used for mmap function specifically to see if there is long mmap (>100 ms)
the time used in sanity check.

The profiling shows time spent on (1) is not completely related to the # of segments in the partition. For example, a partition with 1 segment may takes 35 ms to load while another partition with 12 segments only takes 4 ms to load. Even for the two partitions of the same topic that have the exact same number of segments, the time can differ by 10x from 5 ms to 50 ms. This happens both with and without time index.
When I look at the partitions that takes longer to load (e.g. 2 seconds w/o time index and 6 seconds w/ time index), those segments tend to be loaded later in the segment loading process.
As for time spent on (2), I did not see any long mmap when the number of segments is less than 12000, whether with or without time index. But after the # of segments is greater than 20000, some long mmap() are witnessed. Again, those long mmap call only occur on the log segments that are loaded later in the segment loading process. The loading time was pretty linear when number of segments is 12000. It took about 2 seconds to load the segments without time index, and took about 4 seconds with time index.
Regarding the time spent on (3), when we have to read both the first timestamp and last timestamp in the time index, the sanity check time significantly grew, but still short compared with the time spent on loading the files.
Based on the test I have run, it seems clear that

The biggest part of segment loading time is mmap(), no matter w/ or w/o time index.
It does not seem that loading time index is more expensive than loading offset index.

@becketqin : Thanks for the updates.

What file system were you using in your testing?

Earlier you mentioned that loading 50K segments w/o time index took 11 seconds. How does that compare with say loading 25K segments with time index? If loading time index is no more expensive than loading the offset index, you would expect the latter to take no longer than the former.

@junrao The file system type is xfs. Currently I have ~19000 segments on the broker. Using 10 log recovery threads, it takes about 3 seconds to load without time index and about 10 seconds with time index. If I reduce the log recovery threads number to 5, the loading time becomes 6 seconds.
So it looks that the number of log recovery threads was causing the slow log segments loading. I will test this on a 25K segments cluster with time index as well after the broker reaches there.

@becketqin : Thanks for the info. So using more log recovery threads slows down the loading with time index?
Also, about mmap() becoming more expensive with more log segments. @enothereska mentioned to me that metadata lookups in file systems get more expensive with more open file handles since the lookups are based on an implementation of red-black tree. That's mostly CPU overhead. In your tests, do you notice any I/O activity when mmap() becomes more expensive?

@junrao I did not check I/O activities when mmap() becomes more expensive, it is kind of hard to check because the segments loading only takes a few seconds.
I tested log loading on 25K segments with time index using 5 loading threads. It takes about 13 seconds, which is comparable to the 11 seconds on a 50K segments cluster without time index. I will try to test it on a 40K cluster tomorrow.
If the test results look good, I will update the patch with a few bug fixes and rebase on trunk as well.

@becketqin : Thanks for the update. In your new patch, could you include a summary of the performance impact (and perhaps the ideal number of log recovery threads) in the upgrade doc?

@becketqin : I am trying to hold off https://github.com/apache/kafka/pull/1494 and https://github.com/apache/kafka/pull/1742 to avoid the rebasing overhead of this patch. Do you think you could provide your latest patch in the next day or two?

@junrao I have all the code ready now. I am still trying to find the reasonable number of recover threads. It seems there is system cache effect, i.e. if I do a start-shutdown-start sequence, the second start would be much faster than the first one because some of the disk read may come from the system cache. To avoid this effect, I need to wait for a while between each broker startup. I will upload the patch latest by Thursday. Is that OK?

@becketqin : Thursday is fine. Not sure if this helps, but you can empty dentries/inodes through a command (http://www.tecmint.com/clear-ram-memory-cache-buffer-and-swap-space-on-linux/).

@junrao Thanks for the dentries/inode cleaning command. I just saw them and haven't tried that yet. Currently I have a script reboot a broker every hour and it seems stable enough to give some results.
Some updates on the number of log recovery thread tests. For a cluster with about 40K segments, with one log recovery thread, it takes 14 - 17 seconds to load the logs, and about 20 seconds with 2 threads, > 30 seconds with 5 threads. So it seems that having a single thread loading segments is the best setting.
I have updated the patch after rebasing on trunk. I also squashed some previous commits.

@becketqin : Thanks for the update. Are your latest numbers with time index? What about the numbers w/o time index? Also, how many log dirs are there since the number of recovery thread is per dir?

@junrao The latest numbers are with time index. I see log loading time to be 21 - 23 seconds on a 46K segments with time index. It takes about 10 seconds to load the segments without time index. So the loading time with time index is about 2x of without time index. This is also comparable to the 11 seconds loading time without time index on a 50K segments broker.
I have ~8500 directories (partitions) on the broker. Some of the partitions are pretty big (> 2500 segments).

@becketqin : Thanks. Could you confirm the number of log dirs you have and the number of disks per dir? Intuitively, if there are multiple disks per dir, it seems using more than one 1 recovery thread should lead to better performance.

@junrao We only have one log directory on a RAID 10 containing 10 disk. So in our case, multiple thread might not help. In a JBOD environment, I agree that multiple threads would probably help.

@becketqin : Interesting, in theory, using more than 1 thread should still help since those threads can drive the I/Os on different disks in parallel.

@junrao I am not sure, one thing I noticed is that when there is one thread, the log loading time of each partition is pretty linear to the number of log segments. But if multiple threads are used, the linearity goes away.
For example, the following logs are from running the code using 10 log recovery threads without time index (Old_Mark indicates no time index, and the same random Old_Mark means they are from the same test run).
For TOPIC1, it only took ~400 ms to load about 2000 log segments, at about the same time (300 ms earlier) TOPIC2 takes 22 ms to load ~70 log segments. However, later on the same run (3 seconds later), for some other partitions of TOPIC2 that has about the same number of segments, it takes much longer (~400 ms) to load. This was the reason that I suspected that the later mmap() calls are more expensive, which seems also proven from profiling by measuring the time on mmap calls in different buckets.
I am not sure why this happens, but I didn't see such issue when there is one log loading thread.
2016/08/16 20:57:28.793 INFO [Log] [pool-9-thread-3] [kafka-server] [] (22 ms) Completed load of log **TOPIC2**-7 with 74 log segments and log end offset<PHONE_NUMBER> (Old_Mark<PHONE_NUMBER>390717976)
2016/08/16 20:57:28.794 INFO [Log] [pool-9-thread-1] [kafka-server] [] (22 ms) Completed load of log **TOPIC2**-4 with 84 log segments and log end offset<PHONE_NUMBER> (Old_Mark<PHONE_NUMBER>390717976)
2016/08/16 20:57:28.795 INFO [Log] [pool-9-thread-2] [kafka-server] [] (23 ms) Completed load of log **TOPIC2**-8 with 68 log segments and log end offset<PHONE_NUMBER> (Old_Mark<PHONE_NUMBER>390717976)
2016/08/16 20:57:28.795 INFO [Log] [pool-9-thread-4] [kafka-server] [] (24 ms) Completed load of log **TOPIC2**-2 with 79 log segments and log end offset<PHONE_NUMBER> (Old_Mark<PHONE_NUMBER>390717976)
......
2016/08/16 20:57:29.133 INFO [Log] [pool-9-thread-1] [kafka-server] [] (337 ms) Completed load of log **TOPIC1**-2 with 1973 log segments and log end offset<PHONE_NUMBER> (Old_Mark<PHONE_NUMBER>390717976)
2016/08/16 20:57:29.177 INFO [Log] [pool-9-thread-2] [kafka-server] [] (380 ms) Completed load of log **TOPIC1**-9 with 2020 log segments and log end offset<PHONE_NUMBER> (Old_Mark<PHONE_NUMBER>390717976)
2016/08/16 20:57:29.203 INFO [Log] [pool-9-thread-5] [kafka-server] [] (407 ms) Completed load of log **TOPIC1**-7 with 2076 log segments and log end offset<PHONE_NUMBER> (Old_Mark<PHONE_NUMBER>390717976)
2016/08/16 20:57:29.245 INFO [Log] [pool-9-thread-3] [kafka-server] [] (447 ms) Completed load of log **TOPIC1**-0 with 2078 log segments and log end offset<PHONE_NUMBER> (Old_Mark<PHONE_NUMBER>390717976)
...
2016/08/16 20:57:32.100 INFO [Log] [pool-9-thread-1] [kafka-server] [] (387 ms) Completed load of log **TOPIC2**-112 with 77 log segments and log end offset 371172232 (Old_Mark<PHONE_NUMBER>390717976)
2016/08/16 20:57:32.100 INFO [Log] [pool-9-thread-2] [kafka-server] [] (383 ms) Completed load of log **TOPIC2**-122 with 68 log segments and log end offset 373822294 (Old_Mark<PHONE_NUMBER>390717976)
2016/08/16 20:57:32.110 INFO [Log] [pool-9-thread-4] [kafka-server] [] (393 ms) Completed load of log **TOPIC2**-27 with 81 log segments and log end offset 375406677 (Old_Mark<PHONE_NUMBER>390717976)
2016/08/16 20:57:32.124 INFO [Log] [pool-9-thread-3] [kafka-server] [] (406 ms) Completed load of log **TOPIC2**-17 with 72 log segments and log end offset 371089327 (Old_Mark<PHONE_NUMBER>390717976)

@becketqin : Thanks for the latest patch. I made another pass and only had a few minor comments. Once those are addressed, I can merge in the patch.

@junrao Thanks a lot for all the patient reviews. I updated the patch to address you latest comments and added a few unit tests. Thanks again.

Thanks for the patch. LGTM

@becketqin : Thanks a lot for working on this diligently. A couple of followups.

I left a couple of minor comments after merging. Could you address them in a followup jira?
Do you plan to work on a followup KIP to add a new seekToTimestamp() api to the consumer and a new request for getting offsets based on timestamp?

@junrao Thanks a lot for the patient review. I will submit a follow up PR soon. And yes, I will start working on the KIP to add seekToTimestamp() API now and post the wiki this week.

@becketqin, it would be good to update the Kafka documentation to mention the time index. Maybe an easy way is to search it for offset index and see if it makes sense to mention the time index in those cases as well. Would you be willing to file a JIRA and take this on as well?


Unable to compile when [MemoryPackable] class has multiple constructors

I'm testing MemoryPack (latest .NET 7) with a class that has multiple constructors, and I marked one of the constructors as '[MemoryPackConstructor]'.
However, the project doesn't compile and shows an error during compilation:
"Error	CS0234	The type or namespace name 'Formatters' does not exist in the namespace 'Tests.MemoryPack' (are you missing an assembly reference?)"
And it points to line 44 in generated code:

Here is my full test code:
using MemoryPack;

namespace Tests.MemoryPack
{
    [MemoryPackable]
    public partial class TestObject
    {
        public string testString { get; set; } = "";

        [MemoryPackConstructor]
        public TestObject() { }

        public TestObject(string ignore) { }
    }

    class MemPackTest
    {
        public static void Main()
        {
            TestObject test_object = new();
            test_object.testString = "ABC";
            Console.WriteLine("Initial value before serializing: " + test_object.testString);

            byte[] serialized_object = MemoryPackSerializer.Serialize(test_object);
            TestObject? deserialized_object = MemoryPackSerializer.Deserialize<TestObject>(serialized_object);

            if (deserialized_object != null)
            {
                Console.WriteLine("Final value after deserializing: " + deserialized_object.testString);
            }

            Console.WriteLine("Test completed. Press any key to exit");
            Console.ReadKey();
        }

    }
}

Oops, this was a bug in my code, due to naming the Namespace "Tests.MemoryPack".
Closing.


Open in browser and mark as read is not working

I'm trying out Linux Mint, and have found an issue. When I press O, to open the article in the web browser and mark the article as read, it only opens the article. It does not mark is as read.
This works just fine in Gentoo Linux with the same config. With Gentoo I'm using URxvt, and with Linux Mint, I'm using xfce4-terminal. I have tried using URxvt in Mint as well, but that didn't work.
I have even added this to my config to make sure it's really bound to the correct thing:
unbind-key O
bind-key O open-in-browser-and-mark-read

What could be wrong?
Newsboat version (copy the output of newsboat -v or the first line of git show):
newsboat 2.21.0
System: Linux 5.15.0-94-generic (x86_64)
Compiler: g++ 10.2.1 20201207
ncurses: ncurses 6.3.20211021 (compiled with 6.2)
libcurl: libcurl/7.81.0 GnuTLS/3.7.3 zlib/1.2.11 brotli/1.0.9 zstd/1.4.8 libidn2/2.3.2 libpsl/0.21.0 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.43.0 librtmp/2.3 OpenLDAP/2.5.17 (compiled with 7.72.0)
SQLite: 3.37.2 (compiled with 3.34.0)
libxml2: compiled with 2.9.10
Steps to reproduce the issue:

Press O

Other info you think is relevant:

That makes perfect sense. I do have issues with that. My browser command is:
browser "firefox-bin %u >/dev/null 2>&1 & disown -a"

This does cause a Browser returned error code 127 message. Which I haven't been able to solve. It works just fine in Gentoo.


[Meta] Support project-wide dart format page width configuration

After the new tall style issue, the two formatter issues with the most 👍 are both about configuring page width (1, 2). This has been the #1 feature request for many years with hundreds of users asking for it.
So we are adding support for project-wide page width configuration to the formatter. After weighing a number of ways to author this configuration, the best approach seems to be a combination of two options. Each helps address the problems of the other:
Look for a surrounding analysis_options.yaml file and let it configure the page width
When formatting a file on disk using dart format, we walk up the surrounding directories looking for an analysis_options.yaml file. If we find one and it has a section like:
formatter:
  page_width: 123

Then the file is formatted at that width.
This lets users author the width in one location in a parent directory (likely the root directory of a package), and then all files in that directory tree get formatted the same way. Since analysis_options.yaml files also support an include key to import another analysis_options.yaml file, organizations have a way to have a standardized formatting style shared across all of their projects.
The main problem with using analysis_options.yaml (or any other config file) is that code generators and other tools that format code that only exists in memory don't know what config file to use. If the code generator doesn't use the configured width, but the user then runs dart format on the whole directory, the generated code will be changed. Likewise, if CI tries to validate that every file is formatted using the configured width, that will fail because the generated stuff is formatted at a different width.
Allow a comment inside of a source file to indicate the page width for that file
If you put // dart format width=123 at the top of a file, then the file is formatted at that page width. This is simple to implement, and works with dart format, code generators, and any other tool that wants to format code. It works even without having access to file IO or any knowledge of where the formatted code will end up on disk.
But it's very tedious for users to have to remember to put this in every single file they author. If they forget, they get inconsistently formatted code. No user has asked for this. They want project-wide configuration.
Hybrid approach
We implement both of these. If there is both a project-wide page width set in analysis_options.yaml and the file has a page width comment, the latter takes precedence (just like a // @dart=1.2 language version comment takes precedence over the SDK constraint in a pubspec).
Mixing both approaches lets us avoid the problems with each:

Tools like dart format and IDEs look for a configured project-wide page width in analysis_options.yaml. This is implemented in dart format now (behind an experiment flag). The analysis_server package, which most Dart IDE integrations use, already has support for working with analysis_options.yaml files, so plumbing the page width from that to the files it formats should be straightforward.
This gives users the project-wide configuration they want for the code they write and read.

Tools that generate Dart code and then immediately format in memory without having easy access to the surrounding analysis_options.yaml file can simply add a // dart format width=123 comment to the top of the generated code before they format it.
Since that takes precedence over any project-wide configuration, then formatting that file in memory with no analysis_options.yaml file will produce the same results as when the file is later formatted with dart format or the IDE. That should mean that things like CI steps that check for formatted files won't end up breaking on generated code.

This approach does mean that generated files may have a different width than the user prefers. But these files are fairly rarely read, so that's a tolerable compromise to avoid having every library that works with generated code and the formatter become obligated to support searching for and reading analysis_options.yaml (which isn't possible in some contexts like build_runner).
Implementation tasks
The core formatter work is done now (but behind a flag). The work remaining is largely in other packages:

[ ] https://github.com/dart-lang/linter/issues/5105 Generalize lines_longer_than_80_characters lint to use the configured width.
[ ] https://github.com/dart-lang/sdk/issues/56864 Have analysis_server plumb page width from analysis_options.yaml to formatter
[ ] https://github.com/dart-lang/source_gen/issues/724 Have source_gen include the page width comment in generated code

@dart-lang/analyzer-team, @dart-lang/language-team, @dart-lang/core-package-admins, any other important tools/workflows/packages that should handle this?

If you're going to use formatter as the top-level key (which makes sense), perhaps that name should be reserved on pub.dev. It seems analyzer plugins (or plugin like tools) often use top-level keys matching their package name for configuration.

@parlough 's comment reminds me about a similar discussion we had about namespacing "properties" in pubspec.yaml, too.

Good idea, @parlough. Filed an issue and added it to the list. Thanks!


android: Trust user-added CAs for all secure connections

Fixes #3312
Apps that target API Level 24 and above no longer trust user or
admin-added CAs for secure connections, by default.
This config allows the app to trust user-added certificate
authorities for all secure connections
More details here:

Changes to Trusted Certificate Authorities in Android Nougat
Network security configuration

Hmm. I don't love this solution of reverting to the old defaults. The problem of securing TLS and the CA system is complex and has a number of smart people working on it, and in general I'd much rather take the solutions shipped by major browsers and other expert security teams like those at Google, rather than second-guess them.
For the request in #3312, I think my ideal solution would be to make it a user opt-in: a switch in settings somewhere. Unfortunately there doesn't seem to be a good way to control this from code -- if you need to control it from code, it looks like you're stuck with the notorious TrustManager API, and I don't even see how to express in that API the equivalent of this configuration. (Surely it's there somehow? But it's going to be complicated to pin down.)
A solution that I think I'd be OK with would be to set this configuration except for domains that we know should be identified by certs from global trusted CAs. In particular: zulipchat.com and its subdomains, and chat.zulip.org. That should mitigate the risk substantially. In the blog post, under "Trusting user-added CAs for all domains except some" there's an example of how to do this.
What do you think?

(Also: thanks for sending this!)

Closing because stale. The underlying issue remains tracked as #3312.


Enh: Implement Math::Complex and "i"

plxcji019> Conv2portParams_hz_new.py QA_tttt_25_res.in
File "Conv2portParams_hz_new.py", line 736
s11[(len(s11)-1)+1] = _num(inLst[1])+_num(i()*)inLst[2]
^
SyntaxError: invalid syntax
plxcji019> Conv2portParams_hz_new.py QA_tttt_25_res.in
File "Conv2portParams_hz_new.py", line 736
s11[(len(s11)-1)+1] = _num(inLst[1])+_num(i()*)inLst[2]
^
SyntaxError: invalid syntax
ERROR STATISTICS:  W: 4
Q?:   Also: I did not use -s for import math sin() cos() Should I in this case ??
[Softpano-W325]:  Debug flag is set to 3
LINE 3 [main-W8238]:  Skipped translating /usr/intel/pkgs/perl/5.26.1/lib64/5.26.1/Math/Complex.pm to python for use in use Math::Complex (use '-s' option to translate it (not recommended))
LINE 4 [main-W8238]:  Skipped translating /usr/intel/pkgs/perl/5.26.1/lib64/site_perl/x86_64-linux/File/Spec.pm to python for use in use File::Spec (use '-s' option to translate it (not recommended))
LINE 5 [main-W8238]:  Skipped translating /usr/intel/pkgs/perl/5.26.1/lib64/site_perl/x86_64-linux/Cwd.pm to python for use in use Cwd (use '-s' option to translate it (not recommended))
COnv2portParams_hz_new.PlPY.txt

Joe Cool
Please suggest a manual fix in Python 3.8 in the meantime
From: Joe Cool @.>
Sent: Monday, March 21, 2022 7:14 AM
To: snoopyjc/pythonizer @.>
Cc: Ateshian, PeterX Roy @.>; Author @.>
Subject: Re: [snoopyjc/pythonizer] Version 0.964 Conversion inserts "*" or did not remove a ")" in perl conversion - Attached /tmp/Pythonizer/ (Issue #3)
Ok - this is using the "i" of Math::Complex, which I have not implemented. I will consider this as an enhancement request.
—
Reply to this email directly, view it on GitHubhttps://github.com/snoopyjc/pythonizer/issues/3#issuecomment-1073949632, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AX6LHCUXMPKBN3YABOGW2H3VBB7Z5ANCNFSM5RADCTZQ.
You are receiving this because you authored the thread.Message ID<EMAIL_ADDRESS>

          Manual Changes Proposed: 3-21-22

          #s11[(len(s11)-1)+1] = _num(inLst[1])+_num(i()*)inLst[2]
            s11[(len(s11)-1)+1] = complex ( _num(inLst[1]), _num(inLst[2]) )
            #s21[(len(s21)-1)+1] = _num(inLst[3])+_num(i()*)inLst[4]
            s21[(len(s21)-1)+1] = complex (__num(inLst[3]), _num(inLst[4])  )
            #s12[(len(s12)-1)+1] = _num(inLst[5])+_num(i()*)inLst[6]
            s12[(len(s12)-1)+1] = complex ( _num(inLst[5]), _num(inLst[6]) )
            #s22[(len(s22)-1)+1] = _num(inLst[7])+_num(i()*)inLst[8]
            s22[(len(s22)-1)+1] = complex (__num(inLst[7]), _num(inLst[8])  )
        else:
            #s11[(len(s11)-1)+1] = inLst[1]cos(pi*_num(inLst[2])/180.0)+_num(i()*)inLst[1]sin(pi*_num(inLst[2])/180.0)
            s11[(len(s11)-1)+1] = complex (inLst[1]cos(pi*_num(inLst[2])/180.0) , inLst[1]sin(pi*_num(inLst[2])/180.0)  )
            #s21[(len(s21)-1)+1] = inLst[3]cos(pi*_num(inLst[4])/180.0)+_num(i()*)inLst[3]sin(pi*_num(inLst[4])/180.0)
            s21[(len(s21)-1)+1] = complex (inLst[3]cos(pi*_num(inLst[4])/180.0) , inLst[3]sin(pi*_num(inLst[4])/180.0)  )
            #s12[(len(s12)-1)+1] = inLst[5]cos(pi*_num(inLst[6])/180.0)+_num(i()*)inLst[5]sin(pi*_num(inLst[6])/180.0)
            s12[(len(s12)-1)+1] = complex (inLst[5]cos(pi*_num(inLst[6])/180.0) , inLst[5]sin(pi*_num(inLst[6])/180.0)  )
            #s22[(len(s22)-1)+1] = inLst[7]cos(pi*_num(inLst[8])/180.0)+_num(i()*)inLst[7]sin(pi*_num(inLst[8])/180.0)
            s22[(len(s22)-1)+1] = complex (inLst[7]cos(pi*_num(inLst[8])/180.0) , inLst[7]sin(pi*_num(inLst[8])/180.0)  )

view Conv2portParams_hz_new.py
plxcji019> python3 -m pdb Conv2portParams_hz_new.py
Traceback (most recent call last):
File "/usr/intel/pkgs/python3/3.8.2/lib/python3.8/pdb.py", line 1703, in main
pdb._runscript(mainpyfile)
File "/usr/intel/pkgs/python3/3.8.2/lib/python3.8/pdb.py", line 1572, in _runscript
self.run(statement)
File "/usr/intel/pkgs/python3/3.8.2/lib/python3.8/bdb.py", line 580, in run
exec(cmd, globals, locals)
File "", line 1, in
File "/nfs/pdx/disks/x5e2d_circuit_simulation_gwa001/pateshix/conv_pl_py_1/Conv2portParams_hz_new.py", line 760
s11[(len(s11)-1)+1] = complex (inLst[1]cos(pi*_num(inLst[2])/180.0) , inLst[1]sin(pi*_num(inLst[2])/180.0)  )
^
Second manual Change
Amplitude or magnitude of complex impedance at frequency (x)               - Guessing hueristic - multiply ? (*) ??
                                                      s11[(len(s11)-1)+1] = complex (inLst[1]*cos(pi*_num(inLst[2])/180.0) , inLst[1]*sin(pi*_num(inLst[2])/180.0)  )

or may need Second Guess::
Mag(inLst[1]),inLst[2] ) = sqrt ( inLst[1]) )^2 + (inLst[2])^2   )
From: Joe Cool @.>
Sent: Monday, March 21, 2022 7:14 AM
To: snoopyjc/pythonizer @.>
Cc: Ateshian, PeterX Roy @.>; Author @.>
Subject: Re: [snoopyjc/pythonizer] Version 0.964 Conversion inserts "*" or did not remove a ")" in perl conversion - Attached /tmp/Pythonizer/ (Issue #3)
Ok - this is using the "i" of Math::Complex, which I have not implemented. I will consider this as an enhancement request.
—
Reply to this email directly, view it on GitHubhttps://github.com/snoopyjc/pythonizer/issues/3#issuecomment-1073949632, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AX6LHCUXMPKBN3YABOGW2H3VBB7Z5ANCNFSM5RADCTZQ.
You are receiving this because you authored the thread.Message ID<EMAIL_ADDRESS>

You probably need “import math” and “import cmath” and prefix the functions (sin/cos etc) with the appropriate library.  If the value is in the complex constructor, use math, and if it’s already a complex value, then use cmath.

Oh and ^ is not power, you mean **

Yes ** not ^ correct -
From: Joe Cool @.>
Sent: Monday, March 21, 2022 3:24 PM
To: snoopyjc/pythonizer @.>
Cc: Ateshian, PeterX Roy @.>; Author @.>
Subject: Re: [snoopyjc/pythonizer] Enh: Implement Math::Complex and "i" (Issue #3)
Oh and ^ is not power, you mean **
—
Reply to this email directly, view it on GitHubhttps://github.com/snoopyjc/pythonizer/issues/3#issuecomment-1074481012, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AX6LHCRJ3MCPNLYRJFPMSUDVBDZJFANCNFSM5RADCTZQ.
You are receiving this because you authored the thread.Message ID<EMAIL_ADDRESS>

Got it – Thanks!
I reconstructed the correct acos(x) + ja*sin(x)  from the perl5 script in python3. So I have a manual workaround with
import math, cmath
and then
from math import sin, cos, sqrt
From: Joe Cool @.>
Sent: Monday, March 21, 2022 3:24 PM
To: snoopyjc/pythonizer @.>
Cc: Ateshian, PeterX Roy @.>; Author @.>
Subject: Re: [snoopyjc/pythonizer] Enh: Implement Math::Complex and "i" (Issue #3)
You probably need “import math” and “import cmath” and prefix the functions (sin/cos etc) with the appropriate library. If the value is in the complex constructor, use math, and if it’s already a complex value, then use cmath.
—
Reply to this email directly, view it on GitHubhttps://github.com/snoopyjc/pythonizer/issues/3#issuecomment-1074480499, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AX6LHCUTWTWE3QTRSRIZQ6TVBDZGFANCNFSM5RADCTZQ.
You are receiving this because you authored the thread.Message ID<EMAIL_ADDRESS>

Need to first add these core functions: exp, sin, cos, log

Need to also convert this idiom to INF: 999

Fixed in v0.966 - translated the Math::Complex module automatically into Python (with lots of bugs in Pythonizer fixed along the way and wantarray implemented)


Assignment/annotation workflow and correlation table

This is material for the assignment/processing-related part. There are some general ideas attached with the file "Draft_workflow_proc_assign" and two movies links which give access to view/download movies on the structure validation tool of Mnova (by Johannes): https://video.uni-mainz.de/Panopto/Pages/Viewer.aspx?id=7a9cf6da-5b46-4ab1-9ee2-ac360087f07c
and of TopSpin/CMCse (by Nils):  https://uni-koeln.sciebo.de/s/xso0hXlQdbjS5J8

Draft_workflow_proc_assign_vs041020.pdf

It would be important, if the picked zones of the 2D would end up in a correlation table where:
(1) zones defined in HSQC or HMQC end up as (known) 1-bond correlations and
(2) zones defined in HMBC end up as (unknown) n-bond correlations (to be defined later)
(3) regions from 1D would be joined as "projection" along x or y

OK, I will look. Can you you define more clearly the question you want me to answer. What you expect from me?

BTW, I did not notice this assignement earlyer (it got lost into all the activity of the group - I just restricted and it became visible).

@nes-cgn I did a pull request for a first version of a correlation table. It has to be accepted first and then it will be available for the public.
But here is a description of what the table or panel is able to do at the moment:
https://github.com/cheminfo/nmr-displayer/pull/652

@michaelwenk Thank you. I played already a bit with the possibility to define zones and ranges. The correlation table contains now a lot of useful information. But for instance for teaching, I would prefer a format, where you can intuitively recognise the relation between frequencies in a kind of abstract spectrum like the two below (see also pp.5 ff of the work flow sketch):


Added missing ",".

Not sure if this helps at all.

Thanks, Trygu. I did the same thing directly in the already submitted pull request.


After accepting gift. On clicking "View Account", the app is crashing.

Not reproducing

Verified this issue on staging app v2.0.5(377)


[Driver] Cloudflare D1

https://developers.cloudflare.com/d1/

Thanks for the suggestion. unjs/db0 already supports D1 connector: https://db0.unjs.io/connectors/cloudflare
With Generic SQL driver, we will support D1 for unstorage: https://github.com/unjs/unstorage/issues/400


HHH-16059 Add getters and relax scopes

So that Hibernate Reactive can avoid copy and paste when extending CteInsertStrategy and CteMutationStrategy
Fix https://hibernate.atlassian.net/browse/HHH-16059

LGTM


crypto/x509: export systemRootsPool or equivilant

I frequently have the need to add a custom CA cert to a go app (both my own apps, along with third party apps I run across). Currently you can create your own *x509.CertPool and add the CA cert to it, but now you can't verify using the system provided CAs.
Since the code behind systemRootsPool has a lot of os/distro-specific logic, it would be good to use this logic instead of having to copy/implement it within the app.
If there is a concern about exposing this pool, and code being able to mutate it, then perhaps we could get something similar to x509.initSystemRoots, but which would return a new copy every time it's called.
Edit: After some thought, returning a new CertPool is better. As there is no way to copy a CertPool, it would otherwise be impossible to add a CA cert to the pool without the cert shared everywhere.

I've been in the same position and wanted the system root store.
/cc @agl for any thoughts

@bradfitz Is there a polite way to :+1: these issues?
Basically, we'd like to partition the cert pools while at the same time inheriting the system roots.

@bradfitz @ianlancetaylor - I'm a little confused with the tags that were applied to the issue. On one hand it seems like we have no plans of solving the issue, on the other it seems like we want to solve it by Go1.7. Could you please clarify that?
Thanks,
@luan

We don't have meetings to coordinate these tags.
I set the milestone to "unplanned", meaning that it was probably a good idea but nobody was actually working to implement it.  @bradfitz set the milestone to "go1.7", meaning (I assume) that he thinks we should implement this for the 1.7 release.  But somebody still has to do the work.
If you want to work on this, you should do so regardless of the milestone.

(I often change things from Unplanned to Go 1.7 if I want it myself and am willing to do the work if nobody else gets to it first)

Sent https://go-review.googlesource.com/21293


Add missing withParent HOC

Fixes https://github.com/sanity-io/sanity-plugin-hotspot-array/issues/8 by adding missing withParent HOC introduced to form-builder in v2.34.0.
Without this change, all Studios upgrading to or past v2.34.0 will break, if using the "parent" option of imageHotspotPathRoot.
This project was missing a lock file, and therefore I added it (I think it is missing by accident, see: https://github.com/rexxars/sanipack/commit/c9e4b1bbd945586a07286cd0cbb2df5cfdc40ade). This change was made to force package manager to resolve required >= v2.34.0 of form-builder. In addition, I upgraded all Sanity deps to latest.
The current way of using withParent HOC is not very performative, and both hooks, withParent & withDocument, are required to be included to not introduce breaking change. However, I didn't find any better solution to introduce back the reference to parent.
I have added documentation around the previously undocumented imageHotspotPathRoot field.
I have tested this in my environment, and everything works as expected after upgrading to latest Sanity version. I don't, however, have experience in Sanity tests & publishing, so please guide me if needed.

Thanks for this work and the thorough write-up!


Wrong transform of template string with escaped newlines

Describe the bug
Escaped newlines in template string literal is badly transformed.
Input code
`\
`

Config
{"env":{"targets":{"node": "13"}}}

Expected behavior
`\
`;

Actual behavior
`\\n`;

Version
The version of @swc/core: 1.1.19
Additional context
I'd like to fix it myself but I don't know where to start.

I just manually executed swc_ecma_parser and looks like the parsing itself is OK.
Tokens: [
  TokenAndSpan {
    token: BackQuote,
    had_line_break: true,
    span: Span {
      lo: BytePos(0), hi: BytePos(1), ctxt: #0
    }
  },
  TokenAndSpan {
    token: Template {
      raw: Atom('\\n' type=inline),
      cooked: Atom('' type=static),
      has_escape: true
    },
    had_line_break: false,
    span: Span {
      lo: BytePos(1),
      hi: BytePos(3),
      ctxt: #0
    }
  },
  TokenAndSpan {
    token: BackQuote,
    had_line_break: false,
    span: Span {
      lo: BytePos(3),
      hi: BytePos(4),
      ctxt: #0
    }
  }
]

https://github.com/swc-project/swc/blob/add63826bdc6e4417a3a0fa522806b7177edc92f/ecmascript/codegen/src/lib.rs#L343-L386
Escape codes of plane string literals ('yo' or "yo") are partially handled by multiple replace() call. It partial since escaped hexadecimal values (e.g. \x1b) are not being handled.
But escape codes of template string literals are not handled at all as you can see below:
https://github.com/swc-project/swc/blob/add63826bdc6e4417a3a0fa522806b7177edc92f/ecmascript/codegen/src/lib.rs#L988-L991
There're two options.

Let's perfectly handle and print escape codes, looking at https://www.ecma-international.org/ecma-262/10.0/index.html#sec-literals-string-literals. It'll be tedious for obvious reasons

Let's just print the raw input as it is, without any changes. Super easy and can't be failed.

Me and @kdy1 have agreed to try the option 2.

I took a look at TplElement and I failed to find out how to print the raw input as-is.

I noticed that TplElement.raw stores
`\n`

as
"\\n"

And printing with TplElement.raw should require lots of replacing anyway.
Maybe fixing
https://github.com/swc-project/swc/blob/4b7222cbfeec94b4826558928cb5d72edaad188e/ecmascript/parser/src/lexer/mod.rs#L57-L80
might work

I managed to make a test cases for this issue but need more time to fix it.
https://github.com/simnalamburt/swc/commit/7e15a09e236c8da44fa3047f488051f6f6f57944


Batch Script syntax highlighting issues

Issue Type: Bug
There are a handful of highlighting issues with Windows Batch files:

[ ] ECHO: is not recognised, when ECHO. is; ECHO: is faster as it does not do a file lookup! -- see: https://ss64.com/nt/echo.html

[ ] The OFF in ECHO OFF is not highlighted

[ ] The following code gets highlighting confused (the "$... seems to throw it off)

[ ] No space between GOTO and the label name does not highlight correctly; i.e. GOTO:EOF

[ ] CLS keyword is not highlighted

[ ] EQU / NOT and other comparison operators are not highlighted

[ ] IN & DO are not highlighted

[ ] command concatenator &, and the pipe operators >, | are not highlighted nor pipe numbers for redirection, e.g. >NUL 2>&1 (a common way to suppress all output)

[ ] error-suppressor @ is not highlighted in any way; this needs to be highlighted when used with a command, but not within 'strings' (which may or may not be quoted, according to command used)

[ ] ENABLEEXTENSIONS / DISABLEDELAYEDEXPANSION etc. not highlighted along with SETLOCAL

[ ] Syntax break-down here: (looks like the quoted paren-escape failed to be noticed)

Aside: What would be the best way to broach a discussion on using a PEG-based parser, rather than the regex-infused TextMate system for VSCode? Windows Batch files happen to be one of the best examples of extremely complex mode-like behaviour that would be easily and elegantly handled by a real state machine based approach using PEG/EBNF grammars. Grammar based parsing would also be readily shareable between language servers and other external software.

VS Code version: Code 1.22.1 (950b8b0d37a9b7061b6f0d291837ccc4015f5ecd, 2018-04-06T02:26:57.615Z)
OS version: Windows_NT x64 10.0.16299
Extensions: none

This issue was moved to mmims/language-batchfile#26

Fixes for these issues (https://github.com/mmims/language-batchfile/commit/6235c491be4dff49cd3966b50142874d7f79580a) are ready to be merged.


Updated Win8.1 solution. A few minor source changes needed to appease…

… the 8.1 compiler.

Good catch. Looks good.

@turkycat, Thanks for signing the contribution license agreement so quickly! Actual humans will now validate the agreement and then evaluate the PR.
Thanks, MSBOT;


404 Not Found on isos

hi
I use your library and don't work isos.
I recieve 404 not found when call isos part.

i can confirm this bugs, the ISOs class return 404 .

a simple fix is to change the ISOs.php name to Isos.php and change the classname from ISOs to Isos.

a simple fix is to change the ISOs.php name to Isos.php and change the classname from ISOs to Isos.


No email notification on failing build

Describe the bug
The pre-merge build of my Revisions failed with Phab showing the error. However I did not get an email notification. So I actively have to check the UI to see the failing builds.
Relevant links:

https://reviews.llvm.org/D105014 (some of the patches had problems with applying them)
https://reviews.llvm.org/D105177 (Windows build is failing)

To Reproduce
Steps to reproduce the behavior:

Submit a patch with failing builds
Wait for an email notification from Phabricator

Expected behavior
On every failing build of the pre-merge tests I would like to receive an email telling me that it failed and where I can find the build results.
I could imagine that some folks do not want to be bothered with notifications around failing builds, however I would like to know ASAP so that I can fix these.
Screenshots
If applicable, add screenshots to help explain your problem.

I agree that it should be useful in general to get build results if build failed (but not e.g. cancelled as a new revision was uploaded).
We can implement it by opt-in or opt-out basis. For the beginning I suggest to use existing group https://reviews.llvm.org/project/view/78/ membership as a indication that one want's to particiate.
Bot will add a comment to a revision like we did some time ago (but disabled as some people were complaining).
What do you think?

Sounds like a reasonable plan. I just surprised that there is no out of the box way for build failure notification in Phabricator.
@joker-eph are you aware of any notification mechanism we could use?

I haven't studied this in Phab.


Offsets probably correct but reporting FNames is bad

So I'm trying to dump Palworld, here is my configuration:
offsets.push_back({ OFFSET_ADDRESS | OFFSET_DS, "OFFSET_GNAMES", 0x8719900 });
offsets.push_back({ OFFSET_ADDRESS | OFFSET_DS, "OFFSET_GOBJECTS", 0x087B8FF0 });
offsets.push_back({ OFFSET_ADDRESS | OFFSET_DS | OFFSET_LIVE_EDITOR, "OFFSET_UWORLD", 0x8936D28 });

With UE version UE_5_01
I get a cachin FNames error because the first object name is None
I also get a bunch of "Could not resolve address for object" issues.
I'm at least 100% certain that GObjects and UWorld is correct. I think GNames is also correct but this I'm less sure of. What would you recommend?

I've just checked Perfect Heist 2, now I'm 99.9% sure these are correct because I have the PDB for it. I'm still getting an FNames cache issue. I've changed engine version to 4_25, but also 4_26 as is what's commented in the file.

Update:
Apologies I (think) that I have FNames working, however now all the objects are marked invalid. The only thing that shows is 1 Package called "BasicType". This is for both games.

you could try debugging the root of the issue. if you are unsure about the offsets, theres not much i can do as i will only check the dumper for issues if its confirmed a dumper issue and not a offset issue

you could try debugging the root of the issue. if you are unsure about the offsets, theres not much i can do as i will only check the dumper for issues if its confirmed a dumper issue and not a offset issue

Thanks for replying, I figured it out but may be worth documenting.
I assumed you wanted a pointer to the FUObjectArray structure but you want a pointer to ObjObjects, which is offset by 0x10, maybe just me misreading something


Mission.h can not be found in Ubuntu 14.04

cmake --build .
returns error:
fatal error: Mission.h: no such file or directory
#include <Mission.h>
inclued from MissionSpec.h
is that my problem the same as what is disscussed in MacOS?
I have tried to forced install xsd as:
wget http://www.codesynthesis.com/download/xsd/4.0/linux-gnu/x86_64/xsd_4.0.0-1_amd64.deb
sudo dpkg -i --force-all xsd_4.0.0-1_amd64.deb
sudo apt-get install -f
but the problem still occurs.
should I add command as what has been don in MacOS as below:
wget https://raw.githubusercontent.com/bitfehler/xs3p/master/xs3p.xsl -P $TRAVIS_BUILD_DIR/Schemas
+export MALMO_XSD_PATH=$TRAVIS_BUILD_DIR/Schemas
cd $TRAVIS_BUILD_DIR
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
make
-ctest -VV
+ctest -E Integration -VV
My system is Ubuntu 14.04

Hi @WXFMAV, just to check, have you been following the linux build instructions here: https://github.com/Microsoft/malmo/blob/master/doc/build_linux.md ?
If you look in your build/Malmo/src folder, do you see Misison.h etc in there?
And what do you see if you type /usr/bin/xsd version into a terminal?

it was because my MALMO_XSD_PATH is illegal, now it works well. thanks.

Glad it's working now, thanks for letting us know.


29: Ingest Application log collection

Blocked on the merge of:

[x] https://github.com/NationalSecurityAgency/datawave-ingest-services/pull/30

Services started up and behaved as expected on test system

I will create follow on PR's to apply the same sidecars to the feeder and bundler

Closing in favor of doing all the services in one PR


Clean up CONTRIBUTING.md

Running through CONTRIBUTING.md last night to build platyPS, I made a few edits to clean up the readability and grammar of the document.

This change is

Great!


[BUG]  can't capture to an existing file

Describe the bug
A clear and concise description of what the bug is.
create a new item, set path to be an existing file (valut/Inbox/scratch.md), no template, create if doesn't exist, write to bottom of file, capture format is:
\n{{VALUE:test}}\n
when i run this i get prompted to enter an error message to create a capture path but the file does exist.
Expected behavior
prompt for a value and add to end of the file
Desktop (please complete the following information):
windows 11
version 1.0.3
Additional context
have another quick add to add tasks to bottom of a file that is my daily note and that file exists and the file format id using {{DATE}} values to specify the name

restarted obsidian and now works.  weird


Ensure piaware script matches debian version

Previous Debian buster was used, but the script for building piaware was for Stretch

You just need to remove "v" in front of rtl-sdr version, they removed the tag with it.


Adapt Req.Response.json/3 to not override content type

Follow up on https://github.com/wojtekmach/req/pull/145#issuecomment-1279993636
PS: I've sent you a DM on twitter  👍

Thank you!


Project Updates & Cleanup

Removed /.idea directory from git tracking. In my experience tracking this directory becomes a nightmare since many developers customize their workspace in ways that modify files within this directory. If specific files need to be tracked in this directory it should be done with explicit gitignore rules.
Updated Gradle build tools to the latest supported and recommended version.
Updated Android Support Libraries to the latest supported and recommended version.
Resolves Issue #2

Thank you for your contributions and for helping us clean this up!  I've reviewed and they look clean and concise.


iOS8 or later not supported?

When i compile the demo with Xcode7.3,problem is "framework not found IOSurface",i want to know the  whether the repo support the ios8 or later?

You should include this framework in your app. Just copy it from anywhere :)

i just copy the IOSurface framework from the "/System/Library/Frameworks",the problem is "framework not found CoreSurface",i ask the question in StackOverflow,but no answer can solve the problem.Should i include the CoreSurface in my app?

iOS9 does not support this,http://stackoverflow.com/questions/32239969/iomobileframebuffergetlayerdefaultsurface-not-working-on-ios-9

Heh, you can try to use official Appke ReplayKit, it's 60FPS video, also excluding user notifications.


Create Facebook Page

A Facebook page should serve to notify the RPI community of the new app and keep them up to date with changes, enhancements, and new features

Created https://www.facebook.com/yacsrpi/
first post:
Hello RPI!
Some of you may have been wondering about the status of YACS (or maybe you haven't). Either way, we've been at work this academic year rebuilding YACS from the ground up, and we are happy to say that https://yacs.cs.rpi.edu is ready for the Fall 2016 semester! Please note the new address: the registrar sent out incorrect information and the older yacs.me will not be updated.
We've made efforts to improve the speed, usability, and reliability of YACS to provide you with the best experience even in those registration crunch times!
Please remember that the site is in beta! There are some features we are still working to implement. If you have any issues with your experience, or have any suggestions or feature requests, please do not hesitate to contact us at<EMAIL_ADDRESS>or create an issue on our Github repo. We will work to resolve any problems as quickly as possible.
YACS is an open source project created in RCOS, the Rensselaer Center for Open Source. If you'd like to get involved in RCOS, we hold general meetings with cool talks every Friday at 4PM in DCC 318. This project is sponsored by Red Hat, who we thank for their support! If you'd like to contribute to YACS or just check out the source code, please visit our Github at https://github.com/aosdict/yacs.
YACS is now hosted at RPI with the address https://yacs.cs.rpi.edu/ If you are off campus and experience problems connecting, please be sure to specify https in the address.
Happy Scheduling!


Doesn't recognize characters written with Alt Gr

It currently doesn't accept any characters written with Alt Gr (Ctrl+Alt) which is needed for many non US keyboard layouts.
For example on the German keyboard layout characters like {, }, [, ], , | and ~ all need the Alt Gr key to work. So it's currently pretty much impossible to play this game.

Hello, same problem after 5 years..


is there support for node versions lower than 18?

Hi, was wondering if the sdk would work for node versions lower than 18. I have a project that uses 16.17
Thanks!

The 18.x compatibility requirement is primarily because this module uses native fetch.
It will work for you with the —experimental-fetch flag, as fetch was introduced in v16.15.0 — see https://nodejs.org/docs/latest-v16.x/api/globals.html#fetch

This is also documented in https://github.com/paambaati/neon-js-sdk/commit/4532ec9d5b9da1f0e864a5327f31d236800a97f1.


(cached) server error on<EMAIL_ADDRESS>

Used to work. Broke my observablehq notebook. Tmp build error that is now cached?
<EMAIL_ADDRESS>(used to work)

<EMAIL_ADDRESS>(works)
I'm located in Zürich (maybe only on region cache is broken)

I invalidated it yesterday, sorry I was on a train, could you please try again?

works, thank you


Cannot Use .native() queries inside Services.

Sails version: 0.12.11
Node version: v6.10.0
NPM version: 3.10.10
Operating system: macOS 10.12.3

I am using 'MongoDB' Database and 'sails-mongo' adapter for my project. I was doing hotfixes where I was forced to write the DB aggregation queries inside Services. I am using .native() query functionality provided by Sails.js. But am not able to use .native() queries inside Services. Is sails.js doesn't allow .native() queries inside Services...??

Hi @naveenvprakash, there should be no problem using any of the supported model methods inside of services.  If you can post the code you're using to call .native(), we can try and see what the issue is.

Here is my code trying to do aggregation using .native() inside one of my Services called DataService.js.
function getData(reqData) {
    var lrModel = sails.models.lr;
    lrModel.native(function(err, lrCollection) {
        lrCollection.aggregate([
            { $match: {createdAt:{$gte:ISODate("2017-02-21T00:00:00.000Z")}}}
        ], function(err, result) {
            if(err) {
                return err
            }else if(!result) {
                return null;
            }else{
                return result;
            }
        })
    })
}

PATH = my project dir;
Here is the ERROR:
TypeError: Cannot read property 'native' of undefined
at getData (/PATH/api/services/DataService.js:5:12)
at Object. (/PATH/api/services/DataService.js:28:12)
at Module._compile (module.js:570:32)
at Object.Module._extensions..js (module.js:579:10)
at Module.load (module.js:487:32)
at tryModuleLoad (module.js:446:12)
at Function.Module._load (module.js:438:3)
at Module.require (module.js:497:17)
at require (internal/module.js:20:19)
at /PATH/node_modules/sails/node_modules/include-all/lib/help-include-all-sync.js:271:33
at Array.forEach (native)
at _recursivelyIncludeAll (/PATH/node_modules/sails/node_modules/include-all/lib/help-include-all-sync.js:174:11)
at includeAll (/PATH/node_modules/sails/node_modules/include-all/lib/help-include-all-sync.js:292:5)
at helpBuildDictionary (/PATH/node_modules/sails/node_modules/include-all/lib/help-build-dictionary.js:43:15)
at Function.module.exports.optional (/PATH/node_modules/sails/node_modules/include-all/index.js:67:10)
at Hook.loadServices (/PATH/node_modules/sails/lib/hooks/moduleloader/index.js:424:18)
at Hook.wrapper [as loadServices] (/PATH/node_modules/@sailshq/lodash/lib/index.js:3250:19)
at Hook.loadModules (/PATH/node_modules/sails/lib/hooks/services/index.js:68:21)
at Hook.wrapper [as loadModules] (/PATH/node_modules/@sailshq/lodash/lib/index.js:3250:19)
at Array.modules (/PATH/node_modules/sails/lib/hooks/index.js:79:27)
at /PATH/node_modules/sails/node_modules/async/lib/async.js:591:38
at _arrayEach (/PATH/node_modules/sails/node_modules/async/lib/async.js:85:13)
NOTE: The same code inside models/lr.js  its working fine; But inside Services its not;

Thanks @naveenvprakash.  It looks like something else in your DataService file (on line 28) is trying to run this function when the service is first loaded.  The error your getting isn't actually related to the .native() method; if you look closely, it's saying that sails.models.lr is undefined, likely because the models haven't loaded yet.  You'll need to defer execution of this code until after Sails has finished initializing.  A good place to put startup code like this is in your config/bootstrap.js file.
Also keep in mind that since the code in getData is asynchronous, you'll need to wait for it to finish before using the result!  The return result; at the end of that function makes me thing you're doing something like var data = getData(someVar); and expecting data to get the value of the result, but instead it'll get undefined.  Any function that calls getData will have to be asynchronous as well.
If your service needs some initialization before it can be used, you might consider turning it into a hook instead, which has an asynchronous initialize() method.
I'm going to close this now as it doesn't look like a bug in the Sails core, but if you post back with your full DataService.js file I can make some suggestions as to how to get it to work.

Thanks @sgress454. I figured it out.


ResumeReader Improvements

Modify the tikaFunc method to use the PortableDataStream (a._2) vs the file path. This will enable support of non-local file systems (HDFS, S3). The PortableDataStream should be able to be directly passed to the parse method as opposed to creating file streams.

Add object and method level comments

Use the scopt command line parser as opposed to args. This will allow a more robust interface. See WordMap.scala for an example.

Part 1 complete, Spark now passes only the PortableDataStream (the value of the key-value RDD) to the TikaFunc.  Spark still stores the files in the RDD as a key-value pair in the main method

Added comments

Added comments

Part 2: Added comments to method/object

Part 3: Will added command line arguments using scopt

Note: Method "tikaFunc" was renamed to "extractText"


Documentation update

Anchor links (table of contents) on this doc page (probably others) are prefixed with a question mark. This is incorrect and causes the links to be broken. This PR updates the anchor links to remove unnecessary question mark prefix.
See https://github.com/clearsightstudio/ProMotion/blob/master/docs/Reference/API Reference - ProMotion TableScreen.md for an example.

simple enough!

Yeah, this was from when we were using the Github wiki which broke linking pretty badly.


BDC Controller Manage action should open dashboard to BDC tab

Currently opening a dashboard just opens it to the default tab for the SQL instance - but when opening from a BDC controller it should open up to the BDC tab instead.

Not needed anymore


Bump airflow to 1.10.4, and the deps to latest. Some readme updates.

Can I get a sanity check on this one? :)


deployment

[x] split into spot (ie client), framework, and server packages
[ ] add shrinkwrap to freeze dependencies
[x] add express as webserver, enable https, and have the socket share the port

spot is split in spot (the app / website), spot-server, and spot-framework
the server and framework could be published on npm; for the app that doesn't make much sense.
Deployment can be something like:

npm install spot-server
git checkout spot

https configuration issue can be moved to spot server;
shrinkwrapping versions only necessary for spot-framework and spot-server, for the app we can use the prebuild site from the dist directory


Interesting proyect

I was wondering, what are all the commands I can execute?
Thanks

Sorry for the late answer.
Methods are available in the src/nfc.py file


[Storage] Write migration guide for azure-storage-file-share

As title states 😄

API change check
API changes are not detected in this pull request.


refactor: convert options into polymorphic Caddy modules

This changes the structure of the JSON config from, e.g.:
{
	"apps":{
		"http": {
			"servers": {
				"srv0": {
					"listen": [":443"],
					"listener_wrappers": [
						{
							"wrapper": "ngrok",
							"authtoken": "",
							"tunnel": {
								"type": "http",
								"metadata": "some-metadata"
							}
						}
					]
				}
			}
		}
	}
}

To:
{
	"apps":{
		"http": {
			"servers": {
				"srv0": {
					"listen": [":443"],
					"listener_wrappers": [
						{
							"wrapper": "ngrok",
							"authtoken": "",
							"tunnel": {
								"type": "http",
								"options": {
									"metadata": "some-metadata"
								}
							}
						}
					]
				}
			}
		}
	}
}

It feels more ergonomic, allows for future complex options to benefit form Caddy's module lifecycle, and allows the user to examine the available options when running caddy list-modules.
@jtszalay, thoughts? I had this idea when I saw the complexity in OIDC option in PR #17.

Slept on it and decided against it for being over-engineered