Server-side KaTeX randering of LaTeX math mode into MathML

Hi there, I have been trying for quite some time to convert my existing blog, as well as my business SSG to Hugo. Personally, I don't really care how fast it is, my blog will never be big.
However, I now have some employees that work with me, and trying to switch to Hugo has lead us to two issues that I'm going to post back-to-back, and offer a $250 CAD (~$200 USD) reward for each issue which can be PRd and merged. Not sure if you have a bounty program here, but I'd certainly be happy to contribute in this way. Personally, I don't think I'm quite savvy enough to do these features myself (even if I am a developer by trade).
This first issue is the ability to produce MathML/KaTeX server-side so the final document does not depend on Javascript.
I create accessible documents for the blind, and it is extremely important to me that the documents don't get outdated because of the Javascript. There will always be tools to convert from HTML4 to HTML5, and 5 to 6, or 6 to whatever new standard comes along. The MathML source can also be converted into newer formats as things progress.
However, embedding the LaTeX and hoping that a future JS script is going to just hopefully be able to handle it is not something I want to rely on.
Additionally, including the same Javascript file with every finished HTML document is wasteful.
I want my files to be as self-contained as possible.
Server-side MathML generation (through Jekyll+KaTeX, currently) fixes the above issue.

So if somebody can find a way, without including cgo, which I think I remember seeing as a deal-breaker for a previous implementation, that everyone can agree on and get into some version of Hugo down the line, I would be insanely grateful, and willing to put my money where my mouth is.
Please tell me if this is the wrong place for this, if it's inappropriate to give out bounties this way/at all, or if I'm doing anything that would be considered  problematic.
My goal is to make the world a more accessible place, and this is just one of the ways I'm hoping to help, even if all I can offer is a reward.
Thanks for you time,

I made a demo site to solve this with zero js uing a shortcode.
Demo site: https://hugo-tests-2-git-katex-static-muchina.vercel.app/posts/post/
Site Source: https://github.com/ianmuchina/katex-api-test
API Source: https://github.com/ianmuchina/katex-api-test
Currently the shortcode requires internet access. You can modify it to use a localhost url if you need offline access.
Here's how to use the shortcode in markdown.
{{<katex>}}
\sqrt{x^2+1}
{{</katex>}}

Here's the shortcode source.
<!-- layouts/shortcodes/katex.html -->
{{ $url := printf "https://katex-api-test.deno.dev/?expression=%s&rawHTML=true" (urlquery .Inner) }}
{{ with resources.GetRemote $url }}
  {{ .Content | safeHTML }}
{{ end }}

For it to work you need to import the katex css file.
/* styles.css */
@import url("https://cdn.jsdelivr.net/npm/katex@0.16.0/dist/katex.min.css");

The api responses can be put into version control to speed up subsequent builds with this config.
# config.yml
caches:
  # Cache resources infinitely
  getresource:
    dir: :resourceDir/
    maxAge: -1

Note: this is only a demo. I recommmend creating your own api that you control.

There are some existing efforts on this: https://github.com/graemephi/goldmark-qjs-katex and https://github.com/graemephi/kahugo. Is it to merge them upstream?

Closing in favor of #10894.

As I understand it, #10894 is about allowing to bypass markdown generation for some delimited blocks, which would allow client-side rendering of math without using shortcodes.
Would it be possible to keep the idea of server-side rendering discussed in the present issue as a possible future improvement for Hugo?

Reopening this, with some notes:

I am renaming this issue to indicate a (for now) generic implementation
The MathML implementations that I have tried have all been buggy
The best rendering quality and performance that I have seen is client-side KaTeX
Both client-side KaTeX and client-side MathJax have accessibility features
Given the wide spread client-side usage of both KaTeX and MathJax, I don't see either of them being discontinued in the foreseeable future
Another server-side option is via the MathAPI service (or your own) as described here

Items 3, 4, and 5 are covered by https://github.com/gohugoio/hugo-goldmark-extensions/issues/1.
As mentioned in https://github.com/gohugoio/hugo/issues/10894#issuecomment-1885407097, at some point we could potentially create a render hook that would allow you to pass the content captured by hugo-goldmark-extensions/passthrough through a template function. Something like:
{{ .Inner | transform.LatexToMathML }}
{{ .Inner | transform.LatexToSVG }}

That gives us a lot of flexibility.

Solved by #12679?

@ldeso yes. I'm closing this.


perl-Data-Dumper: 2.154 -> 2.161 + build fix

Motivation for this change
package is broken in 17.03, current 17.09, and master
Things done
removed the devdoc output, updated to latest version from CPAN

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[ ] NixOS
[ ] macOS
[x] Linux

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

cc @rycee

Rebased into master in dcdbe960f8d6852599e01e46f0467ef7546aa39e. Thanks! Note, I had a look in the package and saw that it uses the Perl license so I amended the commit to have the correct license. Unfortunately Perl developers quite often skip inserting the right license and dependencies in the meta data and then nix-generate-from-cpan will produce subpar results :-(

would it be possible to backport this into 17.03?

Yes, I will get it into 17.03 and 17.09 as well. In the middle of another change but will do this right after :-)

Thanks!

In 17.03 and 17.09 now.


Server Token in info.plist

Hello as it documented in the the section

All the button needs to gather this additional information is your server token to be configured in the Info.plist as described in the SDK Configuration section. To display a time estimate, set the product ID and pickup location. To display a price estimate, you need to additionally set a dropoff location

So when I go back to SDK Configuration section i am unable to find the information for which server token key is going to be used in the app info.plist. am i missing something ?
Attaching screenshot please confirm Server token key

I'm having the EXACT same doubt about this.
How am I supposed to set the server token in the Info.plist?
The SDK Configuration section here doesn't say anything about it.

UberServerToken is the correct key to set in Info.plist (this seems to be missing in the SDK Configuration as you said).

Ok great cheers guys 👍

Sorry for the confusion, I will be sure to update the documentation for the next release

Just a reminder that the docs wasn't updated up until now.

Sorry about that, the last release was not a scheduled release and we have not yet had a chance to work through all the documentation updates.
If you want, you could open a PR with what you expected the docs to say and I would be happy to review it

Fixed in v0.5.3


DotLabelable mechanism doesn't suit enum cases with associated values

The whole DotLabelable mechanism could do with a rethink - for example if your state enum cases have associated values (which mine do, and is an awesomely powerful feature of this approach) then you can't easily create the DotLabelable array (unless your associated values are something simple like strings).

Hi,
I understand it can be annoying but I don't see it as a big problem: you just have to provide a value of the type you declare. I'm happy to discuss other approaches but nothing else comes to my mind given the fact that the list of items used to generate the diagram has to be passed to the actual transition logic code in order for the diagram to be always up to date.
Here's an example from my project:
enum Dynamic {
    case Normal
    case Gentle
}

enum Strength {
    case Normal
    case Strong
    case VeryStrong
}

enum SquareNodeEvent: DOTLabelable {
    case Highlight(delay: Double, dynamic: Dynamic, strength: Strength)
    // ...

    static let DOTLabelableItems = [
        Highlight(delay: 0, dynamic: .Normal, strength: .Normal),
        // ...
    ]

    var DOTLabel: String {
        switch self {
            case .Highlight: return "Highlight(delay, dynamic, strength)"
            default:         return "\(self)"
        }
    }
}


tiller canary image out of date

The sha I get when I install the canary image for tiller (either by downloading the client via http://storage.googleapis.com/kubernetes-helm/helm-canary-darwin-amd64.tar.gz and running helm init or by running helm init --canary-image) is fc315ab59850ddd1b9b4959c89ef008fef5cdf89 from Feb 14.
 $ helm version
Client: &version.Version{SemVer:"v2.2.0", GitCommit:"e9e6dd3927c76658fc97c22f9fa285f43ce20438", GitTreeState:"clean"}
Error: cannot connect to Tiller

 $ helm init --canary-image
...

 $ helm version
Client: &version.Version{SemVer:"v2.2.0", GitCommit:"e9e6dd3927c76658fc97c22f9fa285f43ce20438", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.2.0", GitCommit:"fc315ab59850ddd1b9b4959c89ef008fef5cdf89", GitTreeState:"clean"}

/cc @adamreese

I noticed that with the current make workflow, version metadata won't get updated if you're re-running make build on the source code and rootfs/tiller already exists. Perhaps it's just a "false positive" in that the actual server code is up to date, but the version metadata is not, possibly due to CircleCI caching or something?
Just a crackpot theory, take it as you will

Scratch that, seems to be working as intended.
><> git co v2.2.3
Note: checking out 'v2.2.3'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b new_branch_name

HEAD is now at 1402a4d... chore(*): bump to v2.2.3
><> make clean build
GOBIN=/home/bacongobbler/go/src/k8s.io/helm/bin go install  -tags '' -ldflags ' -X k8s.io/helm/pkg/version.Version=v2.2.3 -X k8s.io/helm/pkg/version.GitCommit=1402a4d6ec9fb349e17b912e32fe259ca21181e3 -X k8s.io/helm/pkg/version.GitTreeState=clean' k8s.io/helm/cmd/...
><> ./bin/helm version --client
Client: &version.Version{SemVer:"v2.2.3", GitCommit:"1402a4d6ec9fb349e17b912e32fe259ca21181e3", GitTreeState:"clean"}
><> git co v2.2.2
Previous HEAD position was 1402a4d... chore(*): bump to v2.2.3
HEAD is now at 1b33072... chore(*): bump to v2.2.2
><> make build
GOBIN=/home/bacongobbler/go/src/k8s.io/helm/bin go install  -tags '' -ldflags ' -X k8s.io/helm/pkg/version.Version=v2.2.2 -X k8s.io/helm/pkg/version.GitCommit=1b330722aafcb8123114ae51f69d1e884a326f3e -X k8s.io/helm/pkg/version.GitTreeState=clean' k8s.io/helm/cmd/...
><> ./bin/helm version --client
Client: &version.Version{SemVer:"v2.2.2", GitCommit:"1b330722aafcb8123114ae51f69d1e884a326f3e", GitTreeState:"clean"}

@vdice the image-pull policy is set to only pull if the image is not on the host.  It will use whatever canary you have or pull the latest.


GameServer Idea

Could you combine JSlither for the gameserver? or at least see its source code since its open-source? it would help you to make the game server get finished faster/

I have been testted almost all opensource servers for Plither, only this worked for me but it has alot of bugs.
Serve for Plither: https://github.com/iiegor/slither

This JavaScript version also works. This one is actively developed though

@RowanHarley, Plither is now designed specifically for your development


Allow ICMP response to DHCP messages in DHCP snooping test

In the event that a device has received a DHCP address from server 1, and still maintains communications with server 1, the device is allowed to reject messages from server 2. However, the DHCP snooping test checks the monitor.pcap for any packets not allowed. Currently, that test will report fail if the device has responded with an ICMP port unreachable to a DHCP ACK from server 2. Because the reject ICMP message includes the top part of the packet (which contains the DHCP ACK) this is picked up as a failure.

What issue is being addressed here?  I'm not sure what this solves or if ICMP and DHCP can both be defined within the same packet.

The screen shot still indicates DHCP and ICMP as two separate packets and the code is expecting both protocols to be contained in the same packet, which I don't believe is possible.
Can you add a unit test around this specific failure?


Need a "prepare libraries for AOT" page, like "prepare libraries for trimming"

Should specify how to set up the analyzer (IsAotCompatible=true) and how to set up a test console app.

There's a thorough blog post on this.
@eerhardt do you mind if we use that as a starting point for the official docs?

@eerhardt do you mind if we use that as a starting point for the official docs?

I don't mind it at all! Feel free to use/modify/etc anything in that post. It was written more in a "blog tone", but the content and recommendations are the same.


Cuda error(3), initialization error.

问题类型：PaddleX可视化客户端

NVIDIA-SMI 441.22       Driver Version: 441.22       CUDA Version: 10.2
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Wed_Oct_23_19:32:27_Pacific_Daylight_Time_2019
Cuda compilation tools, release 10.2, V10.2.89
===================================
This log file path is E:\paddlex_workspace_AI\projects\P0008\T0009\err.log
注意：标志为WARNING/INFO类的仅为警告或提示类信息，非错误信息
F:\PaddleX\paddle\fluid\framework.py:301: UserWarning: You are using GPU version Paddle, but your CUDA device is not set properly. CPU device will be used by default.
F:\PaddleX\paddle\distributed\parallel.py:120: UserWarning: Currently not a parallel execution environment, paddle.distributed.init_parallel_env will not do anything.
F:\PaddleX\paddle\tensor\creation.py:125: DeprecationWarning: np.object is a deprecated alias for the builtin object. To silence this warning, use object by itself. Doing this will not modify any behavior and is safe.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
Process Process-1:4:
Traceback (most recent call last):
File "multiprocessing\process.py", line 297, in _bootstrap
File "multiprocessing\process.py", line 99, in run
File "paddlexui\pms\model_tasks\tasks.py", line 58, in _call_paddlex_train
File "", line 983, in _find_and_load
File "", line 967, in _find_and_load_unlocked
File "", line 677, in load_unlocked
File "c:\users\dltp-sz-windows\anaconda3\envs\paddlex30x0\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
File "site-packages\paddlex_init.py", line 20, in
File "", line 983, in _find_and_load
File "", line 967, in _find_and_load_unlocked
File "", line 677, in load_unlocked
File "c:\users\dltp-sz-windows\anaconda3\envs\paddlex30x0\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
File "site-packages\paddlex\cv_init.py", line 15, in
File "", line 983, in _find_and_load
File "", line 967, in _find_and_load_unlocked
File "", line 677, in load_unlocked
File "c:\users\dltp-sz-windows\anaconda3\envs\paddlex30x0\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
File "site-packages\paddlex\cv\models_init.py", line 17, in
File "", line 983, in _find_and_load
File "", line 967, in _find_and_load_unlocked
File "", line 677, in _load_unlocked
File "c:\users\dltp-sz-windows\anaconda3\envs\paddlex30x0\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
File "site-packages\paddlex\cv\models\detector.py", line 25, in
File "", line 983, in _find_and_load
File "", line 967, in _find_and_load_unlocked
File "", line 677, in load_unlocked
File "c:\users\dltp-sz-windows\anaconda3\envs\paddlex30x0\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
File "site-packages\paddlex\ppdet_init.py", line 15, in
File "", line 983, in _find_and_load
File "", line 967, in _find_and_load_unlocked
File "", line 677, in load_unlocked
File "c:\users\dltp-sz-windows\anaconda3\envs\paddlex30x0\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
File "site-packages\paddlex\ppdet\data_init.py", line 16, in
File "", line 983, in _find_and_load
File "", line 967, in _find_and_load_unlocked
File "", line 677, in load_unlocked
File "c:\users\dltp-sz-windows\anaconda3\envs\paddlex30x0\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
File "site-packages\paddlex\ppdet\data\transform_init.py", line 15, in
File "", line 983, in _find_and_load
File "", line 967, in _find_and_load_unlocked
File "", line 677, in _load_unlocked
File "c:\users\dltp-sz-windows\anaconda3\envs\paddlex30x0\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
File "site-packages\paddlex\ppdet\data\transform\operators.py", line 41, in
File "", line 983, in _find_and_load
File "", line 967, in _find_and_load_unlocked
File "", line 677, in load_unlocked
File "c:\users\dltp-sz-windows\anaconda3\envs\paddlex30x0\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
File "site-packages\paddlex\ppdet\modeling_init.py", line 22, in
File "", line 983, in _find_and_load
File "", line 967, in _find_and_load_unlocked
File "", line 677, in load_unlocked
File "c:\users\dltp-sz-windows\anaconda3\envs\paddlex30x0\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
File "site-packages\paddlex\ppdet\modeling\proposal_generator_init.py", line 1, in
File "", line 983, in _find_and_load
File "", line 967, in _find_and_load_unlocked
File "", line 677, in _load_unlocked
File "c:\users\dltp-sz-windows\anaconda3\envs\paddlex30x0\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
File "site-packages\paddlex\ppdet\modeling\proposal_generator\rpn_head.py", line 58, in
File "site-packages\paddlex\ppdet\modeling\proposal_generator\rpn_head.py", line 74, in RPNHead
File "site-packages\paddlex\ppdet\modeling\proposal_generator\anchor_generator.py", line 57, in init
File "site-packages\paddlex\ppdet\modeling\proposal_generator\anchor_generator.py", line 83, in _calculate_anchors
File "site-packages\paddlex\ppdet\modeling\proposal_generator\anchor_generator.py", line 83, in
File "site-packages\paddlex\ppdet\modeling\proposal_generator\anchor_generator.py", line 76, in generate_cell_anchors
File "", line 2, in to_tensor
File "site-packages\paddle\fluid\wrapped_decorator.py", line 25, in impl
File "site-packages\paddle\fluid\framework.py", line 227, in impl
File "site-packages\paddle\tensor\creation.py", line 160, in to_tensor
OSError: (External)  Cuda error(3), initialization error.
[Advise: Please search for the error code(3) on website( https://docs.nvidia.com/cuda/archive/9.0/cuda-runtime-api/group__CUDART__TYPES.html#group__CUDART__TYPES_1g3f51e3575c2178246db0a94a430e0038 ) to get Nvidia's official solution about CUDA Error.] (at C:\home\workspace\Paddle_release2\paddle\fluid\platform\gpu_info.cc:99)

安装的是cuda 11.2，cudnn 8.0？

是2060显卡
安装的是 cuda 10.2   cuDNN 7

升级到cuda 11.2，cudnn 8.0即可


Fix sequence and integrate in MySQL storage implementation

#21

Codecov Report
Attention: Patch coverage is 0% with 5 lines in your changes missing coverage. Please review.

Project coverage is 31.09%. Comparing base (46ec9c2) to head (aa8ab34).
Report is 20 commits behind head on main.

Files
Patch %
Lines

storage/mysql/mysql.go
0.00%
5 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #101      +/-   ##
==========================================
- Coverage   35.80%   31.09%   -4.71%
==========================================
  Files          16       16
  Lines        1363     1563     +200
==========================================
- Hits          488      486       -2
- Misses        801     1003     +202
  Partials       74       74

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Docs improvements; sentinel-1-rtc

Refactor the docs to organize similar how pandas' documentation.
Added a getting started guide for submitting workflows to the dev environment and adding a new dataset.
Also, add the Sentinel-1 RTC dataset

Merging as this is a significant improvement over current documentation and does restructuring that docs for other work should fit to. Fixes and follow up can be done in subsequent PRs


avoid typing_extensions <IP_ADDRESS>

That specific version has a bug on Python 3.10
See https://github.com/python/typing/issues/865
What does this pull request do?

Related issues
closes #ISSUE

:green_heart: Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2021-08-30T12:29:09.389+0000

Duration: 31 min 32 sec

Commit: 33bda938f73fda891a356031fd626c030f9b8fb9

Test stats :test_tube:

Test
Results

Failed
0

Passed
9593

Skipped
8604

Total
18197

Trends :test_tube:

:green_heart: Flaky test report
Tests succeeded.

Expand to view the summary

Test stats :test_tube:

Test
Results

Failed
0

Passed
9593

Skipped
8604

Total
18197


implement turf/circle in react-mapbox-gl (react hooks)

Hi there, im still new in react/react-mapbox-gl, I would like to know how can I implement @turf/circle in layer marker? or is there any other options that I can use?  What i want to achieved, is each marker will have a circle duration of 5km. Thank you very much!

I have the same problem. Someone solved this?


fix for #3266: wrong-type-arg stringp from format

Literate configurations trigger issue #3266 when tangling on interactive save and epdfinfo is missing. For some reason, pdf-tools-install's prompt for installation ends up calling message(nil) which our letf!version does not handle well. My patch ignores these cases, which results in the prompt working and both epdfinfo installation and tangling succeeding when saving a literate config.

Good catch! Sorry for missing that -- fixed!

Thanks!


[Bug]: resetAllMocks restoring mocks instead of resetting them

Version
29.4.0
Steps to reproduce
Simple test to reproduce:
test('it should reset properly', () => {
  const obj = {
    foo: () => 42,
  };

  const fooSpy = jest.spyOn(obj, 'foo');

  console.log(obj.foo());
  expect(obj.foo).toHaveBeenCalled();

  jest.resetAllMocks();

  console.log(obj.foo());
  expect(obj.foo).toHaveBeenCalled();
});

Expected behavior
The test should pass
Actual behavior
The test fails with Matcher error: received value must be a mock or spy function.
Additional context
It seems that resetAllMocks is acting like mockRestore instead of mockReset.
If I replace jest.resetAllMocks() with fooSpy.mockReset(), the test passes as expected, whereas if I replace it with fooSpy.mockRestore() it fails with that error again.
Environment
System:
    OS: Linux 5.15 KDE neon 5.26 5.26
    CPU: (16) x64 AMD Ryzen 7 PRO 6850U with Radeon Graphics
  Binaries:
    Node: 18.12.0 - ~/.nvm/versions/node/v18.12.0/bin/node
    npm: 8.19.2 - ~/.nvm/versions/node/v18.12.0/bin/npm
  npmPackages:
    jest: ^29.4.0 => 29.4.0

Are you working on this?

No, I don't know enough about Jest internals to suggest a fix.
Besides, I'd expect that a certain multi-billion-dollar corporation would be maintaining this. Don't get me wrong, I'm all for contributing to open source but if the project is backed by a huge corporation, I think they can spare the time to fix it.

This line in this PR introduced the regression; https://github.com/facebook/jest/pull/13692/files#diff-7ae0bd704c3c2789b19abe2bbf94aca3505e2a6b3823d85c7f6b316b216d37c9R1411
Looking here (https://github.com/facebook/jest/pull/13692/files#diff-7ae0bd704c3c2789b19abe2bbf94aca3505e2a6b3823d85c7f6b316b216d37c9L1409), the same function is called to restore the mocks. While it's called reset in the lambda function of the new addition, it's still the same function being called.
Unsure if just removing this call would fix the issue, or if something else needs to be done to still retain functionality of that PR

@feliperli Could you take a look, please? Seems like your PR (#13692) introduced this issue. Or is that expected behaviour?

Besides, I'd expect that a certain multi-billion-dollar corporation would be maintaining this.

By the way, Jest is just another open source project maintained by few volunteers. See https://jestjs.io/blog/2022/05/11/jest-joins-openjs

TIL, thanks for pointing that out.
I did end up doing some digging, and while I'm not any closer to finding the root cause, I did notice this:
  resetAllMocks(): void {
    this._spyState.forEach(reset => reset());
    this._mockConfigRegistry = new WeakMap();
    this._mockState = new WeakMap();
  }

  restoreAllMocks(): void {
    this._spyState.forEach(restore => restore());
    this._spyState = new Set();
  }

It seems that both of those methods end up calling all the _spyState functions and on line 1401 just above, you see this._spyState.add(restore); so it seems that resetAllMocks is calling the restore function (but confusingly naming it reset in the forEach).
I'm not sure if this is intentional, or if it's not, what a fix would look like, but hopefully this info at least helps.

I'm not sure if this is intentional, or if it's not, what a fix would look like, but hopefully this info at least helps.

also seeing this impact our unit tests, with only a regenerated lockfile causing the breaking change

@feliperli Could you take a look, please? Seems like your PR (#13692) introduced this issue. Or is that expected behaviour?

Oh no :( I'm looking into it.

I also see this!!!!

I have a different problem, but I think the cause is the same change in mockReset.
Someone called spyOn on a mock function and it caused a maximum call stack error
This is an example that reproduces the error:
describe('test console.log', () => {
  console.log = jest.fn();
  const log = jest.spyOn(console, 'log');

  afterEach(() => {
    log.mockReset();
  });

  test('console.log a', () => {
    console.log('test a');

    expect(log).toBeCalledWith('test a');
  });

  test('console.log b', () => {
    /**
     * An error is thrown on the next line:
     * RangeError: Maximum call stack size exceeded
     * at WeakMap.get (<anonymous>)
     * at ModuleMocker._ensureMockState (node_modules/jest-mock/build/index.js:275:33)
     */
    console.log('test b');

    expect(log).toBeCalledWith('test b');
  });
});

When I remove the line this._mockConfigRegistry.set(f, originalMockImpl); it works fine.

Seems related: I'm also getting random RangeError: Maximum Call Stack Size Exceeded that are pointing to the jest-mock package, reverting pack to 29.3.1 fixes the issue.

Should be fixed in #13866

https://github.com/facebook/jest/releases/tag/v29.4.3

29.4.3 fixes the original issue for me, along with the "Maximum Call Stack Size Exceeded" error. Thanks all!


[move-vm] Relax script function rules

Apologies for the large PR, but per directory, the changes are fairly isolated and all related to the extensions to the session API. I'd recommend starting with the vm runtime, then the bytecode verifier, then the file format, then the transactional test runner, then the tests. This is the sort of logical flow that should help the changes make sense.

Scripts and public(script) functions can now take arguments of any type and return any values
Checks are preserved for legacy file format versions
Checks must be invoked by the adapter

Change log:
VM/Language
- Removed execute_script_function entry point
- Removed execute_function_for_effects
+ Added return values to execute_function and execute script
  * lets you implement execute_function_for_effects
- Removed most checks from script_signature verification
  * logic preserved for old files
  * NOTE visibility checks was not preserved and must be done in the adapter,
    since execute_script function was removed
  * This allows structs, references, and arbitrary signers in entry point functions
  * This allows return values for public(script) functions
    * added necessary checks for these changes
+ Added helpers in script_signature verification for adapter checks
+ Bumped VERSION_MAX

Testing
+ Added printing of return values and mutable inputs

Follow ups:
Return values for public(script) (and really anything from execute_function) are now supported.
The same support should be added to normal scripts
Motivation

We want to move the verification of entry point functions to the adapter layer

Test Plan

Extended transactional test runner
Added many new tests

This is great! I don't have the cycles to review from where I'm right now, but I'm wondering whether a followup step should be to just deprecate the public(script) fun stuff from the language, and also the old style script modules.
If Move is going to deployed in different ways (Aptos, Sui, Async Move, EVM Move, Starcoin, TBD Solana Move, ...) it appears misleading and counterproductive to pretend there would be a 'standard' transaction model. Rather different deployment environments have different notions of transactions with different context restrictions.
Instead of encoding the model of transactions into the language, we could use attributes to flag the transaction model. For example, in EVM Move, we use #[callable] fun, in Async Move we use #[message] fun, and so on. There is already a (bloody primitive but working) attribute derivation and context checking model in language/move-compiler/src/attr-derivation which is best established so far for Async Move.
@sblackshear @davidiw  @jolestar

It is an interesting idea that I hadn't considered (just dropping public(script)).
But because there are no annotations in the CompiledModule, I think the big issue though would be that it then makes a programmers life a bit confusing. It would be unclear at the bytecode level for what functions were intended to be invoked and what functions can actually be invoked by the adapter.
It also makes tooling a bit more difficult, since when you see a compiled module, it isn't entirely clear what functions need a ABI/transaction generation (though this is adapter specific)
Writing this all out I might tweak the session API a bit to do some level of visibility checks...
I agree that we might do away with the whole thing at some point! But it is probably too big of a change for this PR? maybe?

I agree that most entrypoint checking should indeed be done in the adapter, since each deployment will have its own entrypoint rules. However, I think there are two checks that are still useful to do inside the Move verifier:

ensure that entrypoints are segregated from ordinary public functions--i.e., we currently check that public(script) functions cannot be called from public or private functions, but allow public(script) functions to call each other.
ensure that entrypoints are segregated from each other--i.e., cannot be called from any other function, including other entrypoints. We don't current have a visibility for this, but I think it would be valuable (public(entrypoint)?). In Sui, we want this because there are properties we want to check on arguments that the adapter has enough information to check, but these properties cannot be checked from inside Move (and thus would be violated in cross-entrypoint calls).

Some adapters won't care about (or indeed, will not want) either of these restrictions--i.e., they  want entrypoints to be callable from other entrypoints, public functions, and private functions. Such adapters can represent entrypoints via ordinary public functions w/ an annotation, as suggested.

I think it is indeed valuable to maintain a bytecode-level visibility modifier that represents an entrypoint. Without this, there is no way for a programmer to convey that a function should not be an entrypoint. We actually have this problem Sui right now. We have some entrypoint validity checks that the adapter enforces, but we only want a subset of the functions that pass those checks to be callable as entrypoints. We need this enforcement at the bytecode level because occasionally, an attacker calling an "accidental entrypoint" as an entrypoint would cause issues. public(script) (or an equivalent, but better-named bytecode visibility modifier) is the natural mechanism for solving this problem.

Giving the adapter more freedom to customize its entrypoints is good, but one of my concerns is that the VM adapter can only check the entry function at runtime.
The developer writes and deploys the modules and then finds out that the function is not defined correctly when he calls the entry function, which affects the development experience.
So is there a way to allow custom entry functions and at the same time check them in the compiler time?
Maybe the #[callable] fun annotation way can achieve this goal?
For example:
#[script]
public fun main(sender: singer,){
}

The function signature requirements for #[script] and public(script) are the same. Now we can use #[script] to replace public(script), and every adapter can define its annotation.
But maybe we need to keep the annotation with the bytecodes for tools like generating ABI, as @tnowacki says.

Giving the adapter more freedom to customize its entrypoints is good, but one of my concerns is that the VM adapter can only check the entry function at runtime

I think adapters will want/need custom bytecode verifier passes for this (e.g., Sui works this way). At the source level, we should make it easier to write compiler extensions or linters that enforce the same properties as the verifier pass. Neither of these requires annotations from the programmer--both the verifier and compiler can work directly off of the visibility modifier.

Neither of these requires annotations from the programmer--both the verifier and compiler can work directly off of the visibility modifier.

I agree with you that for most customized adapters, technically, this can be done off the existing language visibility modifiers (public(script)). But from a language design perspective, I find it rather problematic that the same syntax in Move would mean different static and dynamic semantics. I think it would be desirable to require attributes for transaction functions to indicate what flavor of Move one is actually targeting,
I also think the current nomenclature of "script" functions is very confusing. Personally, I never understood why we called this "scripts". I've a very different understanding of this notion than how it is used in Move. I would prefer to drop this notion, and mark transaction entry points exclusively via attributes.

I am fully onboard with changing or deprecating the script name, which no longer makes sense. public(entry) is the alternative I like best, but not religious here.

I am fully onboard with changing or deprecating the script name, which no longer makes sense. public(entry) is the alternative I like best, but not religious here.

I have some longer thoughts on this whole space. Which I will writeup for further discussion elsewhere.
But backing up a bit, this PR should be fairly inline with the previously discussed changes to how these functions work. So it would be super helpful to get a review on this shortly.
Given the heavy session/runtime changes, @vgao1996's would particularly helpful :)

But backing up a bit, this PR should be fairly inline with the previously discussed changes to how these functions work. So it would be super helpful to get a review on this shortly.

I have taken a fairly careful look and am happy to approve modulo concerns about keeping the existing API with efficient serialization and a more structured return type are addressed. This will free up Victor to focus on reviewing https://github.com/diem/move/pull/160, which I have less context on.

Ok, point taken. I guess I feel that writing an adapter was/is hard and this type made it a bit easier, so I'm a bit sad to see it go away. But making life easy for adapter-writers is a problem for later, no reason to block this improvement on it...

We could do some fancy things here, but not sure how much "easier" it would make things. I can do some experimentation after this lands

/land


Handle PATCH relationship BaseType in WorkItem API

Related to #526

Current coverage is 68.71% (diff: 100%)

Merging #565 into master will increase coverage by 0.02%

@@             master       #565   diff @@
==========================================
  Files            69         69
  Lines          3810       3813     +3
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits           2617       2620     +3
  Misses          927        927
  Partials        266        266

Powered by Codecov. Last update d96ee8a...416dfc6


NullPointerException Faced when running the Main Class

Attached above is the problem as seen on the GUI

UIManager Class seems to be having an issue in that the start function is facing a null input as reflected

TaskListPanel.java is looking for TaskListPanel.fxml.

But the file name is PersonListPanel.fxml.


DM-8586: Policies for presentations and publications

This adds two new pages describing DM's policies and procedures surrounding conference presentations and publications:

https://developer.lsst.io/v/DM-8586/processes/presenting-at-conferences.html

includes information on how to upload presentations to Zenodo.

https://developer.lsst.io/v/DM-8586/processes/publication-policy.html

Links to Publication Board documentation

TODO

[x] Add DM meetings page: https://confluence.lsstcorp.org/display/DM/DM+Meetings
[x] Figure out status of the Speaker's Bureau and Interaction Reports from the Project Office.
[x] Determine and add requirement/suggestion markers to each section.
[x] Re-add the "pre-approved materials" link; LSST PO will update this material.
[x] Indicate that both presentation and posters go to Zenodo.
[x] Document that proceedings papers go to arXiv and pub board, and do not go to Zenodo.
[x] Provide citation examples for LDM docs and DMTN/SQR technotes. See PUB-39 discussion.

@timj @jdswinbank I think I've got it all sorted it out. Does the current draft look good to you?

👍
Thank you!

Two comments: Firstly, the "preapproved" link seems to have gone. We do really have to treat that as truth that has to be adhered to and it will be updated. Secondly, now that I've read the Zenodo section, posters and presentations go on Zenodo (posters were not explicitly mentioned) and proceedings definitely do not (one line indicates that they do). Proceedings papers go on Arxiv and must be submitted to the publication board (might want to note that as well).

Thanks @timj, will do. One question: the arXiv posting happens after Pub Board approval, right?
Note that I've included https://developer.lsst.io/v/DM-8586/processes/publication-policy.html in this PR. I can have a section in the https://developer.lsst.io/v/DM-8586/processes/presenting-at-conferences.html (maybe right above the Zenodo section) that specifically refers to the workflow for proceedings articles.
Is there anything I can change about the publication-policy page to make it more useful?

Yes, arXiv happens after pub board approval.

The pub policy page is fine. Might be worth adding a couple of examples on how to cite an LDM and a DMTN.

I'm studying the bibtex citation recommendations with AASTeX 6.1.
Here's the simple LDM example:
@misc{LDM-135,
    author =       {Jacek Becla and others},
    title =        "{LSST Database Design}",
    howpublished = {LDM-135, \url{http://ls.st/LDM-135}},
    year =         2013,
}

The LDM-135 citation seems to be missing a field in the aastex bib style, compared to http://journals.aas.org/authors/aastex/aasguide61.html#softwareandthirdparty I think this is the DOI field, but could also be a url or note. I couldn't make it work, though, short of replicating the aastex guide's Zenodo example.

@techreport{slater_2016_192828,
    title        = {{False Positive Rates in the LSST Image
                     Differencing Pipeline}},
    author       = {{Slater}, Colin and
                    {Juri\'{c}}, Mario and
                    {Ivezi\'{c}}, \u{Z}eljko and
                    {Jones}, Lynne},
    institution  = {{LSST}},
    type         = {{LSST Data Management Technical Notes}},
    number = {{DMTN-030}},
    month        = mar,
    year         = 2016,
    doi          = {10.5281/zenodo.192828},
    url          = {https://doi.org/10.5281/zenodo.192828}
}

Ignoring the issue with "Ž", this techreport format seems to work well. Maybe it could be used for LDMs as well?

Those both look fine. We don't have DOIs for LDMs so it doesn't really matter but I don't object to the @techreport option.

@timj are we ready to call this "reviewed" ?


Replace plugin endpoints with placeholder endpoints

Summary
Replace all plugin endpoints with the ones from the Placeholder admin
Documentation checklist

[ ] I have updated CHANGELOG.txt if appropriate
[ ] I have updated the release notes document if appropriate, with:

[ ] general notes
[ ] bug-fixes
[ ] improvements/new features
[ ] backwards-incompatible changes
[ ] required upgrade steps
[ ] names of contributors

[ ] I have updated other documentation
[ ] I have added my name to the AUTHORS file
[ ] This PR's documentation has been approved by Daniele Procida

Coverage remained the same at 78.189% when pulling 5d5ef2792995f2f271246fb86a7b246e0d3b9e3b on Aiky30:replace-plugin-endpoints-with-placeholder-endpoints into 71a68e1a0a195951268bf70ffa01a89f8f00aba7 on divio:release/4.0.x.


fixed small errors

Thank you! I always do a lot of typos and the spell checker is not working when is edit the readme. :sweat_smile:


Zendesk no more possible?

As reported here https://support.zendesk.com/hc/en-us/articles/203664356-Changing-the-address-of-your-Help-Center-subdomain-host-mapping- Zendesk subdomain takeover requires making the subdomain an alias of default address. So it shouldn't be possible get a subdomain takeover without getting access to the domain registrar's control panel.
Am I wrong?

This takeover is possible when a Zendesk account has been deleted, but still remains available in the host domains DNS record.
When you dig the record you will see the CNAME, this corresponds to the name of the Zendesk account you need to (re)create. Once you've done so you'll now have claimed Zendesk on this host domain.
 It's useful for red teaming and social engineering, but I wouldn't expect the bounty payments to be as much as in other scenarios where you can demonstrate more relevant risk.

My experience with a zendesk takeover attempt this morning.

Thank-you @JesseClarkND, I was also familiar with this one through a recent attempt. I've made a change to the repo in line with your proofs and will now close this issue (see: #51).
Appreciate your efforts!

Zendesk is still Vulnerable .. All Just check all the site subdomains if any subdomain is add to zendesk account and the other added too ( Only one will be the cname and the other one will redirect you to The finger print that give you the error )
Example if I add my subdomain ( support.example.com ) to my zendesk account with mine cname and add my other subdomain  ( help.example.com ) to the same cname ..
my first subdomain support.example.com will show me my Zendesk Site But When I go to help.example.com I will find my self redirect to the finger print error

So you can takeover a Subdomain already Exist ?
Sure you can't !

Only if the company have an Only subdomain listed in Zendesk and the subdomain redirect you to finger print error So Sure  you can Takeover!

i tried taking over an expired zendesk subdomain and it worked but after i active the hep center in zendesk setting menu, it should appear on the subdomain but it is still redirecting to the previous error. Anybody can tell me, if it takes time to get published on the main subdomain after activation.

@bgxdoc If you could describe your steps to achieve it that worked, that would be awesome.

I just got a successful takeover today.
Simply located a domain which was redirect to a Zendesk page saying "This help center has been deleted".
Found the CName. Registered that on ZenDesk as my account. Now it's pointing to mine. Setup an SSL cert so that it stops redirecting to my ZenDesk, and instead it actually hosts it on the real subdomain.
Need to figure out a way to get stored XSS via zendesk admin so that I can improve on the impact...

Double dipping there? Get a bounty from ZenDesk and the company?
@Cillian-Collins See if you can use the ZenDesk name to also generate emails from that domain, you can leverage that email to pivot into internal slack/jira instances.
Read:
https://medium.com/intigriti/how-i-hacked-hundreds-of-companies-through-their-helpdesk-b7680ddc2d4c

Takeover is possible, and you can just post HTML (JavaScript, XSS) with the helpdesk webpage.

If the Name of domain not take you can take over it You need to Enable Ssl Certificate and  Enable Host Maping option and it will work

i tried taking over an expired zendesk subdomain and it worked but after i active the hep center in zendesk setting menu, it should appear on the subdomain but it is still redirecting to the previous error. Anybody can tell me, if it takes time to get published on the main subdomain after activation.
upate: i have only trial account, is that why zendesk is not showing it on the subdomain?
Update: it worked.

how it worked what you changed

on above comment, yosefmahmoud has already described the solution. It's the same thing i did to make it work.
" If the Name of domain not taken you can take over it You need to Enable Ssl Certificate in security settings and Enable Host Maping option in account settings and it will work. "

I got one takeover but it seems zendesk need premium option to stop
redirection  any trick to get rid of premium or to get zendesk premium free
for testing

@gauravdrago
I believe Zendesk has Free Trial option, use that. Incase you have used the trial account before then simply create a new account and then use free trial.

But in trial hostmaping not allowed.

@gauravdrago
It worked for me back then. but why not now, I don't know, i guess they might have disabled it for trial accounts.
I will look into this once i get free time from my work.

Ok boss.

Not Sure About The Free Trial and Premium . To Solve This Zendesk has Three cases when you see helpdesk is close :
1-  Deleted  Help Desk You can Take over it
2-  inactive you can't Take over it until 90 days passed
3-  sometimes compaines used only chat widget from zendesk without help center it will respond with the same message but You can't take over
Not Sure if there are any cases or not if someone already faced another case he can put it to solve this issue

still possible

@roblox1488 Did you have a specific method that you used that is separate from the above comments?

@roblox1488 Did you have a specific method that you used that is separate from the above comments?

you have to wait a while after the subdomain stops being in use, then you can just register it normally like you would

There are two possibilities when dealing with a closed Zendesk helpcenter as far as I've encountered so far.
..............

Thanks @JvdHout1011 for good write-up. Yes it is still possible to hijack/takeover subdomain through zendesk portal. I takover the subdomain of support.*******.com and i'm receiving their help desk emails on zendesk dashboard.
POC : https://drive.google.com/file/d/1h4QomND3n7O5dRCzyGmHQ78qdFcpK51L/view?usp=sharing

Thank you @JvdHout1011
I can confirm the takeover still working, i tookover a subdomain through zendesk portal few minutes ago.
POC here :
https://prnt.sc/s80l2i
https://prntscr.com/s80mhi

Update : There's something wrong with the trial period ending just after confirming my email. Happened 5 times in 5 subdomains I was trying to takeover.
Is it cause i was using the same ip? (will try from another ip)
Anyone confirm this issue please?

@JvdHout1011 I can confirm the takeover still works.
@EdOverflow Please update this. Using @JvdHout1011's method, the takeover is still possible.

There is a report already about this take over on H1. I believe it is possible.
https://hackerone.com/reports/810807

@soareswallace I confirm, already took over 10 subdomains the last 3 days, I sent reports to hackerone but most of them are "ineligible" and they said "We do in fact consider bounty payment on a case by case basis for bounty ".

https://www.youtube.com/watch?v=sBTVWKeh7xY

Seems is still vulnerable.
https://www.youtube.com/watch?v=sBTVWKeh7xY

Hello, I didn't test if it's still vulnerable or not (been 4 months since my last test), I will check hat tomorrow and will keep you updated.
(The POC video is mine from my report here : https://hackerone.com/reports/869605 but someone downloaded it and posted it on youtube, so the youtube channel is not mine)

Not sure why this is marked as "Not vulnerable"... As others have reported above,

Zendesk is still vulnerable, although the conditions for that may have changed over time;
Regarding impact, such a subdomain takeover is a phisher's wet dream, and you can also achieve stored XSS in the context of the subdomain, which may have implications for other subdomains of the target (e.g. if they have improperly scoped sensitive cookies).

In my experience, there are two necessary conditions for a host to be vulnerable. Let's take zendesk.example.com as an example. First, zendesk.example.com must have a CNAME targeting some zendesk.com subdomain.
Second, zendesk.example.com must be available as custom domain name on the platform. No need to check in the Web UI: you can check that programmatically by sending a request of the following form (which your browser sends when you sign up for a free trial on Zendesk):
curl -s 'https://www.zendesk.com/wp-content/themes/zendesk-twentyeleven/lib/domain-check.php' \
      -H 'authority: www.zendesk.com' \
      -H 'pragma: no-cache' \
      -H 'cache-control: no-cache' \
      -H 'accept: */*' \
      -H 'x-requested-with: XMLHttpRequest' \
      -H 'user-agent: REDACTED' \
      -H 'content-type: application/x-www-form-urlencoded; charset=UTF-8' \
      -H 'origin: https://www.zendesk.com' \
      -H 'sec-fetch-site: same-origin' \
      -H 'sec-fetch-mode: cors' \
      -H 'sec-fetch-dest: empty' \
      -H 'referer: https://www.zendesk.com/register/' \
      -H 'accept-language: en-GB,en-US;q=0.9,en;q=0.8,fr;q=0.7' \
      --data-raw "domain=$subdomain" \
      --compressed

Inspect the JSON response. You can safely ignore the value of the available field, which I have found to be misleading. What truly matters is the presence of a suggestion field in the response; if that field is present, the custom domain name is unavailable; otherwise, the custom domain name is available.
Be mindful that Zendesk sits behind Cloudflare, and if you hammer it with such requests, you'll likely get a temporary IP ban from Cloudflare.

A subtlety: Zendesk offers you to provision a TLS cert (using Let's Encrypt) for your custom domain, but this provisioning can fail. One reason for failure is when the root domain features on Let's Encrypt deny list; in that case, provisioning of the cert will fail, but Zendesk's frontend won't give you any more details. Even if the root domain doesn't feature on LE's deny list, provisioning may fail multiple times; in that case, be patient and persistent: retry a few times.
If you cannot obtain a certificate for your custom domain, visiting it will simply redirect you to the Zendesk subdomain, which greatly reduces impact.


Decomission m2lines

terraform destroy already run
Buckets were deleted from the web console
GitHub apps were deleted

Fixes https://github.com/2i2c-org/infrastructure/issues/3484

Thank you, @GeorgianaElena!


Crawling Cursor support

Just wondering if there's any plans to integrate crawling cursor support now that mouse controls are supported? https://github.com/jujunjun110/aframe-crawling-cursor
Would it be possible to attach the super-hands component to the crawling cursor itself (which should have a sphere-collider attached to it), while still being able to watch for mousedown and mouseup clicks for grab start/end or drag start/end?

After further investigation, it seems this is possible without any modifications by using crawling-cursor in this way:
<a-ring id="my-cursor" radius-inner="0.05" radius-outer="0.1" sphere-collider="objects:.cube; radius: 0.1" lerp="properties: position, rotation"  super-hands="grabStartButtons: mousedown; grabEndButtons: mouseup;" static-body="shape: sphere; sphereRadius: 0.02;"></a-ring>
<a-entity camera raycaster universal-controls crawling-cursor="target: #my-cursor">

(note: for the above the static-body component only needs to be set if using the physics component. I'm also using the lerp component for smoothing out the position and rotation of the crawling-cursor)
You also have to watch for clicks on the window in JS and trigger the onGrabStartButton() and onGrabEndButton() methods of the super-hands component manually though:
window.addEventListener('mousedown',function(e){
            this.cursorEl.components['super-hands'].onGrabStartButton();
}.bind(this));

window.addEventListener('mouseup',function(e){
            this.cursorEl.components['super-hands'].onGrabEndButton();
}.bind(this));

Hopefully this helps someone!

So it looks like the only thing keeping crawling cursor form working correctly with the cursor-nested-inside-camera setup, which makes the events propagate as expected, is confusion over global coordinates versus local coordinates + parent object offsets in the crawling-cursor component.
If you nest the cursor further within an entity that offsets the camera height, it seems to work:
      <a-camera raycaster crawling-cursor>
        <a-entity position="0 -1.6 0">
          <a-cursor super-hands="colliderEvent: mouseenter;
                                 colliderEventProperty: intersectedEl;
                                 colliderState: cursor-hovered;
                                 grabStartButtons: mousedown;
                                 grabEndButtons: mouseup;
                                 dragDropStartButtons: mousedown;
                                 dragDropEndButtons: mouseup;">

          </a-cursor>
        </a-entity>
      </a-camera>

https://steadfast-art.glitch.me


livecheck: use brewed curl if called for by download URL

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[ ] Have you written new tests for your changes? Here's an example.

[x] Have you successfully run brew typecheck with your changes locally?
[ ] Have you successfully run brew tests with your changes locally?

Initial swing at addressing #12341. Has livecheck use brewed curl if it's using the :stable (formula), :url (cask) or :homepage URL and the download URL specifies using: :homebrew_curl, e.g. https://github.com/Homebrew/homebrew-core/pull/88835.

Review period will end on 2021-11-26 at 05:19:15 UTC.

Review period ended.

If we don't handle it in this PR before merging, I can take care of it in a follow-up PR.

Let's do that in follow-up, yeh 👍🏻


v2 development plan

Here's my plan for creating a new backend:

Interpret what the PHP is doing
Implement in JS here
Remove translated bits
Repeat until there's nothing left

It's going to be messy at times and later refactoring will be necessary. Please submit PRs :smiley_cat:
If anyone knows of unused parts of the old backend, please submit a PR to my fork with what can be skipped.
If you think there's a better approach to creating a new backend, open to suggestions :pray:

SGTM!
One thing we can get rid of right away is ASSETS_DOMAIN. Let’s just use the same domain as the main site.

I think I have all the static content transferred over minus a few things like proper error pages and versioned assets.
To keep things simple, I am planning on using the mysql package to start the dynamic pieces since the PHP uses a lot of raw queries.

I'm blending a lot of what I like in jedireza/aqua and npm/newww. I plan on documenting how I'm thinking about repositories and services, possibly turning into Hapi plugins if it makes sense.
I'm hoping page revisions and comments will be easier now that the big initial page save is done :package:

Is anything big still missing?

the biggest thing left is retrieving test pages by slug. I created a milestone and issues for most of the features I identified from the old _tpl directory. now that it's broken out, hopefully I can get back in the groove or others can more easily jump in. perhaps some could be moved to a v2.1 milestone if they're not super important.

A material design would be a nice gimmick as well, but of course there are far more important things right now.

I am a frontend developer looking to contribute to jsperf, as it has helped me learn JavaScript and working with the dom in efficient ways.
I can cook up a front end in CSS/html using the new material lite framework as well as any html rendering engine that you are working with.
I am not a senior developer but my skills are good enough to contribute to an open source project.
Material design lite: http://www.getmdl.io
Company I work for: https://www.policygenius.com

@abacaj thanks for offering to help. We're currently focused on only changing the server for v2. Updating the client code and/or styles would most likely be a v3 milestone.

Ah I see, well I will be around if needed :+1:

I have an idea that is to use Dropbox public API as your backend so that if anyone who wants to flood your backend, they need to use their own Dropbox for storing their junk.
Of course the drawback is also obvious that normal users have to connect their Dropbox account to jsperf for storing their code.

Any update on this? Need any help in any areas?

@shaunwarman These issues are blocking the v2 release: https://github.com/jsperf/jsperf.com/milestones/v2 Help welcome!

How about force the use to login with github and use their gists to store the tests?
I'm not sure how you would make it public though

I'm sorry if I've misread the code, but it seems like index.php would be vulnerable to sql injection through, for example, $_GET['id'].
I had this problem when I was creating a game, so, instead of placing the values from get directly in the string, I had to bind the parameters like this:
$qry = $conn->prepare("SELECT name FROM submitted WHERE category = ?");
$qry->bind_param('s', $_POST['category']);
Also, $_SERVER['HTTP_HOST'] references a value provided by the user's browser so it can be spoofed.

I'm sorry if I've misread the code, but it seems like index.php would be vulnerable to sql injection through, for example, $_GET['id'].

Can you point to an example where $_GET['id'] is being used without having checked that it’s an integer or without it being escaped?

Also, $_SERVER['HTTP_HOST'] references a value provided by the user's browser so it can be spoofed.

AFAIK it’s only being used here: https://github.com/mathiasbynens/jsperf.com/blob/002825bc3b111714ec0b7b2e02bed72258fdf492/index.php#L21-L23 …which seems harmless. Spoofing this value would only cause a redirect.

I did read it wrong. I missed your cast to int.
And I meant that the php could be accepting data generated by different page, with the browser incorrectly reporting that it was coming from the correct page.

And I meant that the php could be accepting data generated by different page, with the browser incorrectly reporting that it was coming from the correct page.

I don’t understand. Could you clarify, please?

What @danielcre is referring to is that HTTP_HOST is defined as:

Contents of the Host: header from the current request, if there is one.

http://php.net/manual/en/reserved.variables.server.php

That means whatever is sent in the request is what you are using, and the request is up whatever client is sending it. Browsers are not the only things capable of using HTTP to access your back-end. You cannot make any assumptions about what that may contain or that it even exists.
Disclaimer: I am not sure where the usage in question even is; I am just helping clarify.

@mathiasbynens The $_GET is being cast to int but it still is a good idea to use @danielcre suggestion of preparing statements rather than using query method. While everything is probably safe in this iteration, a future iteration of the code could result in a programmer accidentally allowing user passed values to somehow circumvent validation/escaping and then having a prepared statement will offer better protection than concatenating in the values. I look forward to using the page again once it's back up. Thanks.

@mrosata I agree. The new version (i.e. the code in this very repository) is doing it right.


DataGridBodyProps in react-data-grid-react-window does not accept ref: RefObject

The DataGridBody in react-data-grid-react-window does not accept ref: RefObject;
Can ref been added to the type of DataGridBodyProps ? This is required to fix sticky header with body horizontal scroll bar.
Type error on tsc compile:
Type '{ children: RowRenderer<T>; ref: RefObject<HTMLDivElement>; itemSize: number; height: number; listProps: { overscanCount: number; onItemsRendered: any; ref: any; }; onKeyDown: (e: KeyboardEvent<...>) => void; }' is not assignable to type 'IntrinsicAttributes & Omit<DataGridBodyProps, "children"> & { itemSize: number; height: number; width?: string | number | undefined; children: RowRenderer<...>; ariaRowIndexStart?: number | undefined; listProps?: any; }'.
  Property 'ref' does not exist on type 'IntrinsicAttributes & Omit<DataGridBodyProps, "children"> & { itemSize: number; height: number; width?: string | number | undefined; children: RowRenderer<...>; ariaRowIndexStart?: number | undefined; listProps?: any; }'.ts(2322)

@FengQiMS will you be able to solve the sticky header issue?


Can not find the KSCrash.framework

I download the project, opened the KSCrash-iOS.xcodeproj, selected the KSCrash scheme and the ios device. Then I Archive the project.  But nothing happened after finishing building . I can not see the framework in the Finder. Why?😂

Open Products Folder
U will find the framework and right click with open in finder, you can locate the framework.

Same problem, I open KSCrash-iOS.xcodeproj，and Archive，when it finished, nothing happened. (The KSCrash.framework of Products folder appear to be red, I right click it and open in finder, it didn't get the framework location)

u have to select the target "kscrash_static",and then u can archive,it work~


[FEATURE] Fix the spinning disk on Longhorn

Based on the result from #1691 , we found the spinning disk doesn't work well with Longhorn. It's a current limitation but there are still many users using spinning disk out there. We need to investigate the Longhorn data path to find out the bottleneck/root cause of the spinning disk that doesn't work well and easily disconnected from Longhorn.

The way to control latency

Use the following script to create a delayed target via  dmsetup (Ref: https://enodev.fr/posts/emulate-a-slow-block-device-with-dm-delay.html):

#!/bin/sh
size=$(blockdev --getsize $1) # Size in 512-bytes sectors
echo "0 $size delay $1 0 500" | dmsetup create delayed

Run fio tests to verify the latency
Mount the corresponding disk /dev/dm-0 to a mount point then introduce the mount point on Longhorn Node page

@PhanLe1010

The way to control latency

Use the following script to create a delayed target via  dmsetup (Ref: https://enodev.fr/posts/emulate-a-slow-block-device-with-dm-delay.html):

#!/bin/sh
size=$(blockdev --getsize $1) # Size in 512-bytes sectors
echo "0 $size delay $1 0 500" | dmsetup create delayed

Run fio tests to verify the latency
Mount the corresponding disk /dev/dm-0 to a mount point then introduce the mount point on Longhorn Node page

@PhanLe1010

Initial test result:
Environment:

Longhorn version: v1.1.0
Kubernetes distro (e.g. RKE/K3s/EKS/OpenShift) and version: RKE with Kubernetes v1.19.7

Number of management node in the cluster: 1
Number of worker node in the cluster: 3

Node config

OS type and version: Ubuntu 16.04.4 LTS

CPU per node: 2

Memory per node: 8GB

Disk type(e.g. SSD/NVMe): EBS volume using spinning disk. The worst type of spinning disk in AWS

Network bandwidth between the nodes: up to 1 Gbps bandwidth

Underlying Infrastructure (e.g. on AWS/GCE, EKS/GKE, VMWare/KVM, Baremetal): AWS EC2, the nodes are t2.large instances

Workload:
I deploy the following statefulset to test the Longhorn system:
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: shell
spec:
  selector:
    matchLabels:
      app: shell
  serviceName: "shell"
  replicas: VARIABLE
  template:
    metadata:
      labels:
        app: shell
    spec:
      restartPolicy: Always
      containers:
      - name: shell
        image: ubuntu
        command: ["/bin/bash"]
        args: ["-c", "n=0; while true; do dd if=/dev/urandom of=/mnt/file count=100000 bs=1K status=progress; sync; cat /mnt/file > /dev/null; rm /mnt/file; n=$((n+1)); echo running count: $n; done"]
        volumeMounts:
        - name: magnetic
          mountPath: /mnt/
        imagePullPolicy: IfNotPresent
  volumeClaimTemplates:
  - metadata:
      name: magnetic
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "longhorn-magnetic"
      resources:
        requests:
          storage: 5Gi

This workload does intensively read/write to Longhorn volumes
Observation:

Incrementally increase the value replicas: VARIABLE from 1 to 3. Everything running ok. An excerpt of the log from one of the workload pod:
11148288 bytes (11 MB, 11 MiB) copied, 1 s, 11.1 MB/s
19811328 bytes (20 MB, 19 MiB) copied, 2 s, 9.9 MB/s
27597824 bytes (28 MB, 26 MiB) copied, 3 s, 9.2 MB/s
39123968 bytes (39 MB, 37 MiB) copied, 4 s, 9.8 MB/s
51320832 bytes (51 MB, 49 MiB) copied, 5 s, 10.3 MB/s
63299584 bytes (63 MB, 60 MiB) copied, 6 s, 10.5 MB/s
75100160 bytes (75 MB, 72 MiB) copied, 7 s, 10.7 MB/s
87510016 bytes (88 MB, 83 MiB) copied, 8 s, 10.9 MB/s
99921920 bytes (100 MB, 95 MiB) copied, 9 s, 11.1 MB/s
100000+0 records in
100000+0 records out
102400000 bytes (102 MB, 98 MiB) copied, 9.19789 s, 11.1 MB/s
running count: 19

Incrementally increase the value replicas: VARIABLE to 4 and 5. Volumes start degrading and the system enters a chaotic situation in which many volumes start rebuilding.

From the initial test, It seems the spinning disk is not a no-go for Longhorn, and Longhorn can still work with spinning disks until some load limit.

Next step:
I will increase the network bandwidth to see if we can increase replicas: VARIABLE to 4, 5, and above.

Initial test result:
Environment:

Longhorn version: v1.1.0
Kubernetes distro (e.g. RKE/K3s/EKS/OpenShift) and version: RKE with Kubernetes v1.19.7

Number of management node in the cluster: 1
Number of worker node in the cluster: 3

Node config

OS type and version: Ubuntu 16.04.4 LTS

CPU per node: 2

Memory per node: 8GB

Disk type(e.g. SSD/NVMe): EBS volume using spinning disk. The worst type of spinning disk in AWS

Network bandwidth between the nodes: up to 1 Gbps bandwidth

Underlying Infrastructure (e.g. on AWS/GCE, EKS/GKE, VMWare/KVM, Baremetal): AWS EC2, the nodes are t2.large instances

Workload:
I deploy the following statefulset to test the Longhorn system:
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: shell
spec:
  selector:
    matchLabels:
      app: shell
  serviceName: "shell"
  replicas: VARIABLE
  template:
    metadata:
      labels:
        app: shell
    spec:
      restartPolicy: Always
      containers:
      - name: shell
        image: ubuntu
        command: ["/bin/bash"]
        args: ["-c", "n=0; while true; do dd if=/dev/urandom of=/mnt/file count=100000 bs=1K status=progress; sync; cat /mnt/file > /dev/null; rm /mnt/file; n=$((n+1)); echo running count: $n; done"]
        volumeMounts:
        - name: magnetic
          mountPath: /mnt/
        imagePullPolicy: IfNotPresent
  volumeClaimTemplates:
  - metadata:
      name: magnetic
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "longhorn-magnetic"
      resources:
        requests:
          storage: 5Gi

This workload does intensively read/write to Longhorn volumes
Observation:

Incrementally increase the value replicas: VARIABLE from 1 to 3. Everything running ok. An excerpt of the log from one of the workload pod:
11148288 bytes (11 MB, 11 MiB) copied, 1 s, 11.1 MB/s
19811328 bytes (20 MB, 19 MiB) copied, 2 s, 9.9 MB/s
27597824 bytes (28 MB, 26 MiB) copied, 3 s, 9.2 MB/s
39123968 bytes (39 MB, 37 MiB) copied, 4 s, 9.8 MB/s
51320832 bytes (51 MB, 49 MiB) copied, 5 s, 10.3 MB/s
63299584 bytes (63 MB, 60 MiB) copied, 6 s, 10.5 MB/s
75100160 bytes (75 MB, 72 MiB) copied, 7 s, 10.7 MB/s
87510016 bytes (88 MB, 83 MiB) copied, 8 s, 10.9 MB/s
99921920 bytes (100 MB, 95 MiB) copied, 9 s, 11.1 MB/s
100000+0 records in
100000+0 records out
102400000 bytes (102 MB, 98 MiB) copied, 9.19789 s, 11.1 MB/s
running count: 19

Incrementally increase the value replicas: VARIABLE to 4 and 5. Volumes start degrading and the system enters a chaotic situation in which many volumes start rebuilding.

From the initial test, It seems the spinning disk is not a no-go for Longhorn, and Longhorn can still work with spinning disks until some load limit.

Next step:
I will increase the network bandwidth to see if we can increase replicas: VARIABLE to 4, 5, and above.

Notice that the block size you are using is 1K, which may not be able to overwhelm the disk io. And the node CPU maybe not enough, either.

Notice that the block size you are using is 1K, which may not be able to overwhelm the disk io. And the node CPU maybe not enough, either.

Update: we've investigated and identified a few issues with tgt/engine/replica communication on the data path regarding the spinning disk. We need to figure out a solution to fix/improve the situation.

I have some hyperid disk which has ssd cache in front of spinning disks.
When running without longhorn
bash-5.0# mkdir test
bash-5.0# fio -directory=./test -direct=1 -iodepth 1 -thread -rw=randwrite -ioengine=psync -bs=4k -size=1G -numjobs=7 -runtime=120 -group_reporting -name=lhtest --fdatasync=1
lhtest: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.20
Starting 7 threads
lhtest: Laying out IO file (1 file / 1024MiB)
lhtest: Laying out IO file (1 file / 1024MiB)
lhtest: Laying out IO file (1 file / 1024MiB)
lhtest: Laying out IO file (1 file / 1024MiB)
lhtest: Laying out IO file (1 file / 1024MiB)
lhtest: Laying out IO file (1 file / 1024MiB)
lhtest: Laying out IO file (1 file / 1024MiB)
Jobs: 7 (f=7): [w(7)][100.0%][w=468KiB/s][w=117 IOPS][eta 00m:00s]
lhtest: (groupid=0, jobs=7): err= 0: pid=8627: Thu Mar  4 23:32:39 2021
  write: IOPS=345, BW=1384KiB/s (1417kB/s)(162MiB/120049msec); 0 zone resets
    clat (usec): min=52, max=358030, avg=1751.97, stdev=5710.57
     lat (usec): min=52, max=358030, avg=1752.31, stdev=5710.59
    clat percentiles (usec):
     |  1.00th=[    81],  5.00th=[    93], 10.00th=[   101], 20.00th=[   116],
     | 30.00th=[   143], 40.00th=[   174], 50.00th=[   196], 60.00th=[   233],
     | 70.00th=[   306], 80.00th=[   429], 90.00th=[  6849], 95.00th=[ 11076],
     | 99.00th=[ 21365], 99.50th=[ 25822], 99.90th=[ 47973], 99.95th=[ 71828],
     | 99.99th=[120062]
   bw (  KiB/s): min=  168, max=11720, per=99.10%, avg=1370.49, stdev=264.23, samples=1673
   iops        : min=   42, max= 2930, avg=342.56, stdev=66.06, samples=1673
  lat (usec)   : 100=9.68%, 250=53.35%, 500=19.21%, 750=1.84%, 1000=1.59%
  lat (msec)   : 2=0.79%, 4=0.95%, 10=6.39%, 20=4.87%, 50=1.24%
  lat (msec)   : 100=0.07%, 250=0.01%, 500=0.01%
  fsync/fdatasync/sync_file_range:
    sync (usec): min=304, max=463951, avg=18476.02, stdev=30512.81
    sync percentiles (usec):
     |  1.00th=[   453],  5.00th=[   494], 10.00th=[   523], 20.00th=[   570],
     | 30.00th=[   660], 40.00th=[  1090], 50.00th=[ 10159], 60.00th=[ 17171],
     | 70.00th=[ 25035], 80.00th=[ 33817], 90.00th=[ 43254], 95.00th=[ 53740],
     | 99.00th=[121111], 99.50th=[246416], 99.90th=[362808], 99.95th=[400557],
     | 99.99th=[408945]
  cpu          : usr=0.06%, sys=0.49%, ctx=155451, majf=0, minf=0
  IO depths    : 1=200.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,41529,0,0 short=41529,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
  WRITE: bw=1384KiB/s (1417kB/s), 1384KiB/s-1384KiB/s (1417kB/s-1417kB/s), io=162MiB (170MB), run=120049-120049msec

Disk stats (read/write):
  sda1: ios=0/124756, merge=0/0, ticks=0/133504, in_queue=133504, util=44.17%

I have them spread over 6 nodes, running only 1 replica for each volume. I am not even able to spawn more than 5 volumes at the same time. Got a lot of timeout error when attaching volume.

We're running out of time to fix this in v1.1.1 release. Move to v1.1.2.

I made a slow disk with 500ms delay with dmsetup, and I used it to store the longhorn volume.   I edited tgt to hardcode the number of threads to 1 and ran fio.
I ran fio:
...
fio --name=random-writers --ioengine=libaio --rw=randwrite --bs=4k --direct=1 --size=3000m --numjobs=1 --filename=/mnt/file.dat --iodepth=128
...h
With the tgt with only 1 thread, the mount stayed up and was just slow.
I did this test again with tgt using the default of 16 threads and almost immediately the replica timed out:
...
ERRO[0046] I/O error: No backend available
response_process: Receive error for response 3 of seq 137
tgtd: bs_longhorn_request(105) fail to read at<PHONE_NUMBER> for 4096
tgtd: bs_longhorn_request(150) io error 0x5570e1ce6910 28 -14 4096<PHONE_NUMBER>, Success
ERRO[0046] I/O error: No backend available
response_process: Receive error for response 3 of seq 138
tgtd: bs_longhorn_request(105) fail to read at<PHONE_NUMBER> for 4096
tgtd: bs_longhorn_request(150) io error 0x5570e1ce6910 28 -14 4096<PHONE_NUMBER>, Success
ERRO[0046] I/O error: No backend available
response_process: Receive error for response 3 of seq 139
tgtd: bs_longhorn_request(91) fail to write at<PHONE_NUMBER> for 4096
tgtd: bs_longhorn_request(150) io error 0x5570e1d06950 2a -14 4096<PHONE_NUMBER>, Success
ERRO[0046] I/O error: No backend available
response_process: Receive error for response 3 of seq 140
tgtd: bs_longhorn_request(91) fail to write at 0 for 4096
tgtd: bs_longhorn_request(150) io error 0x5570e1ce6910 2a -14 4096 0, Success
ERRO[0046] I/O error: No backend available
response_process: Receive error for response 3 of seq 141
tgtd: bs_longhorn_request(91) fail to write at 0 for 4096
tgtd: bs_longhorn_request(150) io error 0x5570e1ce6910 2a -14 4096 0, Success
...

@keithalucas
Can you show how you find the pending operations in tgt?
Do you mean that you are trying to stop sending more IO to the engine controller when the pending IOs in TGT is big? If so, I still see the timeout problem. Assume that the number of thread in TGT is 16, there will be 16 IOs on the path from the engine controller -> replica client even if there is no other pending IOs on the TGT side. If the replica client cannot handle 16 IOs within the 4-second timeout. We will hit the timeout problem.

Or do you mean that we dynamically increase and decrease the number of threads in TGT instead of the hard-code 16 threads? This would be very interesting.

Thanks @keithalucas  for the great explanation!
I am looking forward to see the impact of reducing number of thread to the performance on a fast disk.
As a side note, after we solve the timeout in TGT, if the disk is slow enough, we can hit the timeout in the ISCSI initiator. I wasn't be able to understand this part yet. Please let me know if you have new finding on this :)

Same concern with Phan. By blocking the message init in tgtd, we threshold the input throughput of longhorn engines so that the underlying engine or replica won't be overwhelmed by the IO requests. But the top layer applications can still have tons of IO requests then the iscsid may be flooded. How can we handle the exceeding requests from applications/filesystem to iscsid, or those from iscsid to tgtd?

I have a thought about this theme which may be helpful or totally out of place:
We recognize the problem of instability of your longhorn volumes. The  volumes break - switch into readonly mode for example. First I thought that this was the spinning disk issue. So I replaced the servers with SSD disks. But the problem still occurs.
Now we realized, that we have a network problem in our cluster. It happens from time to time that the network get lost on a node for a short period of time. For example today one of our 3 node longhorn cluster lost the network. And as a result again our PODs running longhorn volumes on that node fail after some time. The network interruption was only about 1 minute.
Now I wonder, if the problem you are discussing here is not only related to spinning disks but can also be related to Network or I/O in general?
Sorry if this thought is nonsense, I am not on the same level as you discussing I/O issues. But maybe it is a helpful hint....

@rsoika There are multiple sub-senario related to the node/kubelet/network down or restart. And those cases are not related to this spinning disk issue. If the disconnected node still works, and there are local replicas for the volumes on the node, there can be no downtime ideally. But for now, Longhorn will blindly restart the volumes in those cases. We are fixing this issue in #2629.

Artificially limit the concurrent thread number will likely reduce the Longhorn throughput even when the node/disk is fast enough. So it might have a severe impact on the performance. If possible, we need a solution that doesn't have any negative impact on the performance.

@keithalucas moving to analsys from backlog pipeline, because you are working on it.

Due to the difficulty to automate the verification in the engine integration test, we will use the manual test to cover all the test cases.

@keithalucas make sure to add https://github.com/longhorn/longhorn-instance-manager/pull/91 to the checklist - backend code PR. We might need to backport the fix into v1.1.3.

@keithalucas Please help backport to 1.1.3. Thanks.

Validation -  In progress
Setup details:
1 etcd/control plane and 3 worker nodes
CPU - 4 vpucs
RAM - 16GB
Bandwidth - Upto 10Gbps
OS - Ubuntu 18.04

Validation of  intense I/O on the block device
Test steps from https://longhorn.github.io/longhorn-tests/manual/pre-release/resiliency/simulated-slow-disk/
With Longhorn v1.1.2-rc1
Write operation works fine till 200s delay in the disk created on top of the SSD.
Starting with 300s delay, the write operation starts to throw timeout error like below
[slow-vol-9-e] time="2021-07-19T21:47:05Z" level=error msg="Write timeout on replica <IP_ADDRESS>:10001 seq= 994 size= 4 (kB)"
time="2021-07-19T21:47:05Z" level=error msg="Retry  1 on replica <IP_ADDRESS>:10001 seq= 994 size= 4 (kB)"
[slow-vol-9-e] time="2021-07-19T21:47:05Z" level=error msg="Write timeout on replica <IP_ADDRESS>:10001 seq= 995 size= 4 (kB)"
time="2021-07-19T21:47:05Z" level=error msg="Retry  1 on replica <IP_ADDRESS>:10001 seq= 995 size= 4 (kB)"

With Longhorn master-head - 07/16/2021
Tested with 300s delay in the disk created on top of the SSD. The test cases from https://longhorn.github.io/longhorn-tests/manual/pre-release/resiliency/simulated-slow-disk/ passed
The iops observed on the disk was  iops        : min=    2, max=   64
Tested with 500s delay in the disk created on top of the SSD. The test cases from https://longhorn.github.io/longhorn-tests/manual/pre-release/resiliency/simulated-slow-disk/ passed
The iops observed on the disk was  iops        : min=    1, max=   30
Tested with 600s delay in the disk created on top of the SSD. The test cases from https://longhorn.github.io/longhorn-tests/manual/pre-release/resiliency/simulated-slow-disk/ passed
The iops observed on the disk was  iops        : min=    2, max=   30

@keithalucas Please help backport to 1.1.3. Thanks.
cc @khushboo-rancher

I backported it to v1.1.3.

Hi @keithalucas , QA team will close the issue once they verified everything works with v1.1.3. cc @khushboo-rancher

I've validated this with v1.1.3 and updated the test result https://github.com/longhorn/longhorn/issues/2206#issuecomment-882957063
@keithalucas While testing https://longhorn.github.io/longhorn-tests/manual/pre-release/resiliency/timeout/, I observed that the replica failed if the network in disconnected for more than 8 sec. With v1.1.2, the replica doesn't fail until disconnect for more than 15 sec. Is this expected?

That's a good question. Thinking about it, it will depends on how long can filesystem tolerates no response. Someone told me filesystem in fact can tolerate a long time without response. In that case, do we still want to cut the connection for 8 seconds or even 15 seconds?

The previous behavior for ping retried the ping when a ping response wasn't sent for eight seconds.  A new ping message was sent and if a response for the second message wasn't received within eight seconds, the longhorn engine would fail the replica.  The previous behavior required two eight second timeouts.   Once the retry was initiated, any response from the first ping attempt was ignored.  With the previous implementation, the replica still has to respond to a ping within eight seconds for it not to fail.  There could have been an aberration that causes the ping to take more than eight seconds once but subsequently takes less than eight seconds.   However, if the ping response is consistently taking between 9-15 seconds, it will fail with both implementation.    I felt like the retry mechanism didn't have much benefit, especially since the communication is over TCP and there is no possibility that the replica missed the first message.  We could revisit the ping timeout and either only disconnect on read or writes and not use pings or increase the ping value.

I see. This makes sense giving the point that we're only removing the case of first call timeout in 8s then the second call comes back in 8s, which is kind of pretty edge case. I am fine with the change to 8 seconds timeout in total in the current design.

As the validation is done for the fix of supporting higher latency on the disk and detailed testing is planned with https://github.com/longhorn/longhorn/issues/1691, closing this.


Tie resources utilization (CPU, Network/IOs) to Fortio runs

We need some api (or script) to gather CPU, Network (and disk IOs if/when needed) during the fortio load tests so we can characterize the cost per QPS

regpatrol does it and it's indirectly there with grafana dashboards (#5028)


Validate Project window throws exception when camera is no called "Main Camera"

Problem
The Validate Project window expects that the main camera in the scene is called "Main Camera".
If it is not, then an exception is thrown and the window fails to draw correctly.
Repro steps

Open a new scene in a project with the ARR package installed
Rename the camera to "MyCamera"
Open Validate Project window.

Observed result
Exceptions are thrown and the window fails to draw correctly.
Expected result
The main camera can be called anything with no errors thrown. Suggest using Camera.main, instead of finding by game object name.

Hi, just a quick heads up: We merged a fix into our internal branch and I'll update the issue once it's released publicly. Thanks again for the report and helping us to build a better product :)

Hi, we pushed a new release which contains a fix for this issue. Thanks for letting us know!


Mac OS Electron-Packager Issue

Environment
Knex version: 0.16.3
Database + version: SQLite3 4.0.6
OS: OSX El Capitan 10.11.5
Bug

Error seems to be because it can't find the database file. Works totally fine during development before packaging as long as database file is in same directory.
I'm able to develop and package on Windows 10 and it works so seems to be OSX specific.

Error message: "Sorry! Uncaught Exception!TypeError: Error processing argument at index 11, conversion failure from undefined"

similarly, doesn't work when replacing the settings.get with just a "./test.db"

Sounds like user error of electron or bug in sqlite3 driver. Nothing in this report seem to point knex having a bug. First try to use sqlite3 directly and if you can make that work with your electron setup only then try to include also knex.
Can be reopened if this problem is shown to be caused by a bug in knex.


Course review form

Add a course review form to /course/id, should probably be a modal after hitting a 'Add review' button on the course page.
Some fields to enter:

title
description
rating (break it up into useful/easy ratings?)

Done in #73


xcode 12.4 build error： Undefined symbol: OBJC_CLASS$_ATTrackingManager [iOS 14.4]

Hey am sorry to bother again but your advice is very valuable.
I'm running iOS 14.4 on an iPhone 12 Pro Max[14.4],but it always has an error ：
Undefined symbols for architecture x86_64:
"OBJC_CLASS$_ATTrackingManager", referenced from:
objc-class-ref in libTrackingTransparency.a(TrackingTransparency.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
“Undefined symbol: OBJC_CLASS$_ATTrackingManager”
I try to change the build setting and build path to fix it，but it didn't work。
How can i do to fix it？
Thank you

Try cleaning and rebuilding

Try cleaning and rebuilding

I clean the project and clean the DerivedData，but it still didn't work.


2016-ml-contest submittal #1 on Jan. 30th, 2017 by Pet-Stromatolite

team Pet-Stromatolite (JustinGosses) 1st submittal on Jan. 20th 2017

Hi Justin — welcome to the contest! Good to see you. The files score as follows:
3a       0.614
5a       0.621
5_final  0.595

Good results :) I hope you get the chance to tweak some more!

Matt, thanks for the scores. They were better than expected . I'll try to work on this more this evening. Anything else I need to do submittal wise to appear on the rankings board?