Wip  update spire

This aims to remove the 2_13 dependancy on spire. Spire itself was compiled with 3.1.0 as far as I could tell, so I also bumped breezes scala version.
 Upgrading the collection-compact modules, apparently fixed these problems.
With these changes, the tests in the "math" sub project passed (for me).
Health Warnings:
Spire release is "M2". It may be preferable to wait for a 0.18.0
As I don't have a detailed understanding of Breeze, these changes may not be desirable in some larger context I'm not aware of.

Thanks! I have a slight preference for supporting 3.0.x, but if that's what Spire can support, than probably best to bump. I agree it's best to wait for a full release or at least RC before we cut a release with it.

Just had a spire user run into the dreaded Conflicting cross-version suffixes trying to use spire and breeze together on Scala 3 and found my way here. Thanks for working on this!

I have a slight preference for supporting 3.0.x

Thanks, this is helpful feedback. AFAIK nothing in spire depends on Scala 3.1.x (but can't be sure), so we can try rolling back if it's important to you. This issue has been controversial :) We could also wait and see about https://github.com/lampepfl/dotty/pull/14156, I think they originally promised it for 3.1.1 but I haven't followed closely.

I agree it's best to wait for a full release or at least RC before we cut a release with it.

We released an M3 last week. I don't anticipate any more breaking changes going forward, we can probably RC or go straight to final.

lampepfl/dotty#14156 will be release in 3.1.2. RC1 should be release shortly and stable version 6 weeks later (unless there are no critical bugs).

Likely also need to change:  https://github.com/scalanlp/breeze/blob/master/macros/build.sbt#L10

sorry for being slow. Yeah my preference for 3.0.x was driven by the lack of forwards compatibility, but if they're going to fix that, then that's great.

oops. shouldn't merge things when i have a migraine

(reverted the merge in master. don't know how to reopen a PR)

sorry for being slow. Yeah my preference for 3.0.x was driven by the lack of forwards compatibility, but if they're going to fix that, then that's great.

@dlwh do you still feel the same way about sticking to Scala 3.0.x? I think Spire will go straight to Scala 3.1. I experimented with the new -scala-output-version flag but it seems we've already used some Scala 3.1 APIs in macros so it doesn't work. I'd rather not backport these.
[error] -- Error: /workspace/spire/core/src/main/scala-3/spire/syntax/macros/literalMacros.scala:35:14
[error] 35 |  parseNumber(digits.valueOrAbort.parts, BigInt(-128), BigInt(255)) match
[error]    |              ^
[error]    |method valueOrAbort was added in Scala release 3.1, therefore it cannot be used in the code targeting Scala 3.0

See some discussion on the topic in:

https://github.com/typelevel/scalacheck/issues/878

i don't care that much anymore I guess, so fine with me!
On Thu, May 26, 2022 at 2:21 PM Arman Bilge @.***>
wrote:

sorry for being slow. Yeah my preference for 3.0.x was driven by the lack
of forwards compatibility, but if they're going to fix that, then that's
great.
@dlwh https://github.com/dlwh do you still feel the same way about
sticking to Scala 3.0.x? I think Spire will go straight to Scala 3.1. I
experimented with the new -scala-output-version
https://github.com/sbt/sbt/releases/tag/v1.7.0-M2 flag but it seems
we've already used some Scala 3.1 APIs in macros so it doesn't work. I'd
rather not backport these.
[error] -- Error: /workspace/spire/core/src/main/scala-3/spire/syntax/macros/literalMacros.scala:35:14
[error] 35 |  parseNumber(digits.valueOrAbort.parts, BigInt(-128), BigInt(255)) match
[error]    |              ^
[error]    |method valueOrAbort was added in Scala release 3.1, therefore it cannot be used in the code targeting Scala 3.0
See some discussion on the topic in:

typelevel/scalacheck#878
https://github.com/typelevel/scalacheck/issues/878

—
Reply to this email directly, view it on GitHub
https://github.com/scalanlp/breeze/pull/833#issuecomment-1139070682, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAACLIPCES5G4CBUYK5ZEHTVL7TOBANCNFSM5K7PJG7A
.
You are receiving this because you were mentioned.Message ID:
@.***>


Webapp up does not work in 2.47.0

This is autogenerated. Please review and update as needed.
Describe the bug
I've updated my azure cli to 2.47.0 and now the webapp up commands does not work anymore. My pipeline which uses azure cli 2.43.0 is still working so it is not related to the code
Command Name
az webapp up
Errors:
.../Backend should contain only one .csproj file

To Reproduce:

Go to the solution folder of the dotnet folder
az webapp up --plan my-plan --name my-app --os-type linux --location northeurope --resource-group my-rg --runtime "DOTNETCORE:7.0" --sku B1

Expected Behavior
Upload the project to the web app
Environment Summary
macOS-13.3.1-arm64-arm-64bit, Darwin 22.4.0
Python 3.10.11
Installer: HOMEBREW

azure-cli 2.47.0

Extensions:
rdbms-connect 1.0.4

Dependencies:
msal 1.20.0
azure-mgmt-resource 22.0.0

Additional Context

Thank you for opening this issue, we will look into it.

Closing due to no response.


Progressbar regression: label no longer printed if output is not to a tty

The progressbar documentation says:
    :param file: the file to write to.  If this is not a terminal then
                 only the label is printed.

However, this is no longer the case with newer click versions. Take the following test script:
#!/usr/bin/env python3
import click
from time import sleep

with click.progressbar(range(100), label="Foo") as pb:
    for _ in pb:
        sleep(0.01)

In click 5.0:
$ ./ayy.py
Foo  [####################################]  100%
$ echo "$(./ayy.py)"
Foo
Foo

In click 7.0:
$ ./ayy.py
Foo  [####################################]  100%
$ echo "$(./ayy.py)"

Would this be a regression from the fix of #406 then?


Modify zk configuration to have a stricter autopurge policy.

Please do not merge.

Trusty: https://ocd.appscale.com:8080/job/Daily Build/983/

Precise: https://ocd.appscale.com:8080/job/Daily Build/987/

:+1:


Flat List showing only 10 items

Kudos for providing a beautiful component!
The component is rendering only first 10 items when used in a FlatList. The remaining items are not being rendered.

Try this:
https://stackoverflow.com/questions/54302266/react-native-flatlist-only-renders-10-items
It is not the best solution, but it worked well for me.


refactor: use runner-manager to all eg runners lifecycle

What type of PR is this?

refactor: eg runners

register and start runners:

shutdown runners:

What this PR does / why we need it:
use runner-manager to all eg runners lifecycle
Which issue(s) this PR fixes:

NONE

/retest

hey @Xunzhuo sorry for the delay, will review, once rebased on the e2e pass

hey @Xunzhuo trying to better understand if this PR is needed for v0.5.0-rc1 or not, is this a refactor or does it also solve any bugs ?

Not for v0.5.0 I think

let me close this for now, since code structure has been changed a lot.


Suggest to remove property TreeSet_Comparable

The property TreeSet_Comparable (https://goo.gl/3cxBUf) requires the object to be put into a TreeSet/SortedSet instance
to implement Comparable interface. According to Java 6 specification (https://goo.gl/n91l2v), to put an object into a typed
TreeSet, either the type class itself implements Comparable or the TreeSet is constructed with a
user-defined Comparator that takes the type class. If none of the above 2 requirements is satisified,
JVM will throw an Exception. When this property is triggered with successful running,
it can only be false alarms against programs using comparators
for TreeSet construction, which are legal under Java API specification.
We'd suggest to remove this property as it could only trigger false alarms without providing information for eliminating true bugs beside what has been provided by the Exception on JVM.

Some time ago, I proposed a modification to a similar property called SortedSet_Comparable
which only reports violation when it neither implements Comparable nor uses comparator:
https://github.com/runtimeverification/property-db/commit/b065be011c660062654f400184206e8352899ca6
The monitor should still be able to report the 'true' bug no matter whether JVM will throw Exception, so I think we'd better keep this property.
@owolabileg what do you think?

Hi xiaohe, thanks for the feedback! Could you further elaborate on the reason for "The monitor should still be able to report the 'true' bug no matter whether JVM will throw Exception"? We indeed think some of the properties may be useful even though JVM throw Exception, but some of them seem do not provide further useful debugging information than JVM exception itself. We opened a new issue for one of this kind https://github.com/runtimeverification/javamop/issues/229.


IIS in-process mode still has "conhost.exe" as child process

On Windows Server 2012 R2, we just switched to .NET Core 2.2 and switched from IIS out-of-process mode to IIS in-process mode.
While this removed our own .EXE as a w3wp.exe child process, we still see "conhost.exe" as a w3wp.exe child process:

I would have expected that in-process completely avoids additional child processes.
My question:
Is this expected behaviour, or do we have something misconfigured?

This might be a side-effect of AspNetCore module trying to set up standard output redirection. This process wouldn't execute any application code it's just a part of windows console infrastructure.

Thanks a lot, Pavel!


Update CODE_OF_CONDUCT to current version.

Incorporates the changes of CoC from the contributor covenant to version 2.1.

Ha, no problem! We all have something. I will avoid them in the future.


Can't connect to any servers with this config enabled

Ever since they released this update: https://steamcommunity.com/games/L4D2/announcements/detail/4352250327663410167?snr=2___
I've been having issues connecting to all servers including official Valve ones. Deleting the /cfg folder and verifying the game files allows me to do it again so I am pretty sure it's this config.

I've played two games after the update without any issues. I tested the config without my custom changes, and it works on both my servers and Valve servers. Some friends I play with also use this config and haven't had any problems.
Have you changed or added anything to the config?

Very odd, it seems to have just stopped working for me, nothing changed. I will reinstall it and come back.

Do you see anything in the Console when the issue happens?

Didn't throw any errors, no. Trying now.

Seems to be working again, had to execute exec config_default.cfg; exec autoexec.cfg again.


Jenkins builds are stuck?

https://jenkins.mc-stan.org/
I'm seeing a bunch, including the develop build and some from #2970 as having been executing for the past 10 hours, which I think means something is off. Tagging @serban-nicusor-toptal in case he knows what's up.

Hey, they're not stuck but rather in a long queue for gelman-mac.

Are you sure? For example, this started 12 hours ago and is taking up an executor on a Linux box (because it claims it's running linux integration tests, I think): https://jenkins.mc-stan.org/job/Stan/job/develop/704/

Oh yeah, in the blue ocean build you can see that this build hasn't actually spent that long executing. Something seems a little off though, this is a lot of backlog. Was something stuck on the Mac?

We had issues with the integration tests on Linux running out of tests and there were a lot of math PRs waiting on tests (like 7-8) which created a huge backlog after we restarted them.

Mac is a bottleneck because we now also run Stan integration tests on PRs and we only have a single Mac agent. Maybe we should only run those on merges. Its an unlikely scenario where Linux integration tests pass and Mac integration tests do not. And unlikely scenarios can be covered by merge tests I guess.

For now I think we're fine, usually the queue is not this big but before the weekend we had some job failures in Stan and downstream Stan tests. I've restarted quite a lot of jobs across the projects and that spiked the queue to ~40, they're all waiting now for gelman-mac. It will take a bit longer but it should soon clear out.


multi-response vectors

Responses as input vector files are currently assumed to be single, binary vector responses (E.G. each vector is its own response category).  Consider enabling multi-response vectors.

Actually, responses apparently need to be rasters after I modified the data read.  First things first - the data read needs to be reworked to enable responses to be vector format (partway done...through data checks, but the actual reads need to get updates).

Vectors now enabled, but only one response per vector file (multiple files supported).


add built-in keyword deontics

add this to the roadmap for a future syntax release; edit the baby-l4 README to give concrete examples of what this would look like; then we can aim to support it in a future version of the parser.
Discussion to be held between Martin and others Thu 4pm 20210304
Inari requested decomposition:

I'm thinking about functions like MustNotAcceptApp , can we make deontics in baby-L4 to be a separate part like in Meng's version? I'd like to map Accept to WordNet.accept_2_V2, and Must not  (or Shant) as one of enumerated deontics

if the alternative is regex matching against the thing, let's rewrite the thing into two things.

The lexicon is working differently now:
lexicon MustNotAcceptApp -> "{Party} must not accept {Appointment}"

and the string, excluding the placeholders, is parsed with GF. So nobody needs to insert WordNet IDs anymore, and having enumerated deontics is no longer a strictly technical requirement for the NLG.
Otherwise, is this still a goal? In @mengwong's fantasy syntax of L4, enumerated deontics have been a thing all the time.


merge package template from Kronos-Tools/npm-package-template

LICENSE

fix: update LICENSE

Codecov Report

Merging #307 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #307   +/-   ##
=======================================
  Coverage   87.83%   87.83%
=======================================
  Files           1        1
  Lines         222      222
=======================================
  Hits          195      195
  Misses         27       27

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a54e8eb...4877be6. Read the comment docs.


feat(RELEASE-570): add option to make release repositories public

This adds two new tasks to the rh-push-to-external-registry and push-to-external-registry pipelines:

collect-registry-token-secret: This will collect the .registrySecret value from the data file and pass it as a result.
make-repo-public: This will use the token value stored in registrySecret to make API calls to quay.io to make repositories public.

All of this will be optional. If .mapping.defaults.public is true or any .mapping.components[*].public is true, we will require the secret name to be defined in .registrySecret. Otherwise this will be skipped.
The secret needs to contain the key token containing the quay token. And the token needs to have permission to call the quay.io/api/v1/repository/<repository>/changevisibility endpoint for all the repositories set to be made public.

/test release-pipelines-e2e-suite

/test release-pipelines-e2e-suite

/test release-pipelines-e2e-suite

/test release-pipelines-e2e-suite

/test release-pipelines-e2e-suite

/test release-pipelines-e2e-suite

/test release-pipelines-e2e-suite

/test release-pipelines-e2e-suite

/test release-pipelines-e2e-suite


[WIP] Support hipSPARSE

Close #4101.
Still a preliminary work in progress. This compiles (on ROCm 3.5.0) but I haven't run any test yet.
cc: @hodgestar

Let me do a test drive on ROCm 4.x 😁
/test

pfn-public-ci/cupy.py37.rocm-* are happy.

OK, all tests in tests/cupy_tests/test_cusparse.py pass (or xfail) on ROCm 3.5.0 + Radeon VII and ROCm 4.2.0 + MI50.

Major pain points for hipSPARSE that I have so far (tested on ROCm 3.5.0 & 4.2.0):

transa=True
0-size matrices

Marking these xfail would pass a vast majority of tests.

Running test_{coo, csc, csr}.py would open a can of worms...🤦🏻♂ I can't finish testing any of these files; it's either terminated by segfault or hangs indefinitely. Though this is more or less expectable as these tests contain a lot of custom kernels, and given our past experience with ndimage/signal failures I guess we just have to figure out the offending ones and mark them xfail...

@leofang This is great! Thank you for working on it.

Time for a test drive!
/test

Hmm I thought it could also trigger Jenkins....
Jenkins, test this please

Time for a test drive!

On ROCm 4.0.0 there are some failing tests.

Jenkins CI test (for commit fe9875b85cba2034c796ddef79a8840ac84b0244, target branch master) succeeded!

Hi @amathews-amd Another note in case you missed the PR description: In addition to the nnz=0 issue, in many hipSPARSE functions the handling of transposed matrices (trans=True) is also problematic. The returned error code is either "invalid value" or "internal error". Take a look at all skipped/xfailed tests that I marked in this PR and you'll get a long list...This makes CuPy's sparse matrix support significantly crippled on AMD GPUs, so it'd be great to fix them asap...😅
Jenkins, test this please

@leofang , I did not miss it :) . I will take a look later this week.

Jenkins CI test (for commit a31761de6476b80607002e201af4ca597bae2875, target branch master) succeeded!

Jenkins, test this please

All sparse-related tests passed!
https://jenkins.preferred.jp/job/chainer/job/cupy_pr/3976/TEST=rocm-4-0,label=mnj-mi50/
https://jenkins.preferred.jp/job/chainer/job/cupy_pr/3976/TEST=rocm-4-2,label=mnj-mi50/

Jenkins CI test (for commit d2989660e9f0121129ef394563a826acc2d6935d, target branch master) succeeded!

Renamed the title so that users can understand what's changed from release notes.

Jenkins CI test (for commit d2989660e9f0121129ef394563a826acc2d6935d, target branch master) failed with status FAILURE.


Some errors popping up, Process seems slow.!!

These are some errors I am seeing while running this on a Modal.run instance.
2024-01-24 03:16:06.043152838 [E:onnxruntime:Default, provider_bridge_ort.cc:1480 TryGetProviderInfo_CUDA] /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1193 onnxruntime::Provider& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcurand.so.10: cannot open shared object file: No such file or directory

2024-01-24 03:16:06.043193312 [W:onnxruntime:Default, onnxruntime_pybind_state.cc:747 CreateExecutionProviderInstance] Failed to create CUDAExecutionProvider. Please reference https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements to ensure all dependencies are met.
DEBUG:audio_separator.separator.separator:Model loaded successfully using ONNXruntime inferencing session.
DEBUG:audio_separator.separator.separator:Loading model completed.
INFO:audio_separator.separator.separator:Load model duration: 00:00:00

DEBUG:audio_separator.separator.separator:Window applied to the chunk.
/usr/local/lib/python3.10/site-packages/audio_separator/separator/separator.py:630: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:261.)
  mix_part = torch.tensor([mix_part_], dtype=torch.float32).to(self.device)
DEBUG:audio_separator.separator.separator:Mix part split into batches. Number of batches: 1
DEBUG:audio_separator.separator.separator:Processing mix_wave batch 1/1
/usr/local/lib/python3.10/site-packages/torch/functional.py:650: UserWarning: stft with return_complex=False is deprecated. In a future pytorch release, stft will return complex tensors for all inputs, and return_complex=False will raise an error.
Note: you can still call torch.view_as_real on the complex output to recover the old return format. (Triggered internally at ../aten/src/ATen/native/SpectralOps.cpp:863.)
  return _VF.stft(input, n_fft, hop_length, win_length, window,  # type: ignore[attr-defined]

I am processing a 5-minute audio and running twice. usually, it took around 3/4/5 minutes, but now it takes around 10 mins. I was using version 0.7.2 before.
upgraded to newer version and this is how I am using it now.
Run these commands:
    "pip install pydub",
    "pip install moviepy",
    "pip uninstall torch onnxruntime -y",
    "pip install --force-reinstall torch torchvision torchaudio",
    "pip install --force-reinstall onnxruntime-gpu",
    'pip install --force-reinstall "optimum[onnxruntime-gpu]"',
    "pip install --force-reinstall torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118",

from audio_separator.separator import Separator
    # Initialize the Separator class (with optional configuration properties below)
    separator = Separator()

    separator.load_model("UVR_MDXNET_KARA_2")
    # Perform the separation
     primary_stem_path, secondary_stem_path = separator.separate(input_file_name)

Is there anything I am doing wrong.??

libcurand.so.10: cannot open shared object file suggests you don't have a working CUDA environment.
Googling that error and trying some of the suggestions would be one thing to try.
What CUDA version are you using?
Have you tested CUDA working directly? e.g. with nvidia-smi?
Maybe worth following something like this to test that your installed pytorch/cuda versions are working with each other: https://saturncloud.io/blog/how-to-check-if-pytorch-is-using-the-gpu/
Have you tried running it in docker? I provide a docker image which has been tested with CUDA 11 and eliminates you needing to figure out getting different dependencies installed: docker pull beveradb/audio-separator:gpu

Sorry for the delay,
I'm using nvidia/cuda:11.8.0-base-ubuntu22.04 to start,
but checking the CUDA version I can see it's 12.0
I think this is the issue,
can you suggest me an Image with Cuda version 11

Okay got it working now,
with this configuration.
docker_image = Image.from_registry(
        "nvidia/cuda:11.8.0-base-ubuntu22.04",
         add_python="3.11"
    ).pip_install(
        ['pydub', 'moviepy', 'audio-separator[gpu]', 'boto3']
    ).apt_install(
        ['ffmpeg', 'wget', 'libsndfile1', 'git']
    ).run_commands(
        "pip uninstall torch onnxruntime -y",
        'pip install install torch==2.0.1 "optimum[onnxruntime-gpu]==1.15.0"',


Get rank in fellowship identities

In the fellowship_identities example, the system gets the fellowship keys.
Can the example be modified to use entries instead of keys as entries brings the rank object?
I tried to do it myself but struggled with the object type when converting it into an array using the rune object.
Thank you!

If you use entries and then .access(0) on the member rune exposed in the mapArray cb, it should check. Try the following.
import { collectives } from "@capi/collectives"
import { polkadot } from "@capi/polkadot"
import { ArrayRune, Rune } from "capi"

const limit = collectives.FellowshipCollective.MemberCount.value(0)
const members = collectives.FellowshipCollective.Members.entries({ limit })
const identities = members.into(ArrayRune).mapArray(
  (member) => polkadot.Identity.IdentityOf.value(member.access("0")),
)
const result = await Rune.object({ members, identities }).run()
console.log(result)


Add support for React 18 server rendering

With React 18, the client-side rendering process has been updated.  To prepare the way for supporting React 18's features, the new way of rendering needs to be adopted.
This change creates some helpers that support both methods of rendering and hydrating on the client to start the path forward to supporting React 18.
NOTE: Because there is no current support for testing against React versions, I am forgoing adding extra tests for this.  If there is an approach to doing this, we can add that.

This change is

#1408

@kylemellander Big thanks for trying this out!
https://github.com/reactwg/react-18/discussions/5
Please try to use React.version rather than just checking if the function exists to determine what API to use.
https://www.peterbe.com/plog/display-current-react-version

@Judahmeek please review and merge if you approve and CI Passes.
@kylemellander Thanks for the contribution! Please consider joining us on our Slack for React and Rails.

We're getting this error now:

Warning: You are importing createRoot from “react-dom” which is not supported. You should instead import it from “react-dom/client”.


Consider replacing pronouncingpy with NLTK

Since we already switched to NLTK for the syllabification, we could also use:
http://www.nltk.org/_modules/nltk/corpus/reader/cmudict.html
Would allow using same format for custom_dict, maybe?

I'm actually not a fan of the implementation of rhyming_part() in pronouncing, since it grabs everything from the last stressed syllable to the end of the word... that seems overly restrictive.

Also, it doesn't have a syllabifier that runs on the CMU dict's entries, so I'd have to implement that anyway if I want to do a better job breaking down the word into syllables from a pronunciation POV.

So yeah, not super urgent but might make sense to implement my own version of the two features I use from pronouncing:

Stress pattern (which should be dead easy, it's just finding the 0's, 1's, and 2's in the string.
Rhymes (which I want to alter anyway)


[KGO] Add docs for the KonnectExtension feature

Where is the problem?
The KonnectExtension feature has been added to KGO. We need to create docs for it.
What is your suggestion?
Create docs for the KonnectExtension feature.
Code of Conduct and Community Expectations

[X] I agree to follow this project's Code of Conduct
[X] I agree to abide by the Community Expectations

Completed in https://github.com/Kong/docs.konghq.com/pull/8097.


Complete class declaration for FormData spec

Reference: https://xhr.spec.whatwg.org/#interface-formdata

I manually updated the line numbers in the expectations in tests/fetch/fetch.exp to match the new line numbers form the expanded FormData declaration. Those tests are now passing. Got an error from _build/src/parser/test/run_hardcoded_tests.native src/parser/test/hardcoded_tests.js however:
cd /home/travis/build/facebook/flow/src/parser//../..; \
	_build/src/parser/test/run_hardcoded_tests.native src/parser/test/hardcoded_tests.js
Fatal error: exception Not_found

Travis job: https://travis-ci.org/facebook/flow/jobs/174823776
Don’t know why that would be, but I’d be happy to take a look at it if someone could point me in the right direction.

Also, if you’d prefer not to include the entire FormData interface as defined by the spec because it isn’t supported by all major browsers, we can close this PR.

I'm cool including this! And sorry for the delay! I have some notes, but since this is so delayed I'll address them myself :)

Well I feel silly. Went to rebase this branch and found out we already merged a PR with definitions for this (https://github.com/facebook/flow/pull/2471) 9 days ago. So closing this out :)
Thanks again and sorry for the trouble!

Hah, that’s pretty funny! Glad to hear the definitions made it in, and thanks for the work and feedback.


Error in nextTick: "ReferenceError: document is not defined"

There is an error poping using this library:
Error in nextTick: "ReferenceError: document is not defined"
document is not defined

Which refere to this line of code https://github.com/eolant/vuetify-toast-snackbar/blob/master/src/index.js#L10
should we check if document is defined before? as it works, it's just an annoying warning

Thanks for posting this. Can you please check doc example and see if you have the same issue there?
document is a global variable and should be available there in the browser context.
Can you please provide more details on your setup and libraries and versions used as well as the browser with version where you see the error.

Closing due to lack of activity.

This happens during nuxt's server rendering.
Following workaround resolved my issue,
if (process.client) {
this.$toast('Default toast');
}

@mrcndn thanks. Also If using Nuxt it's possible to specify only client plugin.


Rendering issue with JEI in the wolf inventory

Reported by NajitoKomadreja on Curse Forge
Issue Description
My problem is that when I open the interface of a wolf the interface of the JEI appears transparent and this does not happen with interfaces of other mods (I do not speak English)
Reproduction Steps
Steps to reproduce the behavior:

Open JEI and wolf inventory at the same time

Expected Behavior
JEI will render as it normally does (not transparent
Additional Context
satyrnidae: This is probably caused by an unpopped matrix or something in the gui render code.

aha, the gui does not blur the background, causing JEI tio look transparent.  This should be easily fixable.


Force framework to NOT split text

Is there a way we can force TPPDF to not split a text element if it does not fit on the same page? But instead having it drawing the whole text on the next page?
I have a use case, where I want to add a text element at the end of the content - and I want TPPDF to keep the whole text together. If it doesn't fit on the first page - TPPDF should simply draw the whole text on the second page.

So we should add a flag which inidicates TPPDF not to break the text. If the text is moved to a new page, and doesn't fit there either (longer than one page), should it break then?

Year, I think its a good idea to break if its longer than one page - but keep it together otherwise.

I think we need to implement this logic in here. Can you take a look at it? I do not have the time right now to work on issues


GATEWAY TIME OUT

Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:

Go to (https://testnet.movementlabs.xyz/)
Click on 'Connect galxe'
redirects to 'Galxe webpage for authorization'
Click Authorize
Gateway timeout

Expected behavior
seamless connection between galxe and movement
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

Browser [chrome,]
Version 129.0.6668.70 (Official Build) (64-bit)

When I try to authorise my galxe it takes me to this error page

The Intermittent outage has been resolved. Please report parthenon issues in Discord. This repo is for the Movement network protocol.


Warning: Device limits tile size to 22 (setting was 32)

Everytime I run an OpenCL-compiled version of a Futhark program, I get the following message:
Warning: Device limits tile size to 22 (setting was 32)

I'm not sure if this is a problem or can safely be ignored. What does this mean?
I see this on three different computers running Linux. I see it using the Beignet driver for Intel, but I'm fairly confident I saw the same message on the proprietary Nvidia driver (though I'm not 100% on that).

During initialisation, the Futhark-generated code sets various constants that control parameters of execution.  These constants have compile-time defaults, but the defaults may exceed the capacity of the device found at runtime.  In this case, the constant is the "tile size" used for two-dimensional block tiling.  The default is 32, which works fine if the device support OpenCL workgroups with 32*32=1024 threads, but e.g. Intel GPUs only support groups of at most 512 threads.  Thus, after quering for limitations, the code recomputes a new tile size as the square root of the maximum group size; 22 in your case.  This is a bit of a weird number, and probably it ought to be a power of two, but it works fine.
I'd be surprised if you saw this warning on an NVIDIA GPU, although there's in principle nothing that prevents NVIDIA from creating a device or driver with a different group size restriction than 1024.
Perhaps a warning need not be issued for sizes that are still set to their compile-time default.  But this whole size business needs a rethought anyway - they are basically tuning parameters that cannot be optimally determined without knowing what hardware we are running on.

I'll verify on the Nvidia when I get home. I might have been running on the Intel HD by accident, I'm sure it's just a false memory.


Error and warning handling in SFCs

Is there any way of handling errors or warnings in the SFCs, or at least detect if an error has occurred at run-time?
If a do a major no-no like {{null.null}} it will raise a high-level Error that bubbles up to the $root .errorHandler event, and is also captured by any errorCaptured handlers, so that can sort of be dealt with.
However if I pass in a template that has something like  {{ "".toUpperCase] }} the javascript console (in development mode only!) will print: a [Vue warn]: Error compiling template: and  invalid expression: missing ) after argument list in, and the entire template fails to render anything at all.
It does return a VueComponent object, but I'm not sure if there's anything in it I can check for if things went sour or not..
ref:

https://vuejs.org/v2/api/#errorHandler
https://vuejs.org/v2/api/#warnHandler (not available in production mode)
https://vuejs.org/v2/api/#errorCaptured

I am not sure compile-time errors may be bail.


Remove admin submitted claim CSV download

 Remove CSV download

Confirming, but I believe so.

Confirmed. @robbiepaul could you take another look?


Add missing definitions

hasMedicalHistory

hasLifeStage : doesn't exist yet

hasDevelopmentalStage

isCauseOfDeath
isContributorToDeath

hasPlatform: doesn't exist yet
hasPlatformID: doesn't exist yet

hasChromosome


Front end "Services" page

Mockup for home page
Services page

Mockup for home page (card)https://drive.google.com/drive/u/1/folders/1pfXNnGoNoIZH4qTF6zvACIcxEY_R3i5P
Card links to Services page here

I think having a card on the home page will help direct users to the "Services" page


Updated whole Blog page

@PriyaGhosal please review and merge it ( i have made it same as theme )
before

after

@samarth-6 please review and merge it

@PriyaGhosal fixed

@AsifQamar Please resolve this and open a new PR for review.
Thank you!

@samarth-6 Please see preview before merging

@AsifQamar Please resolve this and open a new PR for review. Thank you!

Ok


Fix "has_input(SPCC622.08c (PR:000027564))" in GAF file

The new GAF output code doesn't handle PRO IDs in extensions correctly.


Heat Map / Marker-Cluster

I'm submitting a ...  (check one with "x")
[ ] question
[ ] any problem or bug report
[ X] feature request
plugin version:  (check one with "x")
[ ] 1.4.x
[ X] 2.0.0-beta2
Expected behavior:

Is it possible to get support for MarkerCluster as described in https://developers.google.com/maps/documentation/javascript/marker-clustering ?

If your problem is solved, please consider small amount donation to this project.
Appreciate for your kindness.
https://github.com/mapsplugin/cordova-plugin-googlemaps-doc/blob/master/README.md#buy-me-a-beer

Answer for heatmap, please read this thread.
https://github.com/mapsplugin/cordova-plugin-googlemaps/issues/1518
Answer for marker clustering, please read this thread.
https://github.com/mapsplugin/cordova-plugin-googlemaps/issues/1006


Extract filter store from the search store and convert it to Pinia

Fixes

Related to #1037 by @obulat
Description

Search store was responsible for the state of filters, API query parameters, and conversions between them.
This PR extracts the state of filters to a separate store, and converts it to Pinia. The initial filter creation was extracted to /constants/filters.js in #1038.
The filter store actions are mostly used by the searchStore, and components use its getters.
Testing Instructions

Run the app and see that search and filtering works as usual for all search type, both on the client side and on the server-rendered pages (do a hard page reload, or go directly to an address with a filter checked, such as http://localhost:8443/search/audio/?q=sound&license=cc0&mature, try going back using the browser back button.
Note: It won't be possible to test mature filter using the UI until #542 is fixed.
Checklist

[ ] My pull request has a descriptive title (not a vague title like Update index.md).
[ ] My pull request targets the default branch of the repository (main) or a parent feature branch.
[ ] My commit messages follow best practices.
[ ] My code follows the established code style of the repository.
[ ] I added or updated tests for the changes I made (if applicable).
[ ] I added or updated documentation (if applicable).
[ ] I tried running the project locally and verified that there are no visible errors.

Developer Certificate of Origin

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

Everything seems to work well except for some issues between applied filters:
When I apply the "Use commercially" filter then the Science Museum – UK source filter is disabled.
Similarly, when I apply the "Modify or adapt" filter several Source filters are disabled, for example, the Cleveland Museum of Art, which has some CC0 images so this should not happen.

Thank you for spotting this, @krysal ! I haven't even looked there :) This was caused by the isFilterDisabled working for all filterTypes. "Use commercially" was checking for the string "nc" in the filter code, so sciencemuseum got disabled! I changed the function back so that it only checks the licenses and licenseTypes filters.


Jenayah: Watch jagallaptop.com

Jenayah requests the watch of the watch_keyword jagallaptop\.com. See the MS search here and the Stack Exchange search here.

jagallaptop\.com has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by Makyen in Charcoal HQ


Release Google.Cloud.ServiceControl.V1 version 1.2.0

Changes in this release:

Commit 574bc96:

feat: Added the gRPC service config for the Service Controller v1 API
docs: Updated some comments.

Created release for Google.Cloud.ServiceControl.V1-1.2.0


harminv: fix wrong hash (copy of libgdsii)

Description of changes
Fixes wrong hash of the harminv package, which was a copy of the libGDSII hash by accident.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

There seems to be a build problem.

There seems to be a build problem.

Hu, that was why --enable-maintainer-mode is required :smile: I will enable it again. Was apparently required in the first place.


[WIP] realm: Remove Google Hangouts integration.

Google has removed the Google Hangouts brand, thus we are removing them as video chat provider options.
This commit removes Google Hangouts integration and makes a migration
that sets all realms that are using Hangouts as their video chat
provider to the default, jitsi.
Fixes: #15298
The endpoint still needs to be documented.

You missed a few things to delete:

MAX_GOOGLE_HANGOUTS_DOMAIN_LENGTH
The section in templates/zerver/help/start-a-call

(All found via git grep -i hangouts)
Fixed and merged as ddbde66af56d32fa4a4b7fc01c54e0e88d88982c, with a few improvements to the general start-a-call documentation integrated.
Thanks @clarammdantas!


Throw errors.JellyfishDatabaseTimeoutError on all timeouts

Patch db options usage.
Change-type: patch
Signed-off-by: Ramiro Gonzalez<EMAIL_ADDRESS>

@balena-ci I self-certify!


Numbers as dynamic length strings in wire protocol

Is this the only option or is there a way to force a binary float32 / float64 / int64 type?

@alippai I don't quite understand your question. Would you provide some examples?

Sure.
The floats are coming as string over the wire:
https://github.com/vertica/vertica-python/blob/master/vertica_python/vertica/column.py#L143
Is there a way to change the protocol, so fixed width data rows would be possible?

The wire protocol is used for the delivery of what is stored in database to the client, it does not modify the data. So a float data comes as a string representation of its value and some metadata (e.g. datatype).
So I think you have to modify your program to process the fetched data to fixed width, or change your data to be stored as fixed width type like CHAR(n), or make a customization of vertica-python.

I'm pretty confident it's stored as float64 and int64 server-side and serialization into string (instead of bytes) happens.
https://www.vertica.com/docs/9.2.x/HTML/Content/Authoring/SQLReferenceManual/DataTypes/SQLDataTypes.htm
Floats supposed to be represented as "Signed 64-bit IEEE floating point number, requiring 8 bytes of storage", this is how it should be sent - as a 8 byte value. Why converting it to string and back? It's already fixed-width.
Let me know if there is a reference saying that numbers are stored as ASCII strings in Vertica.

or make a customization of vertica-python.

This was my intention, I'm happy to make the PR if there is a different protocol version (eg. for use with different drivers like ODBC or JDBC) which supports numbers as binary instead of string type.
Otherways I'm happy to wait for an inevitable Apache Arrow support :P

The protocol support two operating modes -- string & binary -- but currently vertica-python only implements the string protocol. While postgres lets you change it on a per-column/type basis in the protocol, Vertica currently does not honor that and instead does it as a session-level setting. So unfortunately that means its an all-or-nothing implementation - all types must be implemented, both for reads (queries) and for writes (server-prepared statement bind values) in order for it to be viable.
That makes it a reasonably large feature, but now that all the protocols for core functionality are implemented, it would be a good project to have on the roadmap. Converting binary format to & from python objects is a bit more complicated than strings, but it should have a good performance benefit, especially when you're dealing with lots of numeric data.
To opt into that, you'd have to change the startup message (https://github.com/vertica/vertica-python/blob/master/vertica_python/vertica/messages/frontend_messages/startup.py#L73) to include 'binary_data_protocol' set to '1' (0 is the default, which is the string protocol), then implement a bunch of new serializers & deserializers. The binary encoding of a given type is the same as what's described in the NATIVE file format, just without the headers and other stuff (https://www.vertica.com/docs/10.0.x/HTML/Content/Authoring/AdministratorsGuide/BinaryFilesAppendix/ColumnDefinitions.htm).

Thanks, this is a perfect answer!

A quick experiment:

started with 3 million rows, one float column, 36+ seconds to read it into a numpy array
adding a self.socket_as_file = socket.makefile('rb', 10*1024*1024) and modifying Connection.read_bytes(n) to use it instantly adds 30-50% boost, it's down to 21-23seconds
setting the binary_data_protocol: '1' saves 20% network bandwidth (51MB instead of 70MB on my sample), reading the result simply with Connection.socket_as_file.read() is faster too (0.7 seconds as binary vs 3-4 seconds as string)
changing to struct.unpack('>d', s) from float(s) is not really noticeable, because parsing the whole message in python is simply slow, still 20+ seconds
using numpy.recarray(shape=(num_rows,), dtype=[header, size, field_count, field_size, value], buf) on the raw buffer makes it superb, it's around 0.8s in total (starting from cursor.execute() to the finished numpy array)

tl;dr it's possible to make it cca 50x faster
The only drawback at this point is that I have to know the result set size (number of rows) to feed the numpy shape with it. @tomwall is there a way to query the result size before it arrives? (I don't see it in the rows description, but this might be an oversight)

is there any reason why this issue is closed?
sounds like a well needed possibility to boost performance
(@sitingren @tomwall)

hey @alippai,
do you know if there are any plans to support one of these in this library?

No, I have zero information regarding any of this. I usually simply use the Turbodbc driver or the COPY protocol (also the built-in Parquet support is really nice) when fetching / pushing a lot of data and this lib when just moving data within Vertica. The Vertica ODBC driver is pretty good.

@alippai thanks a lot, will check that out!

@alippai
what about this optimization?
it seems most of the time is spend in read_bytes, were your modifications only relevant to the binary protocol?

adding a self.socket_as_file = socket.makefile('rb',<PHONE_NUMBER>) and modifying Connection.read_bytes(n) to use it instantly adds 30-50% boost, it's down to 21-23seconds

@alonme There is a plan to implement and release binary_data_protocol: '1' within next few months.


Challenge completed level: basic

site: https://rejection.manos.now.sh/
repo: https://github.com/mfountoulakis/rejection
This was a great exercise! You left a trail of easter eggs leading right to some really nice tips about testing the redux store.
I could have submitted this exercise a lot earlier if I hadn't run into issues with testing. Specifically, unit testing connected components. The short answer is: I don't know what I'm doing.
mapStateToProps seems to be clashing with the simulated state that I'm attempting to pass to my components as props. I suspect that there's an easy solution that I'm missing here, but I will continue to write tests in other areas in the meantime, along with general refactoring.
As always, questions/comments are welcome!

Just a note on the previous unit-testing issue. Turns out I needed to export my components separately from the Redux store. My component files now contain two exports:

The component itself.
The component connected to the Redux store.


Add tvos support in podspec

Summary
Adds tvos support in podspec
Motivation -
This library is being used as a dependency in react-navigation@next and I am developing an app for tvos that will require tvos support.

Test Plan

What's required for testing (prerequisites)?
What are the steps to reproduce (after prerequisites)?
Compatibility

OS
Implemented

iOS
✅❌

Android
✅❌

Checklist

[ ] I have tested this on a device and a simulator
[ ] I added the documentation in README.md
[ ] I mentioned this change in CHANGELOG.md
[ ] I updated the typed files (TS and Flow)
[ ] I added a sample use of the API in the example project (example/App.js)

FYI, since CI is throwing flow check errors, I want to mention that there is no need for a flow check in this PR as only podspec file has changed.


Automator: update istio/api@master dependency in istio/istio@master

Generated by Automator - 2020-09-11T14:32:57+00:00

@istio-testing: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

unit-tests_istio
eb2bdb5612d96663c9ecec095a6466c40f35a7d0
link
/test unit-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@istio-testing: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

unit-tests_istio
eb2bdb5612d96663c9ecec095a6466c40f35a7d0
link
/test unit-tests_istio

integ-pilot-k8s-tests_istio
eb2bdb5612d96663c9ecec095a6466c40f35a7d0
link
/test integ-pilot-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@istio-testing: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-pilot-k8s-tests_istio
eb2bdb5612d96663c9ecec095a6466c40f35a7d0
link
/test integ-pilot-k8s-tests_istio

unit-tests_istio
d21177eabd5c45ada7e5397792492807600cf852
link
/test unit-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


libiscsi: Update Dockerfile (#40).

What this PR does / why we need it:  Removes extra RUN statement from the Dockerfile (best practices).
Which issue this PR fixes: #40.
Special notes for your reviewer: N/A

Thanks for the PR @clickyotomy !


[BUG] SfOverlay not transitioning

Describe the bug
The SfOverlay component doesn't transition in/out even with the default fade transition prop.
Not sure if related to #489
How to reproduce
Steps to reproduce the behavior:

Go to https://storybook.storefrontui.io/?path=/story/atoms-overlay--common
Toggle visible prop
See overlay suddenly appear/disappear instead of fading in/out.

Expected behavior
Overlay should fade in or out by default.
Actual behavior
Overlay suddenly appears or disappears.
Devices (please complete the following information):

Device: All
OS: All
Browser: All

I can have a look at fixing this if it's an issue, not sure if it's intended.

I'd like to have a look at it 👁

Did a fix for this if you want to check my PR.


Chroma/NoodleExtentions/Heck not on ModAssistant

Greetings! I'm unsure what the best method of contact is, but I do not see any of these mods in ModAssistant, despite reading this is the preferred method of installing them.  I apologize for my lack of understanding, I only installed Beat Saber yesterday.

just wait

just wait

What do I wait for?

Noodle extensions, Chroma, and Heck are not yet updated for the latest version of Beat Saber, so they aren't available in ModAssistant yet. When they are ready, it will likely be added.

you can also check on BeatMods

Noodle extensions, Chroma, and Heck are not yet updated for the latest version of Beat Saber, so they aren't available in ModAssistant yet. When they are ready, it will likely be added.

Ah gotcha. Thank you for the straight reply!


unrecognized label

I have a bunch of images with specimen, label and (grotty) scale bar, and inselect doesn't recognize the label as such, just individual words on it (see attached image). How can I get inselect to put a box around the entire label?

I'm surprised it didn't get the whole label, but seeing as there are only three objects in the image it might be easier to manually draw the box around each (right click and drag with mouse). Hope that helps! Ben


[SPARK-6275][Documentation]Miss toDF() function in docs/sql-programming-guide.md

Miss toDF() function in docs/sql-programming-guide.md

Can one of the admins verify this patch?

LGTM. Github sync is behind right now so I cannot merge. I will merge as soon as I can. Thanks!

OK.:)

I've pushed this change to the website.
http://spark.apache.org/docs/latest/sql-programming-guide.html


Optimize the Dockerfiles

Change Description
This PR optimizes the Dockerfiles so that the final build image size is reduced. The big size is mostly caused by the en_core_web_lg package is being cached on /root/.cache. So by adding ENV PIP_NO_CACHE_DIR=1, the pip will not cache it anymore, reducing the image size.
I also merge some RUN commands into one RUN, so that it results to 1 layer instead of many layers.
                         BEFORE   AFTER
presidio-analyzer        2.1GB    1.26GB
presidio-anonymizer      212MB    196MB
presidio-image-redactor  2.34GB   1.46GB

Checklist

[x] I have reviewed the contribution guidelines
[x] I have signed the CLA
[ ] My code includes unit tests
[ ] All unit tests and lint checks pass locally
[ ] My PR contains documentation updates / additions if required

Cool thanks! We'll review it soon.

/azp run


Solve example

Improve documentation for solve

:books: Documentation preview :books:: https://equistore--194.org.readthedocs.build/en/194/

Thanks for the work @frostedoyster!


Node.js used by  sfdx-cli/8.89.2 does not support Windows 7 by default

Summary
The recent update which was automatically pushed includes updating Node.js to v14.15.4. Since this version of Node.js does not support Windows 7, it made SFDX unusable for Windows 7 users without some hack.
When running SFDX, Windows 7 users will get the error:
Node.js is only supported on Windows 8.1, Windows Server 2012 R2, or higher. Setting the NODE_SKIP_PLATFORM_CHECK environment variable to 1 skips this check, but Node.js might not execute correctly. Any issues encountered on unsupported platforms will not be fixed.12:30:07.195
We weren't notified that Salesforce CLI will no longer support Windows 7. The documentation from salesforce mentions "Windows—Windows 7 (64-bit and 32-bit) or later" as the System requirements, so we do expect it to still work without us having to modify the sfdx.cmd file.
Steps To Reproduce:

Using a windows 7 machine, install the latest version of Salesforce CLI
Run sfdx -v

Expected result
Command is executed, and the version is returned.
Actual result
Error message given:
Node.js is only supported on Windows 8.1, Windows Server 2012 R2, or higher. Setting the NODE_SKIP_PLATFORM_CHECK environment variable to 1 skips this check, but Node.js might not execute correctly. Any issues encountered on unsupported platforms will not be fixed.12:30:07.195
Additional information
SFDX CLI Version(to find the version of the CLI engine run sfdx --version):
sfdx-cli/7.89.2-d1d2614d02 win32-x64 node-v14.15.4
SFDX plugin Version(to find the version of the CLI plugin run sfdx plugins --core):
@oclif/plugin-autocomplete 0.3.0 (core)
@oclif/plugin-commands 1.3.0 (core)
@oclif/plugin-help 3.2.2 (core)
@oclif/plugin-not-found 1.2.4 (core)
@oclif/plugin-plugins 1.9.5 (core)
@oclif/plugin-update 1.4.0-2 (core)
@oclif/plugin-warn-if-update-available 1.7.0 (core)
@oclif/plugin-which 1.0.3 (core)
@salesforce/sfdx-trust 3.6.0 (core)
alias 1.1.5 (core)
auth 1.4.8 (core)
config 1.2.4 (core)
etcopydata 0.5.7
generator 1.1.5 (core)
salesforcedx 51.1.1 (core)
├─ limits 1.0.4 (core)
├─ schema 1.0.4 (core)
├─ custom-metadata 1.0.11 (core)
├─ templates 51.3.0 (core)
├─ apex 0.1.4 (core)
├─ user 1.1.2 (core)
├─ salesforce-alm 51.1.1 (core)
└─ @salesforce/sfdx-plugin-lwc-test 0.1.7 (core)
sfdmu 3.9.4
sfdx-cli 7.89.2 (core)
telemetry 1.1.1 (core)
OS and version:
Windows 7

We apologize for not updating the documentation and will do so now. Windows 7 is no longer supported by Microsoft. Node will not support a version of Windows that Microsoft doesn't support, which makes it improbable for us to support it since we are built on nodejs. It doesn't mean that node14 won't work on windows 7, it just isn't supported.
You have two options:

Install the Salesforce CLI using NPM on a node version that doesn't have that check.
Set the environment variable (NODE_SKIP_PLATFORM_CHECK) as highlighted above.
Turn off auto-updates (set SFDX_AUTOUPDATE_DISABLE=true globally) and install an old version.

Again, these are not supported options but should unblock you so you can still use the CLI.


Tracking intermediate stages

I don't know if this needs to be part of the odm-benchmarks project, but I have a particularly large dataset to process, so I am doing some monitoring of individual stages so that I can do a better job predicting processing time over the life of the project. I thought I would document that here in case it is useful to see.

Each stage has a last written file. We can use that last written file to understand when the stage completed. It's possible to do something similar for each sub-stage of OpenSfM as well, but for now, we will restrict ourselves to the overall stage times. The following is with a ~7400 image dataset:
ls -lhtr odm_dem/dtm.tif mve/mve_dense_point_cloud.ply odm_filterpoints/point_cloud.ply
odm_georeferencing/odm_georeferenced_model.laz odm_georeferencing_25d/odm_georeferencing_log.txt odm_meshing/odm_25dmesh
.ply odm_meshing/odm_mesh.ply odm_orthophoto/odm_orthophoto.tif odm_texturing/odm_textured_model_geo.mtl odm_texturing_2
5d/odm_textured_model_geo.mtl opensfm/geocoords_transformation.txt odm_report/shots.geojson img_list.txt
-rw-r--r-- 1 useruser useruser 220K Jun 25 16:20 img_list.txt
-rw-r--r-- 1 useruser useruser  192 Jun 27 00:18 opensfm/geocoords_transformation.txt
-rw-r--r-- 1 useruser useruser  49G Jul  3 15:40 mve/mve_dense_point_cloud.ply
-rw-r--r-- 1 useruser useruser  49G Jul  4 02:05 odm_filterpoints/point_cloud.ply
-rw-r--r-- 1 useruser useruser 7.3M Jul  4 02:23 odm_meshing/odm_mesh.ply
-rw-r--r-- 1 useruser useruser 7.3M Jul  4 05:50 odm_meshing/odm_25dmesh.ply
-rw-r--r-- 1 useruser useruser  36K Jul  4 08:13 odm_texturing_25d/odm_textured_model_geo.mtl
-rw-r--r-- 1 useruser useruser 2.3K Jul  4 09:45 odm_georeferencing_25d/odm_georeferencing_log.txt
-rw-r--r-- 1 useruser useruser  36K Jul  4 10:56 odm_texturing/odm_textured_model_geo.mtl
-rw-r--r-- 1 useruser useruser 4.9G Jul  4 12:26 odm_georeferencing/odm_georeferenced_model.laz
-rw-r--r-- 1 useruser useruser 1.8G Jul  4 18:43 odm_dem/dtm.tif
-rw-r--r-- 1 useruser useruser  16G Jul  4 21:27 odm_orthophoto/odm_orthophoto.tif
-rw-r--r-- 1 useruser useruser  845 Jul  4 21:29 odm_report/shots.geojson

If we do a bit of calculations against these dates, we can see the progression:

date
stage
stage length (h)
overall time (d)
stage percentage
stage overall percentage

6/25/2020 16:20
start
0.00
0.00
0.00%
0.00%

6/27/2020 0:18
opensfm
31.97
1.33
14.45%
14.45%

7/3/2020 15:40
mve
159.37
7.97
72.06%
86.52%

7/4/2020 2:05
odm_filterpoints
10.42
8.41
4.71%
91.23%

7/4/2020 2:23
odm_meshing
0.30
8.42
0.14%
91.36%

7/4/2020 5:50
odm_meshing
3.45
8.56
1.56%
92.92%

7/4/2020 8:13
odm_texturing_25d
2.38
8.66
1.08%
94.00%

7/4/2020 9:45
odm_georeferencing_25d
1.53
8.73
0.69%
94.69%

7/4/2020 10:56
odm_texturing
1.18
8.78
0.54%
95.23%

7/4/2020 12:26
odm_georeferencing
1.50
8.84
0.68%
95.91%

7/4/2020 18:43
odm_dem
6.28
9.10
2.84%
98.75%

7/4/2020 21:27
odm_orthophoto
2.73
9.21
1.24%
99.98%

7/4/2020 21:29
odm_report
0.03
9.21
0.02%
100.00%

Honestly, I expected OpenSfM to be a more expensive piece, and props to @pierotofy for bringing down that odm_dem number so much.

Now I am testing with the same 7400 image dataset but resized using https://github.com/pierotofy/exifimageresize and resizing the images first to 1280 and then setting orthophoto and elevation model resolutions pretty low. The plan is to do hydrological modeling on these data and orthos have already been otherwise generating, so getting a good low res DEM is a good starting product, but why waste the memory and processing time on full resolution products:
First, log in to the docker instance:
docker run -ti -v /home/useruser/outdir:/datasets --entrypoint bash odmnolign

Then, we run the run.py script directly. This gives us some flexibility in resuming, or tweaking as we go:
./run.py --project-path /datasets --project-path /datasets datasetname --dtm --dsm --smrf-scalar 0 --smrf-slope 0.06 --smrf-threshold 1.05 --smrf-window 49 --dem-resolution 414 --depthmap-resolution 320 --ignore-gsd --orthophoto-resolution 138

I am going to keep this thread going and expand a bit to include info about how to process monster datasets. This will be messy, and riddled with contradictions, but I will ultimately probably end up as a section in docs.opendronemap.org, at least the how-to bits.

We should probably add a flag for this, but I have removed the align portion of split merge. For massive datasets, the combination of SfM, error distribution, and OpenDroneMap's use of DOP and other accuracy tags seems to make the alignment step not just unnecessary but a source of error. Here's my branch which removes that step in a brutish way... :
https://github.com/smathermather/ODM/tree/noalign

Also, for massive datsets, perhaps the objective (as it is in my case) is to capture elevation models, not orthophotos. So one trick we can do is to resample the data down quite a bit, and then adjust the output resolutions to save on memory usage and processing time. A good tool for that is UAV4Geos  https://github.com/pierotofy/exifimageresize which will resize an entire directory of images but keep the exif the same (aside from the exif on image dimensions).
I resized my dataset to 1280 which should leave enough features for matching and enough for depthmaps as well.

Also, docker is cool, but when things fail (as they often do with big datasets) reconnecting to the docker machine to inspect can be difficult, since the ODM docker image is meant to turn off automatically when done (whether a failure or not). So, instead of the typical docker run, we can instead start the instance and run the run.py command from within the container. This will ensure we can reconnect, poke around, and do all the needful things:
docker run -ti -v /home/gisuser/outdir:/datasets --entrypoint bash odmnolign

Now we are logged in and can run our OpenDroneMap command:
./run.py --project-path /datasets --project-path /datasets resize --dtm --dsm --smrf-scalar 0 --smrf-
slope 0.06 --smrf-threshold 1.05 --smrf-window 49 --dem-resolution 413 --depthmap-resolution 320

If we need to reconnect, we can look for our docker image:
docker ps

And then use that ID to connect
docker exec -it <docker hash or nickname> bash

We can also use docker logs to check in on the process:
docker logs <docker hash or nickname> | tail

For particularly large datasets, like the 80,000+ image dataset I am currently attempting, we need to invoke split-merge by using the --split option and we probably also want to use hybrid bundle adjustment to increase the efficiency of our structure from motion step:
./run.py --project-path /datasets --project-path /datasets resize --dtm --dsm --smrf-scalar 0 --smrf-slope 0.06 --smrf-threshold 1.05 --smrf-window 49 --dem-resolution 413 --depthmap-resolution 320 --split 7000 --use-hybrid-bundle-adjustment

A lot of folks don't necessarily understand the relationship between depthmap resolution and ground sampling resolution, so I should explain:
Suppose we have a 6400x4200 image with a ground sampling distance (pixel size) of 10cm. The maximum depthmap we want to use would be half the linear image resolution at 3200x2100, which could be specified with --depthmap-resolution 3200. This means the maximum elevation resolution should be 1/2 the ground sampling resolution or 20cm. I often aim for 1/4 the resolution as it is a bit more robust to noise, especially on specular reflectors like metal rooftops. In this case, I would set --depthmap-resolution 1600 and --dem-resolution 40.

A nearly useless utility to track feature extraction:
# Find out how many images there are
totallines=`wc -l ~/outdir/znz/resize/img_list.txt | awk '{print $1}'`

while true
do
    # Count the number of files in the OpenSfM features directory
    num1=`find ~/outdir/znz/resize/opensfm/features/ -type f | wc -l `

    # Calculate percentage of images that have had features extracted
    percentage=`echo $num1 / $totallines \* 100| bc -l`
    echo `printf "%.2f\n" $percentage`

    # Speak less. Smile more.
    sleep 20

    # Check if we are done
    if [ `printf "%.0f\n" $percentage` -gt 99 ]
    then
        # Bing!
        echo -ne '\007'
        echo "All done with feature extraction!"
        break
    fi

done

Just wanted to pop in and say I'm glad you're including this info here @smathermather .

Hey thanks! I was hoping I wasn't creating unneeded noise. But, I need to track this, and I figure it can become a platform for further use and understanding.

I have stopped using the above approach of logging into the docker instance and running the script. It seems like enough to just to remove the -rm flag when using docker run, and then it's possible to login to the machine if needed. I have also modified my branch a little more to default to BOW matching in the hopes that I can get faster matching
docker run -ti -v /home/gisuser/outdir/znz:/datasets odmnolign --project-path /datasets resize --dtm --dsm --smrf-scalar 0 --smrf-slope 0.06 --smrf-threshold 1.05 --smrf-window 100 --dem-resolution 413 --depthmap-resolution 320 --split 7000 --use-hybrid-bundle-adjustment

One of the big challenges with massive datasets is catching issues before they are issues. In my case, that is  finding small pockets of bad data. We probably need more robust error checking in the chain. So, for example, I had 4 files with some issue (
they probably didn't fully come back from glacier storage before being transferred). That seems a pretty reasonable error rate, but the challenge is that each restart after finding bad data is a setback of a few days of processing:
2020-07-23 18:04:15,105 DEBUG: Matching deduped2016-10-11_09.23.08.JPG and deduped2016-10-11_09.21.04.JPG.  Matcher: WORDS (symmetric) T-desc: 0.165 T-robust: 0.057 T-total: 0.223 Matches: 447 Robust: 430 Success: True
2020-07-23 18:04:15,120 DEBUG: No segmentation for geoscleanup2016-10-11_09.29.52.JPG, no features masked.
2020-07-23 18:04:15,388 DEBUG: Matching deduped2016-10-11_09.23.08.JPG and geoscleanup2016-10-11_09.29.52.JPG.  Matcher: WORDS (symmetric) T-desc: 0.163 T-robust: 0.105 T-total: 0.269 Matches: 543 Robust: 531 Success: True
Traceback (most recent call last):
  File "/code/SuperBuild/src/opensfm/bin/opensfm", line 34, in <module>
    command.run(args)
  File "/code/SuperBuild/src/opensfm/opensfm/commands/match_features.py", line 29, in run
    pairs_matches, preport = matching.match_images(data, images, images)
  File "/code/SuperBuild/src/opensfm/opensfm/matching.py", line 43, in match_images
    return match_images_with_pairs(data, exifs, ref_images, pairs), preport
  File "/code/SuperBuild/src/opensfm/opensfm/matching.py", line 67, in match_images_with_pairs
    matches = context.parallel_map(match_unwrap_args, args, processes, jobs_per_process)
  File "/code/SuperBuild/src/opensfm/opensfm/context.py", line 41, in parallel_map
    return Parallel(batch_size=batch_size)(delayed(func)(arg) for arg in args)
  File "/usr/local/lib/python2.7/dist-packages/joblib/parallel.py", line 934, in __call__
    self.retrieve()
  File "/usr/local/lib/python2.7/dist-packages/joblib/parallel.py", line 862, in retrieve
    raise exception.unwrap(this_report)
joblib.my_exceptions.JoblibIOError: JoblibIOError
___________________________________________________________________________
...........................................................................
/code/SuperBuild/src/opensfm/bin/opensfm in <module>()
     29 args = parser.parse_args()
     30
     31 # Run the selected subcommand
     32 for command in subcommands:
     33     if args.command == command.name:
---> 34         command.run(args)
...........................................................................
/code/SuperBuild/src/opensfm/opensfm/commands/match_features.py in run(self=<opensfm.commands.match_features.Command instance>, args=Namespace(command='match_features', dataset='/datasets/resize/opensfm'))
     24     def run(self, args):
     25         data = dataset.DataSet(args.dataset)
     26         images = data.images()
     27
     28         start = timer()
---> 29         pairs_matches, preport = matching.match_images(data, images, images)
        pairs_matches = undefined
        preport = undefined
        data = <opensfm.dataset.DataSet object>
        images = [u'DSC00034_2978.jpg', u'DSC00179_4743.jpg', u'DSC00180_4703.jpg', u'DSC00183_4739.jpg', u'DSC00184_4720.jpg', u'DSC00190_4051.jpg', u'DSC00209_4338.jpg', u'DSC00211_4349.jpg', u'DSC00213_4333.jpg', u'DSC00216_4301.jpg', u'DSC00221_3845.jpg', u'DSC00225_3831.jpg', u'DSC00229_4355.jpg', u'DSC00230_3827.jpg', u'DSC00230_4335.jpg', u'DSC00235_4346.jpg', u'DSC00240_4160.jpg', u'DSC00241_3752.jpg', u'DSC00241_4176.jpg', u'DSC00244_4182.jpg', ...]
     30         matching.save_matches(data, images, pairs_matches)
     31         end = timer()
     32
     33         with open(data.profile_log(), 'a') as fout:
...........................................................................
/code/SuperBuild/src/opensfm/opensfm/matching.py in match_images(data=<opensfm.dataset.DataSet object>, ref_images=[u'DSC00034_2978.jpg', u'DSC00179_4743.jpg', u'DSC00180_4703.jpg', u'DSC00183_4739.jpg', u'DSC00184_4720.jpg', u'DSC00190_4051.jpg', u'DSC00209_4338.jpg', u'DSC00211_4349.jpg', u'DSC00213_4333.jpg', u'DSC00216_4301.jpg', u'DSC00221_3845.jpg', u'DSC00225_3831.jpg', u'DSC00229_4355.jpg', u'DSC00230_3827.jpg', u'DSC00230_4335.jpg', u'DSC00235_4346.jpg', u'DSC00240_4160.jpg', u'DSC00241_3752.jpg', u'DSC00241_4176.jpg', u'DSC00244_4182.jpg', ...], cand_images=[u'DSC00034_2978.jpg', u'DSC00179_4743.jpg', u'DSC00180_4703.jpg', u'DSC00183_4739.jpg', u'DSC00184_4720.jpg', u'DSC00190_4051.jpg', u'DSC00209_4338.jpg', u'DSC00211_4349.jpg', u'DSC00213_4333.jpg', u'DSC00216_4301.jpg', u'DSC00221_3845.jpg', u'DSC00225_3831.jpg', u'DSC00229_4355.jpg', u'DSC00230_3827.jpg', u'DSC00230_4335.jpg', u'DSC00235_4346.jpg', u'DSC00240_4160.jpg', u'DSC00241_3752.jpg', u'DSC00241_4176.jpg', u'DSC00244_4182.jpg', ...])
     38     # Generate pairs for matching
     39     pairs, preport = pairs_selection.match_candidates_from_metadata(
     40         ref_images, cand_images, exifs, data)
     41
     42     # Match them !
---> 43     return match_images_with_pairs(data, exifs, ref_images, pairs), preport
        data = <opensfm.dataset.DataSet object>
        exifs = {u'DSC00034_2978.jpg': {u'band_name': u'RGB', u'camera': u'v2 sony dsc-wx220 1280 960 brown 0 rgb', u'capture_time':<PHONE_NUMBER>.0, u'focal_ratio': 0, u'gps': {u'altitude': 74.435, u'latitude': -6.050773555555556, u'longitude': 39.21183877777778}, u'height': 960, u'make': u'SONY', u'model': u'DSC-WX220', u'orientation': 1, u'projection_type': u'brown', ...}, u'DSC00179_4743.jpg': {u'band_name': u'RGB', u'camera': u'v2 sony dsc-wx220 1280 960 brown 0 rgb', u'capture_time':<PHONE_NUMBER>.0, u'focal_ratio': 0, u'gps': {u'altitude': 88.705, u'latitude': -6.061974111111111, u'longitude': 39.207787083333336}, u'height': 960, u'make': u'SONY', u'model': u'DSC-WX220', u'orientation': 1, u'projection_type': u'brown', ...}, u'DSC00180_4703.jpg': {u'band_name': u'RGB', u'camera': u'v2 sony dsc-wx220 1280 960 brown 0 rgb', u'capture_time':<PHONE_NUMBER>.0, u'focal_ratio': 0, u'gps': {u'altitude': 86.979, u'latitude': -6.061924805555555, u'longitude': 39.20743983333334}, u'height': 960, u'make': u'SONY', u'model': u'DSC-WX220', u'orientation': 1, u'projection_type': u'brown', ...}, u'DSC00183_4739.jpg': {u'band_name': u'RGB', u'camera': u'v2 sony dsc-wx220 1280 960 brown 0 rgb', u'capture_time':<PHONE_NUMBER>.0, u'focal_ratio': 0, u'gps': {u'altitude': 87.518, u'latitude': -6.06194775, u'longitude': 39.20635772222222}, u'height': 960, u'make': u'SONY', u'model': u'DSC-WX220', u'orientation': 1, u'projection_type': u'brown', ...}, u'DSC00184_4720.jpg': {u'band_name': u'RGB', u'camera': u'v2 sony dsc-wx220 1280 960 brown 0 rgb', u'capture_time':<PHONE_NUMBER>.0, u'focal_ratio': 0, u'gps': {u'altitude': 87.861, u'latitude': -6.061936972222222, u'longitude': 39.205986055555556}, u'height': 960, u'make': u'SONY', u'model': u'DSC-WX220', u'orientation': 1, u'projection_type': u'brown', ...}, u'DSC00190_4051.jpg': {u'band_name': u'RGB', u'camera': u'v2 sony dsc-wx220 1280 960 brown 0 rgb', u'capture_time':<PHONE_NUMBER>.0, u'focal_ratio': 0, u'gps': {u'altitude': 79.314, u'latitude': -6.059792916666667, u'longitude': 39.21130225}, u'height': 960, u'make': u'SONY', u'model': u'DSC-WX220', u'orientation': 1, u'projection_type': u'brown', ...}, u'DSC00209_4338.jpg': {u'band_name': u'RGB', u'camera': u'v2 sony dsc-wx220 1280 960 brown 0 rgb', u'capture_time':<PHONE_NUMBER>.0, u'focal_ratio': 0, u'gps': {u'altitude': 88.492, u'latitude': -6.060617416666666, u'longitude': 39.21422186111111}, u'height': 960, u'make': u'SONY', u'model': u'DSC-WX220', u'orientation': 1, u'projection_type': u'brown', ...}, u'DSC00211_4349.jpg': {u'band_name': u'RGB', u'camera': u'v2 sony dsc-wx220 1280 960 brown 0 rgb', u'capture_time':<PHONE_NUMBER>.0, u'focal_ratio': 0, u'gps': {u'altitude': 89.117, u'latitude': -6.060633027777778, u'longitude': 39.21359697222223}, u'height': 960, u'make': u'SONY', u'model': u'DSC-WX220', u'orientation': 1, u'projection_type': u'brown', ...}, u'DSC00213_4333.jpg': {u'band_name': u'RGB', u'camera': u'v2 sony dsc-wx220 1280 960 brown 0 rgb', u'capture_time':<PHONE_NUMBER>.0, u'focal_ratio': 0, u'gps': {u'altitude': 87.932, u'latitude': -6.060612166666666, u'longitude': 39.21292638888889}, u'height': 960, u'make': u'SONY', u'model': u'DSC-WX220', u'orientation': 1, u'projection_type': u'brown', ...}, u'DSC00216_4301.jpg': {u'band_name': u'RGB', u'camera': u'v2 sony dsc-wx220 1280 960 brown 0 rgb', u'capture_time':<PHONE_NUMBER>.0, u'focal_ratio': 0, u'gps': {u'altitude': 86.411, u'latitude': -6.060592916666667, u'longitude': 39.212014583333335}, u'height': 960, u'make': u'SONY', u'model': u'DSC-WX220', u'orientation': 1, u'projection_type': u'brown', ...}, ...}
        ref_images = [u'DSC00034_2978.jpg', u'DSC00179_4743.jpg', u'DSC00180_4703.jpg', u'DSC00183_4739.jpg', u'DSC00184_4720.jpg', u'DSC00190_4051.jpg', u'DSC00209_4338.jpg', u'DSC00211_4349.jpg', u'DSC00213_4333.jpg', u'DSC00216_4301.jpg', u'DSC00221_3845.jpg', u'DSC00225_3831.jpg', u'DSC00229_4355.jpg', u'DSC00230_3827.jpg', u'DSC00230_4335.jpg', u'DSC00235_4346.jpg', u'DSC00240_4160.jpg', u'DSC00241_3752.jpg', u'DSC00241_4176.jpg', u'DSC00244_4182.jpg', ...]
        pairs = [(u'deduped2016-08-02_14.23.18.JPG', u'all2017-05-23_11.29.11.JPG'), (u'deduped2017-02-10_08.52.13.JPG', u'deduped2017-02-10_08.51.39.JPG'), (u'deduped2016-09-08_12.17.18.JPG', u'deduped2016-09-08_14.10.05.JPG'), (u'deduped2000-01-01_00.47.13.JPG', u'geoscleanup2000-01-01_00.47.24.JPG'), (u'deduped2016-09-07_10.42.44.JPG', u'deduped2016-09-07_09.33.17.JPG'), (u'deduped2016-09-27_09.01.55.JPG', u'deduped2016-09-27_09.02.02.JPG'), (u'deduped2016-08-31_09.57.52.JPG', u'deduped2016-08-31_09.59.03.JPG'), (u'geoscleanup2000-01-01_00.23.35.JPG', u'geoscleanup2000-01-01_00.24.35.JPG'), (u'deduped2016-09-21_11.45.01.JPG', u'deduped2016-08-31_09.12.46.JPG'), (u'deduped2017-06-16_08.50.19.JPG', u'deduped2017-06-16_08.50.25.JPG'), (u'deduped2017-06-17_09.04.08.JPG', u'deduped2017-06-17_09.04.03.JPG'), (u'deduped2017-06-24_09.36.32.JPG', u'deduped2017-06-24_09.18.45.JPG'), (u'deduped2016-09-01_09.52.58.JPG', u'deduped2016-09-01_10.10.46.JPG'), (u'deduped2016-09-02_08.38.28.JPG', u'deduped2016-09-02_13.37.31.JPG'), (u'deduped2016-08-03_14.58.03.JPG', u'geoscleanup2016-07-30_14.22.00.JPG'), (u'deduped2016-09-06_11.11.15.jpg', u'deduped2016-09-06_11.11.11.jpg'), (u'deduped2016-09-21_09.43.26.JPG', u'deduped2016-09-21_09.43.31.JPG'), (u'deduped2016-09-01_09.58.39.JPG', u'deduped2016-09-01_09.58.43.JPG'), (u'deduped2016-09-07_10.06.38.jpg', u'deduped2016-09-07_09.32.13.jpg'), (u'deduped2016-09-07_09.3.

Now we are in the matcher stage for these data. As noted above, I forced barrel of words matching to get a speed bump. So, how do we track progress here? We can look at the logs and see how many matches in we are:
docker logs great_golick | grep "Matching" | wc -l
102155

But that just gives us some relative progress. How many matches should we have? The total number of images helps inform this:
wc -l ~/outdir/znz/resize/img_list.txt | awk '{print $1}'
82087

We can assume the total number of matches will be the number of images 82087 times the number of matcher images default: 20 for 1,641,740.
Oof. We've got a way to go.

We can wrap the above in a monitoring script which displays a number each time we progress by 1/10th of a percentage:
#!/bin/bash
# Find out how many images there are
totallines=`wc -l ~/outdir/znz/resize/img_list.txt | awk '{print $1}'`

previouspermillage=0
while true
do
    # Count the number of matches
    num1=`docker logs great_golick | grep "Matching" | wc -l`

    # Calculate percentage of images that have had features extracted
    percentage=`echo $num1 / $totallines / 20 \* 100| bc -l`

    # We calculate permillage for the sake of determining whether we are
    # going to display a new number
    permillage=`echo $num1 / $totallines / 20 \* 1000| bc -l`

    # If our rounded permillage number has increased, then display a new number
    if [ `printf "%.0f\n" $permillage` -gt `printf "%.0f\n" $previouspermillage` ]
    then
#        echo
        echo -n `printf "%.1f\n" $percentage`
    fi

    # Speak less. Smile more.
	echo -n "."
    sleep 5

    # Check if we are done
    if [ `printf "%.0f\n" $percentage` -gt 99 ]
    then
        # Bing!
        echo -ne '\007'
        echo "All done with matching!"
        break
    fi

previouspermillage=$permillage

done

👆🏻 This doesn't really work, btw. I don't know how to estimate how many matches we are supposed to have.

Buying instead of Renting
Usually if we do split-merge, we are renting some infrastructure, and the autoscaler in ClusterODM is the way to go. It's a fantastic tool. But sometimes, we have infrastructure we own, so I will document a bit of the management of owned infrastructure here. It will eventually make it's way into docs.opendronemap.org as it matures, but I want to capture it here while it's fresh.

Parallel shell use:
parallel-ssh is what I am using to manage all the vms that I have across our infrastructure. I have 3 sizes of machine. I have one monstrous 20-core, 768GB machine; a single 12-core, 192GB machine that is no slouch, and 4 x 96GB, 8 core machines as well. So, they need a bit of different care and feeding in use.
Connecting to hosts
Logins to the machines are handled in my .ssh/config:
Host proxyhost
        User useruser
        Hostname proxyhost.someaddress.com
        Port 22

Host webodm
        User loseruser
        Hostname <IP_ADDRESS>
        Port 22
        ProxyCommand ssh -q -W %h:%p proxyhost

Host node1
        User winorloser
        Hostname <IP_ADDRESS>
        Port 22
        ProxyCommand ssh -q -W %h:%p proxyhost

Host node2
        User bob
        Hostname <IP_ADDRESS>
        Port 22
        ProxyCommand ssh -q -W %h:%p proxyhost
....

### Clusters of hosts

Clusters of hosts which need controlled together are than contained in hosts files as follows:

First all the machines for when we want to run commands on all:

```BASH
more .ssh/odmall
webodm
node1
node2
node3
node4
node5

Then the groups of hosts that we want to control. For simplicity, I do have groups of one, just to simplify my work flow. Alternatively, I could use just ssh for these, instead of parallel-ssh.
The parent host file contains just one machine:
more .ssh/odmrent
webodm

The big machine is in a host file all it's own:
more .ssh/odmbigs
node5

All the smaller machines are together in a file:
more  .ssh/odmsmoll
node1
node2
node3
node4

And finally, we have a hosts file with just the child nodes:
more  .ssh/odmhosts
node1
node2
node3
node4
node5

Stitching the cluster together
Now we need a ClusterODM instance to use these nodes. It would be really easy to do this as a docker instance, but then we can't use our host names so easily there, and ClusterODM is easy enough to run locally, so we'll do that. First we add them to our hosts file:
sudo vi /etc/hosts

<IP_ADDRESS>       localhost.localdomain   localhost
::1             localhost6.localdomain6 localhost6

<IP_ADDRESS>       node1
<IP_ADDRESS>       node2
<IP_ADDRESS>       node3
<IP_ADDRESS>       node4
<IP_ADDRESS>       node5

# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

First we get and install:
git clone https://github.com/OpenDroneMap/ClusterODM
cd ClusterODM
npm install

Then we run our ClusterODM node, in this case with a limit on network speed to avoid packet losses. It'll still do things at pretty fast:
node index.js --public-address http://<IP_ADDRESS>:3000 --upload-max-speed 200000000 &

ClusterODM, needs the nodes attached. For this we telnet in:
telnet localhost 8080
Trying ::1...
Connected to localhost.localdomain.
Escape character is '^]'.
Welcome ::1:33106 ClusterODM:1.4.3
HELP for help
QUIT to quit
#>

And we need to add our hosts:
#> NODE ADD webodm 3001
OK
#> NODE ADD node1 3000
OK
#> NODE ADD node2 3000
OK
#> NODE ADD node3 3000
OK
#> NODE ADD node4 3000
OK
#> NODE ADD node5 3000
OK

We can list our available hosts:
#> NODE LIST
1) webodm:3001 [online] [0/1] <version 1.6.1>
2) node1:3000 [online] [0/1] <version 1.6.1>
3) node2:3000 [online] [0/1] <version 1.6.1>
4) node3:3000 [online] [0/1] <version 1.6.1>
5) node4:3000 [online] [0/1] <version 1.6.1>
6) node5:3000 [online] [0/1] <version 1.6.1>

And our settings when we started these nodes will help the load balancing of any split merge work. We can check this with NODE BEST <integer>, e.g.:
#> NODE BEST 5000
1) node1:3000 [online] [0/1] <version 1.6.1>
#> NODE BEST 9000
1) node5:3000 [online] [0/1] <version 1.6.1>
#> NODE BEST 50000
1) webodm:3001 [online] [0/1] <version 1.6.1>


Predicted b/y-ions with ammonia or water loss

Hi,
I am currently comparing my confident experimental spectra with a corresponding predicted spectra from Prosit, for the same precursor. I noticed that the spectral library that I downloaded from the web server, does not provide y/b-ions with ammonia -or water losses. I just wonder if it would be possible to include these ions in the prediction, because it would definitely improve the similarity between experimental and predicted spectra.
I think the tool is amazing, but I just want to help you to make it even better :).
Best,
Marc

Hi Marc,
first of all thank you for giving Prosit a try and providing feedback!
Including neutral losses is on our list we want to do, but unfortunately a set of dependencies come with it. This is why I unfortunately cannot give you an estimate when we will have them available.
You might be able to find an additional experimental spectrum that can help with yours at proteomicsdb.org. I am sorry that we cannot help you with the neutral losses in Prosit at this time!
Best,
Sigi


Cross server gt

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Hi @adumitr, I'm your friendly neighborhood Azure Pull Request Bot (You can call me AZPRBOT). Thanks for your contribution!

It looks like you're working at Microsoft (adumitr). If you're full-time, we DON'T require a contribution license agreement.

If you are a vendor, DO please sign the electronic contribution license agreement. It will take 2 minutes and there's no faxing! https://cla.azure.com.

TTYL, AZPRBOT;

How long does it usually take for changes to get pulled in? Having to wait so long is eating into my deadline


Request for new project : Toxicity Grading Model using TensorFlow and Gradio

The project repository encompasses code for a comprehensive toxicity grading model, employing TensorFlow and integrating Gradio for seamless testing. The model undergoes training to ascertain the level of toxicity in comments, providing insights into potential harm or negativity in the text.
Key Steps and Components:

Dependency Installation: Guidelines for installing necessary dependencies.
Data Preprocessing: Procedures for preprocessing the dataset to prepare it for model training.
Sequential Model Creation: Detailed steps on creating a sequential model using TensorFlow.
Prediction Generation: Description of the process to generate predictions based on the trained model.
Model Evaluation: Methods and metrics used for evaluating the performance of the toxicity grading model.
Testing using Gradio: Instructions and insights for testing the model using the Gradio interface.

Please feel free to reach out if any additional information or clarification is needed.

Great Work


chore: research.msl + analytics mapping updates

approved preview deployment take a look and let us know if you would like to merge @tribixbite


🐛 Bug Report: HeaderTabs have a bad aria-label

📜 Description
The HeaderTabs component seem to render some example aria-label made a possible copy paste from Mui docs? 😅
👍 Expected behavior
Either respect the aria-label being passed in from outside, or remove entirely, or set to something reasonable.
👎 Actual Behavior with Screenshots
https://github.com/backstage/backstage/blob/56b70f2f03d382549096e70169d056838af9d45e/packages/core-components/src/layout/HeaderTabs/HeaderTabs.tsx#L107
👟 Reproduction steps
Open the inspector and check the aria-label
📃 Provide the context for the Bug.
No response
🖥 Your Environment
No response
👀 Have you spent some time to check if this bug has been raised before?

[X] I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

[X] I have read the Code of Conduct

Are you willing to submit PR?
No response

Can I take up this issue.

Alright, assigned!

aria-label, with what i understood should describe the function of that element to help visually impaired users.
we could write
aria-label="automatic scroll button"
i am just starting to contribute. are we suppose to discuss the solution to the bug in comments?

I'd say it's best to switch out the aria label to one that is a generic description of the component. If that's tricky I think we can remove the label instead.
At this point I think it's best to open up a PR with a proposed solution, typically you don't want to get too far into problem solving specifics in issues.


ci: add check for markdown links

Summary
This PR adds a new workflow that checks the markdown files in our codebase for any dead links.
Limitation

This setup does not check for 'internal' links. (example) This is a limitation with the tool I am using here. I will file a follow-up issue for this, and fork the tool to create a fix.

Closes #731

@Dhaiwat10 About the PR description/mention of the limitation regarding internal links, it shouldn't be a problem. As we spoke in the last sync, Vitepress will check for these already. (cc @Torres-ssf)

@Dhaiwat10 About the PR description/mention of the limitation regarding internal links, it shouldn't be a problem. As we spoke in the last sync, Vitepress will check for these already. (cc @Torres-ssf)

Exactly @arboleya. Vitepress fails to build if it finds any dead links. If this happens, the project build will also fail. Therefore, we do not need to extend the job here to internal links.


Unable to install CocoaPods

My project has recently stopped working.
I'm getting numerous messages like this:
The iOS Simulator deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 9.0, but the range of supported deployment target versions is 11.0 to 16.2.99. (in target 'FirebaseCrashlytics' from project 'Pods')
I've tried deleting the Podflie.lock & then re-instaling using arch -x86_64 pod install & I'll see this at the top:
Writing result bundle at path:     	/var/folders/f8/xmd2p83n0gjbk29mcqzxxmv00000gn/T/flutter_tools.pnHGn1/flutter_ios_build_temp_dirqnUU9P/temporary_xcresult_bundle : Error: Member not found: 'FirebaseAppPlatform.verifyExtends'. ../…/src/firebase_app.dart:18         FirebaseAppPlatform.verifyExtends(_delegate);                             ^^^^^^^^^^^^^
I'm at a loss on what else to try.
My Flutter Doctor:
flutter doctor -v:
[flutter] flutter doctor -v
[✓] Flutter (Channel stable, 3.3.10, on macOS 13.1 22C65 darwin-arm, locale en-US)
• Flutter version 3.3.10 on channel stable at /Users/lukeirvin/Development/flutter
• Upstream repository https://github.com/flutter/flutter.git
• Framework revision 135454af32 (3 weeks ago), 2022-12-15 07:36:55 -0800
• Engine revision 3316dd8728
• Dart version 2.18.6
• DevTools version 2.15.0
[✓] Android toolchain - develop for Android devices (Android SDK version 32.1.0-rc1)
• Android SDK at /Users/lukeirvin/Library/Android/sdk
• Platform android-33, build-tools 32.1.0-rc1
• Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (build 11.0.13+0-b1751.21-8125866)
• All Android licenses accepted.
[✓] Xcode - develop for iOS and macOS (Xcode 14.2)
• Xcode at /Applications/Xcode.app/Contents/Developer
• Build 14C18
• CocoaPods version 1.11.3
[✓] Chrome - develop for the web
• Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
[✓] Android Studio (version 2021.3)
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 11.0.13+0-b1751.21-8125866)
[✓] VS Code (version 1.74.2)
• VS Code at /Applications/Visual Studio Code.app/Contents
• Flutter extension version 3.56.0
[✓] Connected device (3 available)
• iPhone 14 Pro Max (mobile) • 9264C232-AB9C-49BD-9CBE-A80FA501866B • ios            • com.apple.CoreSimulator.SimRuntime.iOS-16-2 (simulator)
• macOS (desktop)            • macos                                • darwin-arm64   • macOS 13.1 22C65 darwin-arm
• Chrome (web)               • chrome                               • web-javascript • Google Chrome 108.0.5359.124
[✓] HTTP Host Availability
• All required HTTP hosts are available
• No issues found!

The iOS Simulator deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 9.0, but the range of supported deployment target versions is 11.0 to 16.2.99. (in target 'FirebaseCrashlytics' from project 'Pods')

@lukeirvin This message indicates you need to update your minimum deployment target from Xcode to the required value, ie, to 11.0 which should resolve this error.

Writing result bundle at path: /var/folders/f8/xmd2p83n0gjbk29mcqzxxmv00000gn/T/flutter_tools.pnHGn1/flutter_ios_build_temp_dirqnUU9P/temporary_xcresult_bundle : Error: Member not found: 'FirebaseAppPlatform.verifyExtends'. ../…/src/firebase_app.dart:18 FirebaseAppPlatform.verifyExtends(_delegate); ^^^^^^^^^^^^^

Regarding this error, you'll need to upgrade all of your flutterfire plugins you are using, to the latest and greatest version.
See this similar issue for your reference.
Closing this issue, as this is not a framework bug but a setup issue at your end.


Control # of posts for archive pages (post/_index.md)

Feature Request
Is your feature request related to a problem? Please describe.
It seems to be impossible to control the number of posts on the archive pages. You can set content.count in the file content/home/posts.md, but not in content/post/_index.md.
Describe the solution you'd like
The same parameter in content/post/_index.md to control how many entries are on one page (I'd like to have all on one).
Describe alternatives you've considered
There don't seem to be any.
Additional context
n/a

See https://github.com/wowchemy/starter-academic/blob/c00b9ca28dccdaeac245ce0439653df8f78fbe92/config/_default/config.toml#L33

sorry, I disagree. I cannot set this to "0" for "all posts" (I have to set it to e.g. "99999"), which is (a) not the same, (b) not as convenient, and (c) not adjustable independantly for different archive types.

sorry, I disagree. I cannot set this to "0" for "all posts" (I have to set it to e.g. "99999"), which is (a) not the same, (b) not as convenient, and (c) not adjustable independantly for different archive types.

(but to be honest - I didn't know about it, that much is true :) )

(but to be honest - I didn't know about it, that much is true :) )


Only allow cite attribute on blockquote and restrict schemes

This limits the schemes that can be used with the cite attribute on blockquote elements.
/cc @jch @ytrezq @oreoshake @ptoomey3 @gregose

@mastahyeti : Did you opened it because you checked September 26 e‑mails or because I updated the comment.
Also why restrict it only for blockquote ? Why not doing it the same way as sanitize.rb in it’s basic configuration ?

I hope this change will also takes place on GitHub’s owned web sites !

Yeah, we've run into some trouble updating the Gem on our servers because of some incompatibilities with internal libraries. We're working on fixing this.

Also why restrict it only to blockquote ?

From my searches, the cite attribute is only allowed on blockquote tags.

Why not doing it the same way as sanitize.rb inside it’s basic configuration ?

Because GitHub doesn't make use of the whitelist in that Gem. I'm sure rgrove would appreciate a pull request though.

@mastahyeti : As per ᴡ3ᴄ spec It’s a valid attribute on “blockquote; del; ins; q”

Thanks for finding that @ytrezq. Apparently, I didn't Google hard enough :smile:. I updated this PR.

@mastahyeti : You definitely have comprehension troubles. I told “There is pending ᴘʀ on your repository for reflecting the spec on this ᴘʀ.”
But apparently you’re prefering doing the work twice

Diff is easier to read when turning off whitespace: https://github.com/jch/html-pipeline/pull/223/files?w=1
Looks good to me :+1:


bundle.css is bundled again

I have the following fuse.js settings:
const fuse = FuseBox.init({
    homeDir: 'src/',
    output: 'static/$name.js',
    target: 'browser',
    plugins: [
        [
            SassPlugin({ importer: true }),
            CSSResourcePlugin({
                dist: 'static/assets/resources',
                resolve: f => `assets/resources/${f}`
            }),
            CSSPlugin({
                group: 'bundle.css',
                outFile: `static/assets/css/bundle.css`
            })
        ]
    ],
    shim: {
        jquery: {
            source: 'node_modules/jquery/dist/jquery.js',
            exports: '$'
        },
        tether: {
            source: 'node_modules/tether/dist/js/tether.js',
            exports: 'Tether'
        }
    }
});

fuse.dev({ httpServer: true, hmr: true, port: 3000 });
fuse.bundle('assets/js/bundle').instructions('>index.ts').watch().hmr();
fuse.run();

All scss and ts files are in src folder
All bundles (bundle.css and bundle.js) are generated to static folder

bundle.css file is bundled to a new bundle as you can see in the following log:
--------------------------
Bundle "assets/js/bundle"

    index.js
    ts/sliders.js
    app.scss
    bundle.css
└──  (4 files,  1.7 kB) default
└── jquery 203 Bytes (1 files)
└── tether 208 Bytes (1 files)
└──<EMAIL_ADDRESS>1.8 kB (0 files)
└──<EMAIL_ADDRESS>2.9 kB (0 files)
└──<EMAIL_ADDRESS>9.8 kB (0 files)
└──<EMAIL_ADDRESS>97.6 kB (0 files)
└──<EMAIL_ADDRESS>82.5 kB (0 files)
└──<EMAIL_ADDRESS>1.5 kB (0 files)
size: 198.2 kB in 9s 110ms

---------------------------------------------------
Development server running http://localhost:3000
---------------------------------------------------

22:36:00: HMR is enabled

This show an error in browser:

The whole project is available here:
https://github.com/jezikk/fuse-box-test

@jezikk it's not resolved correctly
http://fuse-box.org/plugins/css-plugin#inject

@nchanged  I'm so sorry, but I still don't understand where is the problem. I use this settings from docs:
CSSPlugin({
        group: "app.css",
        outFile: `${tmp}/app.css`
    })

So I assume that all scss > css files will be bundled into app.css and then this file can be statically linked in index.html.

@jezikk they aren't linked. Unless you link it manually. The script is being appended to the HEAD, therefore we need to know how resolve it. Use inject tag

@nchanged that is correct, I link this bundle manually in my index.html as follows:
<link rel="stylesheet" href="assets/css/bundle.css">

This is my file structure:
src/
static/
-- assets/
         -- css/
               -- bundle.css
         -- js/
               -- bundle.js
-- index.html

and my server is running in the static folder. So I don't know where is the problem, because all bundles are placed in the assets folder. I would expect this kind of error bundle.js:19073 GET http://localhost:3000/assets/css/bundle.css  and not this one bundle.js:19073 GET http://localhost:3000/bundle.css . I think that problem is somewhere in the bundle.js.

Did you set inject : false in this case?

Yes, it solved this problem. Thank you!