npm: fixed dependency for<EMAIL_ADDRESS>

Hello,
here is pullrequest to fix dependencies for<EMAIL_ADDRESS>https://github.com/npm/npm/releases/tag/v3.0.0

+1

#1221

No real reason to have a range with a direct dependency

If you mean „range for version“, then it makes sense since npm@3 will cherrypick the right version between multiple dependant modules. Its part of it’s flattening process.

Fair point. It should be done for every dep then, not just this one

How does this change affect npm@2 users?

On So. Juli 12 2015 22:04:07 Tobias Koppers wrote:

How does this change affect npm@2 users?
—
Reply to this email directly or view it on GitHub https://github.com/webpack/webpack/pull/1197#issuecomment-120758304.

It doesn’t. Stuff will be as it was before for npm@2. But it will have a pretty impact on npm@3.

Thanks


LoginAsync loop

Hi,
I'm trying to use App.FacebookSessionClient.LoginAsync from the WP7 client sample.
It keeps looping in the loginAsync method and never set the FacebookSession object since a InvalidOperationException is throw by this method.
I added this IF only to check the progress:
if(App.FacebookSessionClient.LoginInProgress)
{
continue;
}
And I noticed that the login is always in progress and never end after the webview is dismissed with the login.
How can I get it working.
Thanks!

With the rewrite of the LoginWithBehavior and removing the LoginAsync, this issue has been fixed. Try the Session.ActiveSession.LoginWithBehavior method to login.

Hi,
When this was implemented? Is there a new release? The current sdk still use grap v1.1 while it is comming to deprecate really soon.
Thanks
Sent from my iPhone

On 30/01/2015, at 23:13, Sanjeev Dwivedi<EMAIL_ADDRESS>wrote:
With the rewrite of the LoginWithBehavior and removing the LoginAsync, this issue has been fixed. Try the Session.ActiveSession.LoginWithBehavior method to login.
—
Reply to this email directly or view it on GitHub.

Facebook.dll already has 2.1 support in the alpha branch. Facebook.Client.dll had this functionality added just a week ago.
-----Original Message-----
From: "galvesribeiro"<EMAIL_ADDRESS>Sent: 1/30/2015 5:19 PM
To: "facebook-csharp-sdk/facebook-winclient-sdk"<EMAIL_ADDRESS>Cc: "Sanjeev Dwivedi"<EMAIL_ADDRESS>Subject: Re: [facebook-winclient-sdk] LoginAsync loop (#29)
Hi,
When this was implemented? Is there a new release? The current sdk still use grap v1.1 while it is comming to deprecate really soon.
Thanks
Sent from my iPhone

On 30/01/2015, at 23:13, Sanjeev Dwivedi<EMAIL_ADDRESS>wrote:
With the rewrite of the LoginWithBehavior and removing the LoginAsync, this issue has been fixed. Try the Session.ActiveSession.LoginWithBehavior method to login.
—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub.=


The ‘Target directory for download’ stage is taking too long

For example, this is JoyTag, but every time I change the image, it takes 250 seconds to process, most of which is spent on ‘Target directory for download’ and ‘Fetching’.
got prompt
Target directory for download: D:\AI\ComfyUI_windows_portable\ComfyUI\models\LLavacheckpoints\files_for_joytagger
Fetching 6 files: 100%|██████████████████████████████████████████████████████████████████| 6/6 [01:21<00:00, 13.54s/it]
Model path: D:\AI\ComfyUI_windows_portable\ComfyUI\models\LLavacheckpoints\files_for_joytagger
Model path: D:\AI\ComfyUI_windows_portable\ComfyUI\models\LLavacheckpoints\files_for_joytagger
Prompt executed in 251.05 seconds

I don’t remember it taking this long before… I can’t think of any reason why.

well, it should not do that, maybe did you check the inside this folder? are files there?

It seems that the files are properly in the specified folder. Of course, it’s an SSD.
It might not be related, but when I ran the program with the PC disconnected from the internet, the following error occurred.
Error occurred when executing Joytag:

[Errno 2] No such file or directory: 'C:\\Users\\---\\.cache\\huggingface\\hub\\models--fancyfeast--joytag\\snapshots\\6b7f16331a6ccf0fdce37d5a9564715f6e772b22\\config.json'

can you change the local_files_only=True in the snapshot_download function inside nodes/joytag.py

I tried it right away, but despite being connected to the internet, I started getting the same error as when the internet connection was cut off.
Error occurred when executing Joytag:

[Errno 2] No such file or directory: 'C:\\Users\\---\\.cache\\huggingface\\hub\\models--fancyfeast--joytag\\snapshots\\6b7f16331a6ccf0fdce37d5a9564715f6e772b22\\config.json'

can you change this
def download_joytag():
    # Ensure the correct behavior based on the existence of the local directory
    print(f"Target directory for download: {files_for_joytagger}")

    # Call snapshot_download with specified parameters
    path = snapshot_download(
        "fancyfeast/joytag",  # Example repo_id
        local_dir=files_for_joytagger,
        force_download=False,  # Set to True if you always want to download, regardless of local copy
        local_files_only=False,  # Set to False to allow downloading if not available locally
        local_dir_use_symlinks="auto"  # or set to True/False based on your symlink preference
    )
    print(f"Model path: {path}")
    return path

def download_joytag():
   # Ensure the correct behavior based on the existence of the local directory
   print(f"Target directory for download: {files_for_joytagger}")

   # Call snapshot_download with specified parameters
   path = files_for_joytagger
   print(f"Model path: {path}")
   return path

to this

It works perfectly! The speed is fantastic.

nice!

Thank you so much!
However, I’m not very familiar with how git works. Will this code be overwritten in future updates?
I feel like we could avoid such issues by checking if files_for_joytagger exists before executing snapshot_download. What do you think?

yes it could be done, i have tried it messed it up but i will make it work soon.

There’s also a high possibility that the snapshot issue is being caused by my environment, so I’ll report back if I find out anything.
Thank you😄

all i need to check is if the folder is empty or not, otherwise it should use the ````files_for_...``` for each folder. i will do it soon.


Adds string and boolean field filters in the sidebar

Resolves https://github.com/voxel51/fiftyone/issues/827 and https://github.com/voxel51/fiftyone/issues/833.
Very happy with this refactor. Lots of technical debt has been banished.

Adds string and boolean field filters in the sidebar
Adds color by label support for classification and scalar field bubbles in the sample grid

Needs more testing, I have yet to test video and classifications. The implementation is complete. though.

One more thing on this branch: can you get per-label colors working for classifications, strings, and boolean fields? I think the per-label colors option doesn't apply to one or more of these.
These requests are motivated by viewing datasets that have been evaluated via https://github.com/voxel51/fiftyone/pull/849 in the App and wanting to filter by correct/incorrect (boolean field), TP/FP/FN/TN (string field), and wanting to see the different values for these fields in different colors (so that red = correct, green = incorrect, for example).
In searching for an existing issue about this, I found https://github.com/voxel51/fiftyone/issues/725, which is less-critical but related.

Will be undrafted today.

Undrafting

I think Filter no label/value is working in reverse- I'd expect it to remove objects/samples that have no label/value, but on the test code I put above, it removes all samples

Also, are you able to get string fields to appear under the scalars tab? I'm not seeing a graph for the string field when running my sample code above.

Here's some test code that has None values in all field types:

import random

import fiftyone as fo
import fiftyone.zoo as foz
from fiftyone import ViewField as F

ANIMALS = [
    "bear", "bird", "cat", "cow", "dog", "elephant", "giraffe",
    "horse", "sheep", "zebra"
]

dataset = foz.load_zoo_dataset("quickstart").clone()

dataset.add_sample_field("string", fo.StringField)
dataset.add_sample_field("bool", fo.BooleanField)

with fo.ProgressBar() as pb:
    for sample in pb(dataset):
        for detection in sample["ground_truth"].detections:
            if random.random() < 0.1:
                detection.label = None

        for detection in sample["predictions"].detections:
            if random.random() < 0.1:
                detection.label = None

        if random.random() < 0.1:
            sample["uniqueness"] = None

        s = random.choice(ANIMALS)
        if random.random() < 0.1:
            s = None

        sample["string"] = s

        r = random.random()
        if r < 0.33:
            b = True
        elif r < 0.66:
            b = False
        else:
            b = None

        sample["bool"] = b
        sample.save()

print(dataset.filter_labels("ground_truth", F("label") == None).count("ground_truth.detections"))
print(dataset.filter_labels("predictions", F("label") == None).count("predictions.detections"))
print(dataset.match(F("uniqueness") == None).count())
print(dataset.match(F("string") == None).count())
print(dataset.match(F("bool") == None).count())

session = fo.launch_app(dataset)

Also in running the test code with None labels above, I think I found a bug: the App crashes if objects have label == None.

I did a bit more thinking about filtering over in https://github.com/voxel51/fiftyone/issues/803#issuecomment-776817054.
Do you think you can sneak support into this PR for hiding the Filter no values toggles when there are no None values? This would reduce cognitive overload of the filtering options.

Do you think you can sneak support into this PR for hiding the Filter no values toggles when there are no None values? This would reduce cognitive overload of the filtering options.

Yes, will do.

Also, are you able to get string fields to appear under the scalars tab? I'm not seeing a graph for the string field when running my sample code above.

Working on it. I forgot about this request.

@benjaminpkane what do you think about adding (a) from https://github.com/voxel51/fiftyone/issues/833 in here? For labels it would be + filter by label and for other values + filter by value.
I'm not quite sure on part (b) from there anymore. May need more discussion.

Will change to (a). Makes sense to me.
For (b) I would propose Exclude no value or Exclude None (where None is code highlighted.

ok yeah I like exclude for the None cases.
I think that terminology should be used for Exclude true and Exclude false in the boolean filters too, until we figure out something better as per https://github.com/voxel51/fiftyone/issues/803#issuecomment-776826126.

Comments addressed, though I am still testing. Happy to tweak things still, like Exclude ... (See screenshots)

. Still digesting some comments made on issues outside this PR. May sneak some more changes in, though.

Looking good! I noticed a couple things in expanded sample mode:

The Exclude None filter for labels doesn't seem to be working. Perhaps this is because the label has been cast to a string or something like that? On a related note: I think it would be preferable to render label == None as label == "" so that users can have boxes with no labels if that's what they'd like
I noticed that applying label filter to a field that has been unselected does not cause the label fields background to change colors. The color change is important regardless of whether the label field is toggled on because it conveys whether their is an active filter on that field (in my opinion)

I hate to keep stringing along this PR, but do you think you can squeeze in hiding the Exclude None element when there are no None-valued items in the current view?
I bring this up because of a second point of clarification I just found: when using the filter by label/value option in the App to select certain labels you want to see, I think that None valued fields should also be hidden, since their label/value doesn't match the search any more than a non-None-non-matching label value does. If the Exclude None element were always shown, then this would necessitate auto-checking the Exclude None box when a filter is performed. But, it would be cleaner if the Exclude None element just disappeared since it is no longer relevant.

I hate to keep stringing along this PR, but do you think you can squeeze in hiding the Exclude None element when there are no None-valued items in the current view?

In the extended view?

I hate to keep stringing along this PR, but do you think you can squeeze in hiding the Exclude None element when there are no None-valued items in the current view?

In the extended view?

yes in https://github.com/voxel51/fiftyone/pull/848#issuecomment-777044873 I'm back in extended view on the sample grid modal

Ok, I'll address the new comments today. I am seeing some weird behavior from the state library (recoil), so I need to work through that.

Ok, comments addressed, I hope. My None solution was to add it as a special case in the search results.
I could add an inversion/negation checkbox in the morning so the selected values can be excluded instead.

list.

Thanks for all the work on this!
I do think it would be really useful to have a field exclusion option. My thoughts on that are in point 2 of https://github.com/voxel51/fiftyone/issues/803, but it occurs to me that selecting one label and excluding another one independently doesn't make sense; it's either all selection or all exclusion. So I think your checkbox idea sounds promising.
Here's my last set of observations. And I'll go ahead and approve now to avoid holding you up :)

In expanded sample modal: the scalar fields are clickable but it doesn't do anything. Can you remove the click event (like is true for tags, for example)?

Can you make list selection dropdowns auto-hide after something is selected? I think the typical case is to select one thing at a time, so defaulting to auto-hide saves the user time. It's easy to click into the modal again if multiple clicks are desired.

I think if a range filter is applied to a numeric field, this should automatically remove any None-valued fields. This is how the equivalent view stage would work, btw, since a < None < b == False in MongoDB.


[Question]: Ingestion pipeline duplicates with Postgres

Question Validation

[X] I have searched both the documentation and discord for an answer.

Question
Hello,
I've got a question that I suspect is a configuration issue rather than a bug.  When I run my ingestion pipeline (code below) repeatedly on the same set of source documents, my vector database increases in size on each run, suggesting that some of the information is being duplicated rather than upserted.  I believe I have the vector store, docstore, and cache set up correctly with Postgres, as they are populated on ingestion and queries work fine.  I've reviewed these issues:  1, 2.  In both it's suggested that the docstore be explicitly saved and loaded.  However,  I don't know if that applies in my case, since I am using a database rather than a local directory.
I'd be very appreciative of any help, either in advancing my understanding of how repeated ingestions on the same source material are supposed to work or in directly fixing my problem.  Thanks in advance.
Perform ingestion:
vector_store = get_vector_store()
cache = get_cache()
docstore = get_docstore()

def perform_ingestion():

    try:
        documents = SimpleDirectoryReader(
            local_vault_copy_path,
            recursive=True,
            required_exts=included_extensions,
            file_metadata=get_meta,
        ).load_data()

        pipeline = IngestionPipeline(
            transformations=transformations,
            vector_store=vector_store,
            cache=cache,
            docstore=docstore
        )

        pipeline.run(documents=documents)
        return "Ingestion completed successfully."
    except Exception as e:
        return f"Ingestion failed with error: {e}"

Functions to create document store, vector store and cache:
def get_vector_store():
    conn = get_database_connection()
    full_database_url = (
        f"{config['Database']['Connection string']}/{config['Database']['Name']}"
    )
    url = make_url(full_database_url)

    vector_store = PGVectorStore.from_params(
        database=url.database,
        host=url.host,
        password=url.password,
        port=url.port,
        user=url.username,
        table_name=config["Database"]["Vector table name"],
        embed_dim=config["Ollama settings"]["Embedding dimensions"],
    )
    return vector_store

def get_docstore():
    conn = get_database_connection()
    full_database_url = (
        f"{config['Database']['Connection string']}/{config['Database']['Name']}"
    )
    url = make_url(full_database_url)

    docstore = PostgresDocumentStore.from_params(
        database=url.database,
        host=url.host,
        password=url.password,
        port=url.port,
        user=url.username,
        table_name=config["Database"]["Document store table name"],
    )
    return docstore

def get_storage_context():
    vector_store = get_vector_store()
    docstore = get_docstore()
    storage_context = StorageContext.from_defaults(
        vector_store=vector_store, docstore=docstore
    )
    return storage_context

def get_cache():
    conn = get_database_connection()
    full_database_url = (
        f"{config['Database']['Connection string']}/{config['Database']['Name']}"
    )
    url = make_url(full_database_url)

    cache = IngestionCache(
        cache=PGCache.from_params(
            database=url.database,
            host=url.host,
            password=url.password,
            port=url.port,
            user=url.username,
            table_name=config["Database"]["Cache table name"],
        )
    )
    return cache

Do the documents have the same document ID when you run them the second time? This ID needs to be constant in order to properly upset

Thanks @logan-markewich .  For some reason, they do not.  I tested the ingestion on a single file.  Running the ingestion a few seconds apart for the same file, without modifying the file between runs, gives me two different IDs.
First ingestion
{"doc_hash": "d87e7ab6dc1a238af8719edc739f337efa095a9f1414e315b95665b368933ff7"}
{"__data__": {"id_": "9b9f2a3c-fbbf-47e0-93eb-774ef624f0fc" . . . .

Second ingestion
{"doc_hash": "d87e7ab6dc1a238af8719edc739f337efa095a9f1414e315b95665b368933ff7"}
{"__data__": {"id_": "331b10c8-c02d-42e0-b4f9-9f72e8124595" . . . .

Obviously, the hash is the same but the ID is different.

I discovered that if I add filename_as_id=True in my call to SimpleDirectorReader, the duplication issue does not occur.
        documents = SimpleDirectoryReader(
            local_vault_copy_path,
            recursive=True,
            required_exts=included_extensions,
            file_metadata=get_meta,
            filename_as_id=True
        ).load_data()

This effectively solves my problem, but I am still a confused as to why the programmatically generated document id was being duplicated before.


LISTS AND SUM FUNCTION

firstly created a list of range 10 then by using python function sum : add all the integers given in the list.

Merged

This is an automated message from Fork, Commit, Merge [BOT].

Thank you for your contribution! Your pull request has been merged. The files have been reset for the next contributor.
What's next?
If you're looking for more ways to contribute, I invite you to check out our forkcommitmerge.io website repository. The repository contain real issues that you can help to resolve or create some of your own if you see any bugs or want to add new features to the site, etc. You can also check out the Influences section in the README to find more projects similar to this one.
Also please leave a star to this project if you feel it helped you, i would really appreciate it.
I look forward to seeing your contributions!


[feature request] Add Bulk mail check

Is it possible to consider adding a multiple mail check feature? This feature would be a valuable addition, simplifying the management of multiple emails. Your consideration of this feature request would be greatly appreciated.😄

Hey! Sorry for the delay. Could you give more information on your scenario? Amount of parallel checks expected, output format?

The application should test about 1k emails from a txt file and output the results in a separate txt file.


Implement Exercise: Acronym

Ref: https://github.com/exercism/abap/issues/221

./exercises/practice/acronym/zcl_acronym.clas.testclasses.abap[7, 10] - Variable "CUT" contains unknown: REF, unable to resolve zcl_acronym (unknown_types) [E]
./exercises/practice/acronym/zcl_acronym.clas.testclasses.abap[30, 5] - Method "parse" not found, methodCallChain (check_syntax) [E]
I dont understand why its failing on this.

🎉

bugfix = https://github.com/abaplint/abaplint/pull/2735 to check the missing whitespace

closed by mistake, opened again. Thanks for the pointers, please do the needful

merging, thanks 🚢


Grammar and end links in article: Add a tile layer

Per prior arrangement, if this PR needs approval, or if the commits need to be pulled into the private repo, only @eriklindeman should be notified. Thanks.

@stevemunk @eriklindeman - merging this, but please see my question above and submit a new PR if changes should be made. thanks.


Remove honeybadger exception when a branch is not merged

Remove Honeybadger exception which was originally used for tracing an error on ST search page https://github.com/FundingCircle/shipment_tracker/pull/70

LGTM

LGTM


Prepare for Performance 2.0.0

Current dependencies on/for this PR:

main

PR #57   👈

This comment was auto-generated by Graphite.


Updated Polish translations & fixed #1289

Do these changes match the file on Transifex?

@jjeffryes yes, I made all translations on Transifex, downloaded JSON file from Transifex and placed it as js/languages/pl.json

@programmerpeter awesome. That's exactly what I was hoping for.


NuGet update doesn't retain CopyToOutputDirectory in Proj file setting for NLog.config

Type (choose one): Bug
NLog version: 4.4.13
Platform: .Net 4.5
Current Result
The update changed this in the .csproj file:
<Content Include="NLog.config">
  <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</Content>

to:
<Content Include="NLog.config">
  <CopyToOutputDirectory>Always</CopyToOutputDirectory>
</Content>

Expected Result
It should leave it set to "PreserveNewest" if that's what is already there.
BECAUSE:
When you set it to "Always" that forces a rebuild of that project (and all of its dependencies) in VS2017 every single time you run your solution even if nothing has changed in the project.  So anyone who has set this to "PreserveNewest" has taken the time to figure this out and doesn't want it changed on every nuget update.

It should leave it set to "PreserveNewest"

Not sure if we could control this
This will only be set after a nuget install, isn't?
Guess you could control this by install nlog and not the nlog.config nuget package
Anyway, "PreserveNewest", is maybe a better default

I just upgraded from 4.4.12 to 4.4.13 and it changed it.  I caught this because I had just spent a lot of time two weeks ago tracking down why every one of my projects was rebuilding every time I ran the solution even though nothing had changed.
I don't know anything about nuget updates, so I don't know if it is deleting the entry and re-adding it or not.  If it is, that's probably going to make it harder to fix.  Hopefully it's just updating it and it can decide to leave the setting alone.


Document the binary formats for cryptographic structures defined by the specification

Within the draft there are several new cryptographic structures that are encoded in binary, their format should be documented to support interoperable implementations, these include

ProofOfKnowledge (signature proof)
Commitment
Commitment ProofOfKnowledge
BBS Signature

@andrewwhitehead

A topic for future discussion is whether to use a structured (e.g CBOR) vs custom binary format, current implementations use a custom binary format however structured solutions like CBOR could be considered for future revisions.

I’m hoping you document CBOR versions of these (and not the IPFS version of CBOR). We are increasingly using CBOR in wallet interoperability standards as it allows airgap solutions like UR:

https://github.com/BlockchainCommons/crypto-commons#urs
https://github.com/BlockchainCommons/Research/blob/master/papers/bcr-2020-005-ur.md

#66 adds a preliminary serialisation format for the signature, however the proof representation is still un-documented

Signatures have been addressed in #66 only outstanding rules are for proofs therefore closing this issue in favour of #153


[BUSINESS CRITICAL] aurajs.com - github pages site is inaccessible

When visiting either aurajs.com or aurajs.github.io I hit the following error page:

Site Disabled
Oh no! It looks like this Pages site has been disabled due to a misconfigured custom domain.

Are you the site owner? You can quickly fix this issue by updating the DNS record for your Pages site:

If you're using a subdomain (coolthing.example.com, www.example.com, etc.), make sure you have a CNAME record that points to your-username.github.io.
If you're not using a subdomain (example.com), make sure you have two A records set to the following IPs: <IP_ADDRESS> and <IP_ADDRESS>

More detailed instructions for setting up a custom domain with GitHub pages are also available if you need more guidance.

You can find more information about these changes in our GitHub Pages legacy IP deprecation blog post. If you have any questions, please contact support.

This is business critical as I was unable to find out what aurajs is and thus cannot use it or spread its use.

Hey @addyosmani I think you have the control of that domain, can you please check why this is happening ?
Thanks

Hello @sbellity , @addyosmani
Any updates on this issue?

Hey folks. Sorry for the downtime! I was only just notified about this again on Twitter and will find out what happened to the domain.

DNS settings were apparently changed in GH documentation. I've updated and will let you know once the domain is back up.

The site is now back online.

@addyosmani It's down again.


hook release filename changed

Expected Behaviour

The default sandbox configuration should work with hook, since it is set as default.
Release assets should not be updated/replaced without a new release being made. It is extremely confusing when release assets change and the date is not updated to reflect that there was indeed a recent change.

Current Behaviour
Sandbox will not be able to pull hook since it references OSIE_DOWNLOAD_URL="https://github.com/tinkerbell/hook/releases/download/5.10.57/hook-x86_64.tar.gz" and that file no longer exists. The x86_64 release for hook now lives at https://github.com/tinkerbell/hook/releases/download/5.10.57/hook_x86_64.tar.gz. This name change also seems to break the osie/lastmile.sh script in sandbox since filenames have changed inside the hook release archive.
We may have also experienced additional issues with the latest hook asset available on the release page. It seemed like hook wasn't properly trusting or able to connect to the sandbox registry after we made some modifications to work with the current version of hook available on the release page.
Possible Solution
Keeping consistent with filenames for hook would be very beneficial if other things depend on that naming. Additionally, some documentation for general hook configuration and troubleshooting would be helpful.
Steps to Reproduce (for bugs)

Examine deploy/compose/.env in sandbox
Compare default OSIE URL in .env against URL for current hook release

Context
We are attempting to develop an ansible role to deploy the tinkerbell sandbox and this issue has perplexed us quite a bit. It is our opinion that release artifacts should not change without also some separate verbose acknowledgement that there's been a change to the release.
Your Environment

Operating System and version (e.g. Linux, Windows, MacOS): Fedora 34 and Debian

How are you running Tinkerbell? Using Vagrant & VirtualBox, Vagrant & Libvirt, on Packet using Terraform, or give details: Custom ansible.

Link to your project or a code example to reproduce issue: https://github.com/jkl92/sandbox/tree/ansible-podman_container

@thebsdbox - any chance this issue is on your radar?

@tstromberg do you have any pointers on why hook would seem to be unable to contact the sandbox registry? We're pretty much following the same patterns in the sandbox docker-compose but deploying with ansible instead. We were kind of stumped when it came to debugging how hook determines:

where the registry is located
how to trust the registry
how hook is made aware of any crypto keys that are necessary to trust or communicate with the registry

Does hook or the container payloads it runs ever communicate directly with the tink server? I couldn't really find any documentation about this aspect. If hook or payloads talk to the tink server it is also possible there was an issue with hook communicating with the workflow/tink server.

Hey @jmpolom, I'll try and answer these.
- where the registry is located
Hook has a built-in app (bootkit) that runs at Hook startup which will parse kernel command line args. These cmdline args will be used when bootkit starts the tink-worker.
The parsing of the cmdline file happens here in bootkit.
For the container registry, bootkit is looking in the cmdline file for docker_registry= to determine where to pull images.
- how to trust the registry
To enable pulling images from an insecure container registry, tink-docker downloads the container registry's public TLS cert and putting it in the local certificate trust store.
It gets the URL for this file via the cmdline file using the packet_base_url= value and appending ca.pem. So from the example cmdline below the TLS cert would be downloaded from http://<IP_ADDRESS>:42114/cert/ca.pem.
- how hook is made aware of any crypto keys that are necessary to trust or communicate with the registry
The previous response answers how the insecure registry is trusted in Hook. The last part is that the container registry uses username/password for authentication. You'll see below in the cmdline examples that these creds are passed along to Hook via registry_username and registry_password. Bootkit parse them from the cmdline file and uses them to login when downloading the tink-worker. See here. These creds are also passed into the Tink worker (see here) when it's started up so that it can pull images when running workflow.
- Does hook or the container payloads it runs ever communicate directly with the tink server?
Tink worker communicates with tink server. I don't believe Hook ever does.
One final note. I've just put in a PR (https://github.com/tinkerbell/sandbox/pull/102) to get the sandbox back up and running with this new release of Hook.

Example /proc/cmdline
ip=dhcp modules=loop,squashfs,sd-mod,usb-storage alpine_repo=http://<IP_ADDRESS>:8080/misc/osie/current/repo-x86_64/main modloop=http://<IP_ADDRESS>:8080/misc/osie/current/modloop-x86_64 tinkerbell=http://<IP_ADDRESS> syslog_host=<IP_ADDRESS> parch=x86_64 packet_action=workflow packet_state=provisioning docker_registry=<IP_ADDRESS> grpc_authority=<IP_ADDRESS>:42113 grpc_cert_url=http://<IP_ADDRESS>:42114/cert instance_id= registry_username=admin registry_password=Admin1234 packet_base_url=http://<IP_ADDRESS>:8080/workflow worker_id=0eba0bf8-3772-4b4a-ab9f-6ebe93b90a94 packet_bootdev_mac=08:00:27:9e:f5:3a facility=onprem plan=c2.medium.x86 manufacturer= slug= initrd=initramfs-x86_64 console=tty0 console=ttyS1,115200

user-friendly format
ip=dhcp
modules=loop,squashfs,sd-mod,usb-storage
alpine_repo=http://<IP_ADDRESS>:8080/misc/osie/current/repo-x86_64/main
modloop=http://<IP_ADDRESS>:8080/misc/osie/current/modloop-x86_64
tinkerbell=http://<IP_ADDRESS>
syslog_host=<IP_ADDRESS>
parch=x86_64
packet_action=workflow
packet_state=provisioning
docker_registry=<IP_ADDRESS>
grpc_authority=<IP_ADDRESS>:42113
grpc_cert_url=http://<IP_ADDRESS>:42114/cert
instance_id=
registry_username=admin
registry_password=Admin1234
packet_base_url=http://<IP_ADDRESS>:8080/workflow
worker_id=0eba0bf8-3772-4b4a-ab9f-6ebe93b90a94
packet_bootdev_mac=08:00:27:9e:f5:3a
facility=onprem
plan=c2.medium.x86
manufacturer=
slug=
initrd=initramfs-x86_64
console=tty0
console=ttyS1,115200

@jacobweinstock thanks for the quick response on a Friday before a holiday weekend in the US. Very helpful info for sure.d

For the container registry, bootkit is looking in the cmdline file for docker_registry= to determine where to pull images.

Where is this set? I don't see it being set as an env var in the sandbox compose file. I'm guessing there are some defaults that are used (perhaps inferred from $TINKERBELL_HOST_IP?) but unsure where/how they are communicated to the host booting via iPXE. I'd imagine this happens somewhere within boots and is put into the dynamically generated auto.pxe iPXE script.
Definitely see how /proc/cmdline is being read there but I think our stumbling point is further up the chain. I'll test the changes you committed but we'd certainly like to understand how things are supposed to work here. I've been kind of mindlessly poking around the boots codebase and haven't yet pinpointed where/how boots is creating the iPXE script that hook (or OSIE) get booted with.

Yeah, sorry. This is an area that is sorely missing documentation. Hopefully, this helps explain a little.
You are correct, "within boots and is put into the dynamically generated auto.pxe iPXE script". This happens in this function here in Boots, and here and here.
This docker registry is set via environment variable in Boots. You can see this in the sandbox, here.

I was just traversing those same sources about the time you posted your comment.  Between the new knowledge from your explanations and updated sandbox we probably have enough to at least be dangerous.
Documentation would certainly be helpful here but luckily go isn't too hard to read through.

https://github.com/tinkerbell/sandbox/pull/102 was merged which should address the primary issue here. I suggest that any outstanding ideas/issues be moved into their own issues for separate tracking.

Moved the release automation work to #78
/cc @thebsdbox


Validation of certificate

Hi
I've implemeted https server.
How should I handle validation of incoming request?
I would like to check if incoming request with certificate matches server certificate.

You could create a handler to validate incoming requests. However if you want to check the certificate as part of the SSL handshake that isn't possible at this moment.


MySQL DML use antlr

Fixes #ISSUSE_ID.
Changes proposed in this pull request:

Pull Request Test Coverage Report for Build 7105

46 of 46 (100.0%)  changed or added relevant lines in 8 files are covered.
127 unchanged lines in 30 files lost coverage.


Make copy and paste more visible/obvious

A few people (me included) didn't realize you could copy and paste features, might be good to make this more visible, maybe add to the walkthrough? Or someone make it more clear. This is a super handy feature for buildings and huts and such.

Also mentioned in #2508

Yes, lets just track this on #2508


Redirects/add netlify plugin

Redirect works as expected on netlify throwing a 301 code first and loading the right page


document flutter's -d argument more in flutter help

When you have two or more devices connected, flutter run needs you select which one you want to deploy to. This requires copy-pasting the device identifier. "flutter run" could check an environment variable to see which device to deploy to if more than one are connected.

You can give partial (case-insensitive) identifiers too (if that helps).
For example, I commonly use -d i and -d n to differentiate from my iDevice vs. my Nexus device.

That is good enough for my purposes, but it isn't discoverable; maybe let's document the -d command more in "flutter help"?

Thanks!


#[derivative(Debug(value = "..."))]

It would be great to have the option to format a struct field or enum variant as a hardcoded string. For example:
#[derive(Derivative)]
#[derivative(Debug)]
struct Foo {
    #[derivative(Debug(value = "Box<dyn Any>"))]
    data: Box<dyn Any>,
}

// Foo { data: Box<dyn Any> }
println!("{:?}", foo);

another example:
#[derive(Derivative)]
#[derivative(Debug)]
pub enum EffectDescriptor<'cls> {
    Output(#[derivative(Debug(value = "closure[thunk]"))] Box<dyn (Fn() -> TypeBox) + 'cls>),
    UpdateVariable {
        ident: String,
        #[derivative(Debug(value = "closure[thunk]"))]
        value: Box<dyn (Fn() -> TypeBox) + 'cls>,
    },
    PushScope,
    PopScope,
}


creativeEffect not working ? - Nextjs - webComponent

Check that this is really a bug

[ ] I confirm

Reproduction link
https://codesandbox.io/p/devbox/swiper-creativeeffect-test-error-zrz7vf?file=%2Fapp%2Fpage.tsx%3A42%2C10
Bug description
I am trying to implement this effect
https://codesandbox.io/p/sandbox/swiper-effect-creative-forked-272t82?file=%2Findex.html%3A138%2C18-138%2C26 but as you can see in my reproduction, the "cards" just dont overlap
Expected Behavior
I need it to work just like in the example, the cards should overlap
Actual Behavior
No response
Swiper version
11.0.6
Platform/Target and Browser Versions
Chrome, Brave. Firefox
Validations

[X] Follow our Code of Conduct
[X] Read the docs.
[X] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.
[X] Make sure this is a Swiper issue and not a framework-specific issue

Would you like to open a PR for this bug?

[ ] I'm willing to open a PR

You also need to set init="false" on swiper-container if you initilalize it manually


Add a select performance section for modulo based sampling

This depends on https://github.com/crate/crate/pull/8438 - maybe we should note here in the guide that _docid will only be available in 4.0?

@mfussenegger might be easier if I add an edit commit to your branch. is that ok?

@mfussenegger might be easier if I add an edit commit to your branch. is that ok?

Sounds good

Can you elaborate a bit more on what you're expecting? There isn't much more to it. Processing all records takes a lot of time, processing fewer records takes less time. That's it.

@mfussenegger see my previous/first comment in my first review. I don't understand this sentence: "Similar products sometimes limit the amount of records that are processed". what is a "product" and why does it limit the processed records

"Similar products sometimes limit the amount of records that are processed". what is a "product" and why does it limit the processed records

products here is referring to the product "CrateDB" - similar products are our competitors.
I think the "why does it limit the processed records" is implicitly explained by the whole "trade accuracy for performance" part?
Not sure what the expectations towards the guide are, but the intention behind documenting this is for people who are already familiar somewhat with this approach of reducing accuracy and are specificly looking for doing that in CrateDB. It's not intended as a full fledged tutorial.

@mfussenegger aha! "database products" is what I needed to contextualize and make sense of the rest of the text. I was thinking "products" was some sort of mathematical/internal term
I have provided a final set of edits that resolve all of my previous comments on this pr

It looks like my edits in https://github.com/crate/guide/commit/bff23178b0fcc3e78077fa344140bec91abae108#diff-28f5f1f949f42ed9211bb7e61348fdccR49 weren't included in the last edits. Do you think it is better to leave that section about using the LIMIT out of it?


_ Patch Compile Tests _

+1 :green_heart:
mvninstall
4m 52s
the patch passed

+1 :green_heart:
compile
0m 36s
the patch passed

+1 :green_heart:
javac
0m 36s
the patch passed

+1 :green_heart:
shadedjars
8m  4s
patch has no errors when building our shaded downstream artifacts.

-0 :warning:
javadoc
0m 20s
hbase-common in the patch failed.

_ Other Tests _

+1 :green_heart:
unit
2m  5s
hbase-common in the patch passed.

33m 13s

Subsystem
Report/Notes

Docker
Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2484/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2484

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux 6ab865abbe72 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / ee02e673b7

Default Java
2020-01-14

javadoc
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2484/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt

javadoc
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2484/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt

Test Results
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2484/1/testReport/

Max. process+thread count
282 (vs. ulimit of 30000)

modules
C: hbase-common U: hbase-common

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2484/1/console

versions
git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@ddupg @infraio PTAL?


Make the Music Editor for mobile devices more accessible

You can't use the letters on your keyboard to assign a note.

already have one #106


Added product link in designation

Issue #289

Merged via https://github.com/prixite/prixite.github.io/pull/293


PP-11978 improve pre-filled payment link error message

PP-11978  Improve error message associated with validation of amount on pre-filled payment links
Summary of changes:

Amend page title and page heading for prefilled payment link problems
Reference new payment link template
Update associated unit test to reflect changes
Update Cypress integration test
Remove logic to test for pre-filledpayment link specific message from common error template
Add template dedicated for pre-filled payment link problems

Screenshots (see preview tab)
The screen shots below can also be found in the jira ticket PP-11978

Could you add some screenshots of what this looks like please?

Screen shots now added. They can also be viewed alogside comments in the Jira ticket - PP-11978.


[Feature]: Regional chapters button on the Slack bot

Contact Details
<EMAIL_ADDRESS>The Problem
It'd be great to also showcase and let newcomers know about the regional chapters that are in place. CHAOSS Africa, CHAOSS LAC, and CHAOSS Balkans.
How to Fix It
We could link the button to this website: https://chaoss.community/kb/local-chapters/
As well as the links to the Slack active channels # chaoss-lac # chaoss-africa
Additional Context
No response
Specific File Path(s) on the Repo to be Modified/Refered to
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

I would love to take this issue

I would love to take this issue

Assigned to you! If you have any questions while solving this issue, ask @misspee007 !

Hi @oliverTwist2, do you have any update on this?

Hey👋🏽 @oliverTwist2
Please let me know if you have a blocker, or if you need this to be reassigned to someone else.


Permalinks do not work

Permalinks do not work with this config.  How do I set the nginx re-write rules?

I faced the same issue and eventually I was able to solve it following these 2 steps.
1 - New Location Rule:
New NGINX Location rule
location / {
    try_files $uri $uri/ /index.php?$args;
}

2 - How to edit the NGINX location rules:
Configure NGINX with .ebextensions
Place that script in a new file located in .platform/nginx/conf.d/elasticbeanstalk, I named mine  permalink.conf
.platform
    nginx
         conf.d
             elasticbeanstalk
                 permalink.conf

Optitional - Elementor Fix
After using the new location rule I was still facing issues with Elementor.
Elementor Fix
Replacing permalink.conf with the following code fixed it.
location /{
    try_files $uri $uri/ /index.php$is_args$args;
}

Final permalink.conf code :
#pemalink fix

# Enable WordPress Permalinks Support (default .htaccess replacement)
# .platform/nginx/conf.d/elasticbeanstalk/permalink.conf
location / {
    try_files $uri $uri/ /index.php$is_args$args;
}

Hope these work!


Update code.gitea.io/git

Bring in latest changes from https://github.com/go-gitea/git. Includes fix for https://github.com/go-gitea/gitea/issues/491 and https://github.com/go-gitea/gitea/issues/502 (see https://github.com/go-gitea/git/pull/53)

@ethantkoenig since some function params changed, you have to change the invoke places.

I'm confused as to why make test-vendor is failing in the CI; it passes when I run it locally

@ethantkoenig you fetch the commit of your PR that is not the same as the one in master + you edit file after.
Please just do govendor fetch code.gitea.io/git to have the last commit of master and all file in sync.

Doing govendor fetch code.gitea.io/git I doesn't get the same hash.
[sapk@sapk-ultra gitea]$ govendor fetch code.gitea.io/git
[sapk@sapk-ultra gitea]$ git diff
diff --git a/vendor/vendor.json b/vendor/vendor.json
index 60e47f03..2a910aef 100644
--- a/vendor/vendor.json
+++ b/vendor/vendor.json
@@ -3,7 +3,7 @@
        "ignore": "test appengine",
        "package": [
                {
-                       "checksumSHA1": "/QkpRq7qiUkZzBbLFMZ0FFZYIOQ=",
+                       "checksumSHA1": "iqK8QebzwEcNL2iIRpMAOYwb7Zg=",
                        "path": "code.gitea.io/git",
                        "revision": "c98a6f353a13546ae13e7391020cadd750181585",
                        "revisionTime": "2017-05-29T11:48:12Z"

LGTM

LGTM


pd-ctl: add auto-completion for pd-ctl

UCP: #2170
What problem does this PR solve?
fixed: #2170
What is changed and how it works?
Added new subcommand completion
Usage:
    pd-ctl completion SHELL

Check List
Tests

Manual test (add detailed scripts or steps below)
No code

Check shell version and install bash-completion

Qiannan-MBP:bin qiannan$ echo ${BASH_VERSION}
3.2.57(1)-release
Qiannan-MBP:bin qiannan$ brew info bash-completion
bash-completion: stable 1.3 (bottled)
Programmable completion for Bash 3.2
https://salsa.debian.org/debian/bash-completion
Conflicts with:
  bash-completion@2 (because Differing version of same formula)
/usr/local/Cellar/bash-completion/1.3_3 (189 files, 607.9KB) *
  Poured from bottle on 2020-03-29 at 16:27:16
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/bash-completion.rb
==> Caveats
Add the following line to your ~/.bash_profile:
  [[ -r "/usr/local/etc/profile.d/bash_completion.sh" ]] && . "/usr/local/etc/profile.d/bash_completion.sh"

Bash completion has been installed to:
  /usr/local/etc/bash_completion.d
==> Analytics
install: 12,271 (30 days), 39,011 (90 days), 172,085 (365 days)
install-on-request: 11,294 (30 days), 36,102 (90 days), 158,051 (365 days)
build-error: 0 (30 days)

Help info:
Qiannan-MBP:bin qiannan$ pd-ctl completion -h

Output shell completion code for the specified shell (bash).
The shell code must be evaluated to provide interactive
completion of pd-ctl commands.  This can be done by sourcing it from
the .bash_profile.

Usage:
  pd-ctl completion SHELL

Examples:

	# Installing bash completion on macOS using homebrew
	## If running Bash 3.2 included with macOS
	    brew install bash-completion
	## or, if running Bash 4.1+
	    brew install bash-completion@2
	## If you've installed via other means, you may need add the completion to your completion directory
	    pd-ctl completion bash > $(brew --prefix)/etc/bash_completion.d/pd-ctl

	# Installing bash completion on Linux
	## If bash-completion is not installed on Linux, please install the 'bash-completion' package
	## via your distribution's package manager.
	## Load the pd-ctl completion code for bash into the current shell
	    source <(pd-ctl completion bash)
	## Write bash completion code to a file and source if from .bash_profile
		pd-ctl completion bash > ~/completion.bash.inc
		printf "
		# pd-ctl shell completion
		source '$HOME/completion.bash.inc'
		" >> $HOME/.bash_profile
		source $HOME/.bash_profile

Global Flags:
      --cacert string   Path of file that contains list of trusted SSL CAs.
      --cert string     Path of file that contains X509 certificate in PEM format.
  -h, --help            Help message.
      --key string      Path of file that contains X509 key in PEM format.
  -u, --pd string       Address of pd. (default "http://<IP_ADDRESS>:2379")

Load the completion code to current shell
source <(pd-ctl completion bash)

Type pd-ctl cl into the current shell, then type tab, it will auto-complete command pd-ctl cluster.
Code changes

added new file tools/pd-ctl/pdctl/command/completion_command.go
modified file tools/pd-ctl/pdctl/ctl.go

Side effects
N/A
Related changes

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 2 committers have signed the CLA.:white_check_mark: nolouch:x: hsqluYou have signed the CLA already but the status is still pending? Let us recheck it.

@nolouch Added zsh support, PTAL. Thanks

LGTM. BTW, is there any way to support auto-completion in interactive mode?

I have no ideas. Any recommended info about that? Thanks

/merge


DMap compaction deletes non-empty fragments after some time

It looks like the compaction goroutine removes non-empty fragments. Insert some keys and waits 10 minutes to reproduce the error. You can decrease the compaction interval to see the error.
➜  olric git:(master) olricd -c cmd/olricd/olricd-local.yaml
2022/04/23 13:51:23 [INFO] pid: 93542 has been started
2022/04/23 13:51:23 [INFO] Olric 0.5.0-beta.2 on darwin/arm64 go1.18.1 => olric.go:310
2022/04/23 13:51:23 [INFO] Join completed. Synced with 0 initial nodes => discovery.go:59
2022/04/23 13:51:23 [INFO] Routing table has been pushed by <IP_ADDRESS>:3320 => operations.go:86
2022/04/23 13:51:23 [INFO] The cluster coordinator has been bootstrapped => discovery.go:42
2022/04/23 13:51:23 [INFO] Memberlist bindAddr: <IP_ADDRESS>, bindPort: 3322 => routingtable.go:413
2022/04/23 13:51:23 [INFO] Cluster coordinator: <IP_ADDRESS>:3320 => routingtable.go:414
2022/04/23 13:51:23 [INFO] Node name in the cluster: <IP_ADDRESS>:3320 => olric.go:371
2022/04/23 13:51:23 [INFO] Olric bindAddr: <IP_ADDRESS>, bindPort: 3320 => olric.go:377
2022/04/23 13:51:23 [INFO] Replication count is 1 => olric.go:379
2022/04/23 13:52:23 [INFO] Routing table has been pushed by <IP_ADDRESS>:3320 => operations.go:86
2022/04/23 13:53:23 [INFO] Routing table has been pushed by <IP_ADDRESS>:3320 => operations.go:86
2022/04/23 13:54:23 [INFO] Routing table has been pushed by <IP_ADDRESS>:3320 => operations.go:86
2022/04/23 13:55:23 [INFO] Routing table has been pushed by <IP_ADDRESS>:3320 => operations.go:86
2022/04/23 13:56:23 [INFO] Routing table has been pushed by <IP_ADDRESS>:3320 => operations.go:86
2022/04/23 13:57:23 [INFO] Routing table has been pushed by <IP_ADDRESS>:3320 => operations.go:86
2022/04/23 13:58:23 [INFO] Routing table has been pushed by <IP_ADDRESS>:3320 => operations.go:86
2022/04/23 13:59:23 [INFO] Routing table has been pushed by <IP_ADDRESS>:3320 => operations.go:86
2022/04/23 14:00:23 [INFO] Routing table has been pushed by <IP_ADDRESS>:3320 => operations.go:86
2022/04/23 14:01:23 [INFO] Empty DMap fragment (kind: Primary) has been deleted: dmap.bench on PartID: 0 => janitor.go:65
2022/04/23 14:01:23 [INFO] Empty DMap fragment (kind: Primary) has been deleted: dmap.bench on PartID: 1 => janitor.go:65
2022/04/23 14:01:23 [INFO] Empty DMap fragment (kind: Primary) has been deleted: dmap.bench on PartID: 2 => janitor.go:65
2022/04/23 14:01:23 [INFO] Empty DMap fragment (kind: Primary) has been deleted: dmap.bench on PartID: 3 => janitor.go:65
2022/04/23 14:01:23 [INFO] Empty DMap fragment (kind: Primary) has been deleted: dmap.bench on PartID: 4 => janitor.go:65
2022/04/23 14:01:23 [INFO] Empty DMap fragment (kind: Primary) has been deleted: dmap.bench on PartID: 5 => janitor.go:65
2022/04/23 14:01:23 [INFO] Empty DMap fragment (kind: Primary) has been deleted: dmap.bench on PartID: 6 => janitor.go:65
2022/04/23 14:01:23 [INFO] Empty DMap fragment (kind: Primary) has been deleted: dmap.bench on PartID: 7 => janitor.go:65
2022/04/23 14:01:23 [INFO] Empty DMap fragment (kind: Primary) has been deleted: dmap.bench on PartID: 8 => janitor.go:65
2022/04/23 14:01:23 [INFO] Empty DMap fragment (kind: Primary) has been deleted: dmap.bench on PartID: 9 => janitor.go:65
...

When you try to stop the server:
...
^C2022/04/23 14:09:44 [INFO] Signal catched: interrupt
2022/04/23 14:09:44 [INFO] Awaiting for background tasks
2022/04/23 14:09:44 [INFO] Press CTRL+C or send SIGTERM/SIGINT to quit immediately
2022/04/23 14:09:44 [ERROR] Failed to acquire semaphore: context canceled => eviction.go:82
2022/04/23 14:09:44 [ERROR] Failed to acquire semaphore: context canceled => compaction.go:82
2022/04/23 14:09:44 [ERROR] Failed to acquire semaphore: context canceled => compaction.go:82
2022/04/23 14:09:44 [ERROR] Failed to acquire semaphore: context canceled => compaction.go:82
2022/04/23 14:09:44 [ERROR] Failed to acquire semaphore: context canceled => compaction.go:82
2022/04/23 14:09:44 [ERROR] Failed to acquire semaphore: context canceled => compaction.go:82
2022/04/23 14:09:44 [ERROR] Failed to acquire semaphore: context canceled => compaction.go:82
2022/04/23 14:09:44 [ERROR] Failed to acquire semaphore: context canceled => compaction.go:82
2022/04/23 14:09:44 [ERROR] Failed to acquire semaphore: context canceled => compaction.go:82
2022/04/23 14:09:44 [ERROR] Failed to acquire semaphore: context canceled => compaction.go:82
2022/04/23 14:09:44 [ERROR] Failed to acquire semaphore: context canceled => compaction.go:82
2022/04/23 14:09:44 [ERROR] Failed to acquire semaphore: context canceled => compaction.go:82
2022/04/23 14:09:44 [ERROR] Failed to acquire semaphore: context canceled => compaction.go:82
2022/04/23 14:09:44 [ERROR] Failed to acquire semaphore: context canceled => compaction.go:82
2022/04/23 14:09:44 [ERROR] Failed to acquire semaphore: context canceled => compaction.go:82
2022/04/23 14:09:44 [ERROR] Failed to acquire semaphore: context canceled => compaction.go:82
2022/04/23 14:09:44 [ERROR] Failed to acquire semaphore: context canceled => compaction.go:82
...

I checked every possible component and tried to find a real problem in DMaps and its compaction implementation. Everything seems okay.
ERROR: Failed to acquire semaphore: context canceled:
This message is quite normal because a semaphore limits the number of concurrent compaction workers, and there are some worker goroutines in the background who wait for their turn. I filtered the context cancelled message.
I used the following code snippet to create a load on a single Olric node. It creates random keys and inserts them into the cluster. I couldn't detect any missing key. It seems that memtier_benchmark somehow triggers DMap fragment creation, but it doesn't insert any keys. It looks all fine.
package main

import (
	"context"
	"crypto/rand"
	"encoding/hex"
	"fmt"
	"log"
	"sync"
	"time"

	"github.com/buraksezer/olric"
	"golang.org/x/sync/semaphore"
)

func main() {
	c, err := olric.NewClusterClient([]string{"localhost:3320"})
	if err != nil {
		log.Fatalf("olric.NewClusterClient returned an error: %v", err)
	}

	dm, err := c.NewDMap("bucket-of-arbitrary-items")
	if err != nil {
		log.Fatalf("olric.NewDMap returned an error: %v", err)
	}

	ctx, cancel := context.WithCancel(context.Background())

	var mtx sync.Mutex
	keys := make(map[string]struct{})

	var wg sync.WaitGroup
	sem := semaphore.NewWeighted(50)

	for i := 0; i < 1000000; i++ {

		err = sem.Acquire(ctx, 1)
		if err != nil {
			log.Println(err)
			continue
		}

		wg.Add(1)
		go func(idx int) {
			defer wg.Done()
			defer sem.Release(1)

			token := make([]byte, 16)
			_, err = rand.Read(token)
			if err != nil {
				log.Fatalf("Failed to read random bytes: %v", err)
			}

			key := hex.EncodeToString(token)
			err = dm.Put(ctx, key, fmt.Sprintf("Olric Rocks-%d", idx))
			if err != nil {
				log.Fatalf("Failed to call Put: %v", err)
			}

			mtx.Lock()
			keys[key] = struct{}{}
			mtx.Unlock()

			if idx%100000 == 0 {
				fmt.Println("PUT ==>", idx)
			}
		}(i)
	}

	wg.Wait()

	mtx.Lock()
	fmt.Println("Number of unique keys", len(keys))
	mtx.Unlock()

	for key, _ := range keys {
		gr, err := dm.Get(ctx, key)
		if err != nil {
			log.Fatalf("Failed to call Get: %v", err)
		}

		_, err = gr.String()
		if err != nil {
			log.Fatalf("Failed to read Get response: %v", err)
		}
	}

	ctx, cancel = context.WithTimeout(context.Background(), 10*time.Second)
	defer cancel()

	err = c.Close(ctx)
	if err != nil {
		log.Printf("Failed to close ClusterClient: %v", err)
	}
}


SSR explodes

This runs for a while
% clj -Sdeps '{:deps {io.github.nextjournal/clerk {:mvn/version "0.13.842"}}}'
Clojure 1.11.1
user=> ((requiring-resolve 'nextjournal.clerk/build!) {:paths ["Sync/src/ir-notebook.clj"] :ssr true :output-path "/home/kevin/.local/tmp"})
👷🏼 Clerk is building 1 notebooks…
🧐 Parsing… Done in 492.096ms. ✅
🔬 Analyzing… Done in 946.555ms. ✅
🔨 Building "Sync/src/ir-notebook.clj"… Done in 389.608ms. ✅
🧱 Server Side Rendering…

Then spits out what seems to be the rendering of the notebook as html as part of an error message that ends with:
(node:874841) ExperimentalWarning: Network Imports is an experimental feature and might change at any time
    at emitExperimentalWarning (node:internal/util:273:11)
    at new DefaultModuleLoader (node:internal/modules/esm/loader:109:7)
    at createModuleLoader (node:internal/modules/esm/loader:424:10)
    at loadESM (node:internal/process/esm_loader:18:19)
    at evalModule (node:internal/process/execution:51:28)
    at node:internal/main/eval_stdin:29:5
    at Socket.<anonymous> (node:internal/process/execution:205:5)
    at Socket.emit (node:events:523:35)
    at endReadableNT (node:internal/streams/readable:1367:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
https://storage.clerk.garden/nextjournal/clerk-assets@27N1QPbuHKe9U4oeTAJPmQ69pphG/viewer.js?immutable=true:1928
nb.redirected&&!U.has(nb.url)&&U.set(nb.url,ab);return nb.json()}));return ab}function v(Ya){function ab(Kb){if("string"!==typeof Kb)return Kb;let Ub=hc.get(Kb);Ub||hc.set(Kb,Ub=new Promise((ya,fb)=>{const Ga=document.createElement("script");Ga.onload=()=>{try{ya(W.pop()(nb(Kb)))}catch(ob){fb(new Sa("invalid module"))}Ga.remove()};Ga.onerror=()=>{fb(new Sa("unable to load module"));Ga.remove()};Ga.async=!0;Ga.src=Kb;window.define=T;document.head.appendChild(Ga)}));return Ub}function nb(Kb){return Ub=>
                                                                                ^

ReferenceError: document is not defined
    at https://storage.clerk.garden/nextjournal/clerk-assets@27N1QPbuHKe9U4oeTAJPmQ69pphG/viewer.js?immutable=true:1928:210
    at new Promise (<anonymous>)
    at ab (https://storage.clerk.garden/nextjournal/clerk-assets@27N1QPbuHKe9U4oeTAJPmQ69pphG/viewer.js?immutable=true:1928:179)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Node.js v20.2.0
 1: 0x7f1d0a87a17c node::Abort() [/lib64/libnode.so.115]
 2: 0x7f1d0a88d2f2  [/lib64/libnode.so.115]
 3: 0x7f1d0beba8fa v8::internal::FunctionCallbackArguments::Call(v8::internal::CallHandlerInfo) [/lib64/libnode.so.115]
 4: 0x7f1d0bebae58  [/lib64/libnode.so.115]
 5: 0x7f1d0bebb644 v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) [/lib64/libnode.so.115]
 6: 0x7f1d0bd28df6  [/lib64/libnode.so.115]

looks like it is the doc :/ the ssr.clj above renders

I removed a vega-lite chart and now it renders

ReferenceError: document is not defined

All viewers which manipulate DOM nodes as vegalite or plotly aren't suitable for SSR builds because we're generating html using node at the moment. We should at least try to make it clear in a more informative error?

@hiredman If you need pre-rendering of vega-lite plots for a SSR context, the darkstar library uses GraalJS to convert a vega/vega-lite spec to SVG. I think it would be a suitable workaround for your use case.

I guess not all JS libs are suited for SSR and this isn't something clerk can fix, probably. Good to close @mk or want to keep this open?

Unless we're willing to support multiple SSR backends, like a headless browser for example. Could be worth exploring.


AutoSuggestBox inside NavigationView truncates text when text scaling applied

Describe the bug
When using an AutoSuggestBox inside a NavigationView (using the NavigationView.AutoSuggestBox property) and you increase the text size through the settings app, the AutoSuggestBox stays the same height and the text gets truncated at the bottom.
The AutoSuggestBox when used on its own does increase in height when you apply text scaling, so that all the text fits in the box.
The behavior of the AutoSuggestBox inside the NavigationView should be fixed so that when text scaling is applied, the box expands.  This will help low visibility users not get confused by truncated text.
Steps to reproduce the bug

Create a blank UWP
Add a NavigationView with an AutoSuggestBox
Launch the app and type some text in the box
Go to the Settings app > Accessibility > Text size and increase to 225%
Observe that the AutoSuggestBox does not grow enough in height and the text in the box gets truncated

Expected behavior
When using just an AutoSuggestBox (not inside a NavigationView) increasing the text size also increases the height of the AutoSuggestBox so that the full text fits in the box.  The AutoSuggestBox should also do this when placed inside a NavigationView
Screenshots
This shows an AutoSuggestBox inside a NavigationView truncating the bottom of the text, while an AutoSuggestBox by itself does not truncate the bottom.

NuGet package version
No response
Windows app type

[X] UWP
[ ] Win32

Device form factor
Desktop
Windows version
No response
Additional context
No response

NavigationView's template should allow for the Autosuggest box to expand vertically in this scenario.

@karenbtlai FYI


Remove deterministic argument from som function in pyFlowSOM

What is the purpose of this PR?
Because som will eventually not support deterministic, it's best that we remove it in advance. We can merge in this PR once that update is made in pyFlowSOM.
How did you implement your changes
Remove the deterministic flag in cluster_helpers.py. This will eventually be replaced by seed, which can either be its own PR or added to this PR.

This one will depend on angelolab/pyFlowSOM/#10 and a new pyFlowSOM release being made.

New pyFlowSOM release has been made. This PR will also add an explicit way to change the seed in the call to pyFlowSOM.som.


catalyst: Rename to catalyst-browser

Renames the catalyst package to catalyst-browser to be more consistent with upstream changes and other software repositories.

[x] I have read the Contributing Guide.

Recreated PR because the old one was broken.


An exception has been thrown during the rendering of a template ("The partial 'recaptcha' is not found.").

              <textarea id="comments" name="comments" rows="8" cols="80"></textarea>

    {% partial 'recaptcha' %}

          <button id="simpleContactSubmitButton" type="submit" class="btn btn-default">Submit</button>

      </form>

Can you please help me with this problem ?
I am getting An exception has been thrown during the rendering of a template ("The partial 'recaptcha' is not found."). error.
I did tried finding many things but i didn't worked out.

Hello @ajinkyakambe

which build are you using?
if you copy recaptcha.htm from component to your partial, the problem is solved?
can you try this code?:

{% partial __SELF__ ~ '::recaptcha' %}

1] I am using the latest build.
2] I tried copying the recaptcha.htm from component to my theme, the problem is solved but I cannot see the ReCaptcha on the front end. When I saw the front end, I saw this

--
  |
  |
  |
  |
  | Submit
  |
  |
3] I tried using this code {% partial SELF ~ '::recaptcha' %},
But it doesn't work.
Still, i see.
An exception has been thrown during the rendering of a template ("The partial 'recaptcha' is not found.").
H:\root\home\shelloman45-001\www\site3/themes/asaffa/pages/business-enquiries.htm line 167
My server is windows one.

It's possible for you to test on a linux machine?
I don't have access to a windows pc to test if it's working ...

I have the same problem on OSX

Place component on page
Expand component via October texteditor
Get this error

Didn't have this error when just drag n dropping it on the page


Sub Groups and prefixes

When adding a player to 2 groups, only one prefix shows.

Working as intended.
PS - It's the chat management plugin which displayed prefixes and suffixes in-game, not the permissions plugin.  ;-)


⚠ SceneTime has degraded performance

In 8ac9240, SceneTime ($ST) experienced degraded performance:

HTTP code: 200
Response time: 1762 ms

Resolved: SceneTime performance has improved in e969244 after 6 minutes.


Deploy a new non-snapshot release

This commit https://github.com/owainlewis/yaml/commit/7341c5a6a293707a4858c9c23522130b343aad84 fixes CVE-2017-18640
It would be good to deploy a new non-snapshot release that contains this fix.
Many thanks.

Please? 🙏

Bump?


Queue Manager: panic: close of closed channel

Bug Report
What did you do?
Ran prometheus with remote write enabled.
What did you expect to see?
No panic
What did you see instead? Under which circumstances?
Got a panic for close of closed channel .  I suspect it occurs because (t *QueueManager) Stop() is called in the middle of (t *QueueManager) reshardLoop().
Environment

System information:
Linux 4.15.0-1037-azure x86_64

Prometheus version:
prometheus, version 2.8.1 (branch: remote-write-fix, revision: 4a913999f7e6f50ac8186f589561498c07e6dd53)
build user:       user@machine
build date:       20190405-17:39:10
go version:       go1.12.1

This was a build of PR #5429 and Brian's TSDB changes 1099999.  I'm pretty sure this issue is present in master.

Prometheus configuration file:

remote_write:
  - url: "http://HOST:7201/api/v1/prom/remote/write"
    queue_config:
      capacity: 100000
      min_shards: 5
      max_retries: 10

Logs:

Apr 06 14:39:49 my-host prometheus[19149]: ts=2019-04-06T14:39:49.5887504Z caller=dedupe.go:111 component=remote level=info queue=0:http://<IP_ADDRESS>:7201/api/v1/prom/remote/w
Apr 06 14:39:49 my-host prometheus[19149]: panic: close of closed channel
Apr 06 14:39:49 my-host prometheus[19149]: goroutine 183 [running]:
Apr 06 14:39:49 my-host prometheus[19149]: github.com/prometheus/prometheus/storage/remote.(*shards).stop(0xc03ee227e0)
Apr 06 14:39:49 my-host prometheus[19149]:         /Users/my.user/Work/github/prometheus/storage/remote/queue_manager.go:585 +0x59
Apr 06 14:39:49 my-host prometheus[19149]: github.com/prometheus/prometheus/storage/remote.(*QueueManager).Stop(0xc06a8da340)
Apr 06 14:39:49 my-host prometheus[19149]:         /Users/my.user/Work/github/prometheus/storage/remote/queue_manager.go:325 +0x236
Apr 06 14:39:49 my-host prometheus[19149]: github.com/prometheus/prometheus/storage/remote.(*Storage).ApplyConfig(0xc0001525b0, 0xc0020d6270, 0x0, 0x0)
Apr 06 14:39:49 my-host prometheus[19149]:         /Users/my.user/Work/github/prometheus/storage/remote/storage.go:106 +0x3f4
Apr 06 14:39:49 my-host prometheus[19149]: main.reloadConfig(0x7ffe37d62d0b, 0x2e, 0x24df6e0, 0xc0007f0270, 0xc0002d0b80, 0x7, 0x7, 0x0, 0x0)
Apr 06 14:39:49 my-host prometheus[19149]:         /Users/my.user/Work/github/prometheus/cmd/prometheus/main.go:742 +0x285
Apr 06 14:39:49 my-host prometheus[19149]: main.main.func13(0xc000129770, 0x125)
Apr 06 14:39:49 my-host prometheus[19149]:         /Users/my.user/Work/github/prometheus/cmd/prometheus/main.go:565 +0x4ed
Apr 06 14:39:49 my-host prometheus[19149]: github.com/oklog/oklog/pkg/group.(*Group).Run.func1(0xc0002e8180, 0xc0002e80c0, 0xc000252030)
Apr 06 14:39:49 my-host prometheus[19149]:         /Users/my.user/Work/github/prometheus/vendor/github.com/oklog/oklog/pkg/group/group.go:38 +0x27
Apr 06 14:39:49 my-host prometheus[19149]: created by github.com/oklog/oklog/pkg/group.(*Group).Run
Apr 06 14:39:49 my-host prometheus[19149]:         /Users/my.user/Work/github/prometheus/vendor/github.com/oklog/oklog/pkg/group/group.go:37 +0xbe
Apr 06 14:39:50 my-host systemd[1]: prometheus.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Apr 06 14:39:50 my-host systemd[1]: prometheus.service: Failed with result 'exit-code'.
Apr 06 14:41:20 my-host systemd[1]: prometheus.service: Service hold-off time over, scheduling restart.
Apr 06 14:41:20 my-host systemd[1]: prometheus.service: Scheduled restart job, restart counter is at 2.
Apr 06 14:41:20 my-host systemd[1]: Stopped Prometheus TSDB service.
Apr 06 14:41:20 my-host systemd[1]: Started Prometheus TSDB service.

@bismarck do you have the logs from before the logs you posted? was there a reshard or config reload taking place?

@cstyan Looks like both a reshard and reload took place:
Apr 06 14:38:09 my-hostprometheus[19149]: ts=2019-04-06T14:38:09.4844057Z caller=dedupe.go:111 component=remote level=info queue=0:http://<IP_ADDRESS>:7201/api/v1/prom/remote/write msg="Remote storage resharding" from=28 to=1
Apr 06 14:38:39 my-hostprometheus[19149]: ts=2019-04-06T14:38:39.4844581Z caller=dedupe.go:111 component=remote level=info queue=0:http://<IP_ADDRESS>:7201/api/v1/prom/remote/write msg="Remote storage resharding" from=1 to=26
Apr 06 14:38:49 my-hostprometheus[19149]: ts=2019-04-06T14:38:49.4843317Z caller=dedupe.go:111 component=remote level=info queue=0:http://<IP_ADDRESS>:7201/api/v1/prom/remote/write msg="Remote storage resharding" from=26 to=1
Apr 06 14:38:59 my-hostprometheus[19149]: ts=2019-04-06T14:38:59.4844702Z caller=dedupe.go:111 component=remote level=info queue=0:http://<IP_ADDRESS>:7201/api/v1/prom/remote/write msg="Remote storage resharding" from=1 to=32
Apr 06 14:39:09 my-hostprometheus[19149]: ts=2019-04-06T14:39:09.4843355Z caller=dedupe.go:111 component=remote level=info queue=0:http://<IP_ADDRESS>:7201/api/v1/prom/remote/write msg="Remote storage resharding" from=32 to=1
Apr 06 14:39:19 my-hostprometheus[19149]: ts=2019-04-06T14:39:19.4843703Z caller=dedupe.go:111 component=remote level=info queue=0:http://<IP_ADDRESS>:7201/api/v1/prom/remote/write msg="Remote storage resharding" from=1 to=28
Apr 06 14:39:29 my-hostprometheus[19149]: ts=2019-04-06T14:39:29.4845374Z caller=dedupe.go:111 component=remote level=info queue=0:http://<IP_ADDRESS>:7201/api/v1/prom/remote/write msg="Remote storage resharding" from=28 to=1
Apr 06 14:39:49 my-hostprometheus[19149]: level=info ts=2019-04-06T14:39:49.5201524Z caller=main.go:724 msg="Loading configuration file" filename=/etc/prometheus/prometheus-platform-config.yml
Apr 06 14:39:49 my-hostprometheus[19149]: ts=2019-04-06T14:39:49.5252961Z caller=dedupe.go:111 component=remote level=info queue=0:http://<IP_ADDRESS>:7201/api/v1/prom/remote/write msg="Stopping remote storage..."
Apr 06 14:39:49 my-hostprometheus[19149]: ts=2019-04-06T14:39:49.5887504Z caller=dedupe.go:111 component=remote level=info queue=0:http://<IP_ADDRESS>:7201/api/v1/prom/remote/write msg="Remote storage stopped."
Apr 06 14:39:49 my-hostprometheus[19149]: panic: close of closed channel
Apr 06 14:39:49 my-hostprometheus[19149]: goroutine 183 [running]:
``

We adjusted min_shards in the remote write config from 1 to 5 and the panics haven't occurred since.


support Oracle DB in reverse engineering dialog

support Oracle DB in reverse engineering dialog, as I did not find it after choosing any app server.

If Oracle DB is registered here [Services > Database ] then it should be visible in DB reverse engineering dialog.


Add Build Status section to README.md

Codecov Report

Merging #127 into master will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #127   +/-   ##
=========================================
  Coverage     64.34%   64.34%
  Complexity      225      225
=========================================
  Files            17       17
  Lines           892      892
  Branches        105      105
=========================================
  Hits            574      574
  Misses          243      243
  Partials         75       75

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cb3b59b...f3f1add. Read the comment docs.


setup codecov.io

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@573ca2c). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master       #4   +/-   ##
=========================================
  Coverage          ?   50.75%
=========================================
  Files             ?       13
  Lines             ?     1521
  Branches          ?      307
=========================================
  Hits              ?      772
  Misses            ?      669
  Partials          ?       80

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 573ca2c...487e5d7. Read the comment docs.


Add protect link button switch

Fixes #112
Also slightly refactors code (removes redundancies)

@pyplacca Thank you for the contribution!! 🥳🥳🥳

@all-contributors add @pyplacca for Code


Provide authors/contributors information

Hey @mayalubecks
I had a look into the datasets and I am excited about its content. Since one of the main goals of the openwashdata project is to give credit to all contributors to a dataset, we would like you to add the author information for the datasets to the repository.
You can do it in a similar way as you did with the dictionary.
Step 1
You can download the authors.xlsx file from the following link:
https://github.com/openwashdata/cbssuitabilityhaiti/raw/main/data-raw/authors.xlsx
Step 2
Open the file on your computer and fill in the rows with the authors information:

first_name, given names
last_name, last names
email
role, provide one or more three-letter codes specifying the role of the contributor.
These are the most common ones:
cre: creator or maintainer, maintains the package and is the contact person
aut: authors, made significant contributions
ctb: contributors, made smaller contributions
dtc: data contributor,  contributed data sets
fnd: funder
orcid, add the authors/contributors Orcid number so that the data package can be linked automatically later on.

There are two examples already in the spread sheet.
Step 3
Upload the completed authors.xlsx file back into the data-raw folder using the same steps as in Issue #1.

Please feel free to add all known contributors, also the people who conducted/supervised a survey/interview or collected spatial data points for example. This way, everybody can get acknowledged and have a reference to show.
If you have any questions please go ahead and contact me.
Thank you very much for working through this with us!

If you would like to see how the authors are currently presented, please go to the page with the drafted documentation or to an already published package page. You may also like to click cite cbssuitabilityhaiti.

@sebastian-loos I just uploaded the authors sheet with added info for myself and Sasha Kramer. Unfortunately I don't have specific author names for the creators fo the datasets however I do have specific text that is supposed to be used for crediting the data:
World Bank flood extent data should be attributed with "Plan d’Élaboration propre basée sur les données de la Banque Mondiale: Inondation de Récurrence (5 ans / 25 ans / 100 ans)"
Sanitation zoning assessment data should be attributed with "These data were developed under the USAID Water and Sanitation Project in collaboration with the Cap-Haitian municipal government and DINEPA. These data do not reflect the opinion of USAID or the US Government."
I'm not sure how we can add these credits within the same format as the excel sheet you provided?
Thanks!

Thanks a lot @mayalubecks for providing these details.
@sebastian-loos: The two additional references can be added to the documentation of the individual data sets and also shown on the README.rmd. An example that I often use is the palmerpenguins package. It's well done there:

README.rmd: https://github.com/allisonhorst/palmerpenguins/blob/c19a904462482430170bfe2c718775ddb7dbb885/README.Rmd#L201
penguins.R roxygen @source entry: https://github.com/allisonhorst/palmerpenguins/blob/c19a904462482430170bfe2c718775ddb7dbb885/R/penguins.R#L18

Thanks a lot @mayalubecks!
I did go ahead and included your added authors information to all the package documentation files. Do you and Sasha also have a Orcid ID? We will publish the data set package via Zenodo and it will get its own doi. It would help people to know your work and since it will show an additional publication in your Orcid account:)
You mentioned the World Bank flood extent data that can be found on humdata.org, right?
@larnsce I went through the commit history and saw that it is the dataset you removed in the beginning of the package making, do you remember why we did that? @mayalubecks I took a look into the data (code, images), what does it show exactly?
I included the accreditation text for the Sanitation zoning assessment data (okap). @larnsce can you check if that's the right way please?

README.Rmd
okap.R

I did not do it for the World Bank flood extent data since the data is not included in the package at the moment (can be discussed).
@mayalubecks Is there something I have to take into account for the mwater data ?
Thanks a lot for all your help!

R data package is now complete and available at: https://openwashdata.github.io/cbssuitabilityhaiti/
ORCIDs of @mayalubecks and Sasha could still be added if provided.


Could we reconsider involved the spark history server  in this repo?

background:
[#164] issue
reason:

Spark history server has not a stable available chart since helm/chart repo  archived
As a user of spark-on-k8s-operator, there is a very high probability that Spark history server is needed, because the webUI of spark diver will be inaccessible after the spark executor is completed.

Of course, if you have a better spark history visualization solution, you can also provide it. I really did not find this information in the document.
What do you think?

There is an online plan：
https://github.com/datamechanics/delight

There is an online plan：
https://github.com/datamechanics/delight

There is an online plan：
https://github.com/datamechanics/delight

Thanks for your solution.
This is a great spark history visualization solution.(The visual style is very modern and the configuration is simple and easy to use)
But its limitations are also obvious. The visualization page can only be used online, and cannot be deployed by ourselves, so it is not suitable for our scenario.

You could write your own deployment and run history server using ./sbin/start-history-server.sh, I have located an alternative hosting of the charts at https://artifacthub.io/packages/helm/spot/spark-history-server which might help.

I am writing spark events to s3, so I build a new docker container adding a couple jars and just change the entry point to run spark history server.
FROM gcr.io/spark-operator/spark:v3.1.1-hadoop3

USER root

ADD https://xxx.com/artifactory/apixio-spark/org/apache/hadoop/hadoop-aws/2.7.4/hadoop-aws-2.7.4.jar $SPARK_HOME/jars/
ADD https://xxx.com/artifactory/apixio-spark/com/amazonaws/aws-java-sdk-bundle/<IP_ADDRESS>/aws-java-sdk-<IP_ADDRESS>.jar $SPARK_HOME/jars/

ENTRYPOINT bash /opt/spark/sbin/spark-daemon.sh start org.apache.spark.deploy.history.HistoryServer 1

Then just a deployment yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: spark-hs-custom
    version: 3.1.1
  name: spark-hs-custom
spec:
  replicas: 1
  selector:
    matchLabels:
      app: spark-hs-custom
      version: 3.1.1
  template:
    metadata:
      labels:
        app: spark-hs-custom
        version: 3.1.1
    spec:
      containers:
        - env:
          - name: SPARK_NO_DAEMONIZE
            value: "false"
          - name: SPARK_HISTORY_OPTS
            value: -Dspark.history.fs.logDirectory=s3a://my-bucket-name/eventLogFolder
          image: xxx.dkr.ecr.us-west-2.amazonaws.com/xxx-spark-hs:v0.0.4
          imagePullPolicy: IfNotPresent
          name: spark-hs-custom
          ports:
            - containerPort: 18080
              name: http
              protocol: TCP
          resources:
            requests:
              cpu: "2"
              memory: 10Gi
            limits:
              cpu: "2"
              memory: 10Gi

@stephbat Yea, I think I hit that issue as well when I was running as someone other than root. Google's base image uses the user 185, but I wasn't able to get it to work with that one. Can you post what you did if you figure out a solution?

yea, confirmed I get that exception when I try to do a USER 185
21/07/13 22:00:35 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Exception in thread "main" org.apache.hadoop.security.KerberosAuthException: failure to login: javax.security.auth.login.LoginException: java.lang.NullPointerException: invalid null input: name
        at jdk.security.auth/com.sun.security.auth.UnixPrincipal.<init>(Unknown Source)
        at jdk.security.auth/com.sun.security.auth.module.UnixLoginModule.login(Unknown Source)
        at java.base/javax.security.auth.login.LoginContext.invoke(Unknown Source)
        at java.base/javax.security.auth.login.LoginContext$4.run(Unknown Source)
        at java.base/javax.security.auth.login.LoginContext$4.run(Unknown Source)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at java.base/javax.security.auth.login.LoginContext.invokePriv(Unknown Source)
        at java.base/javax.security.auth.login.LoginContext.login(Unknown Source)
        at org.apache.hadoop.security.UserGroupInformation$HadoopLoginContext.login(UserGroupInformation.java:1926)
        at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1837)
        at org.apache.hadoop.security.UserGroupInformation.createLoginUser(UserGroupInformation.java:710)
        at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:660)
        at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:571)
        at org.apache.spark.util.Utils$.$anonfun$getCurrentUserName$1(Utils.scala:2476)
        at scala.Option.getOrElse(Option.scala:189)
        at org.apache.spark.util.Utils$.getCurrentUserName(Utils.scala:2476)
        at org.apache.spark.SecurityManager.<init>(SecurityManager.scala:79)
        at org.apache.spark.deploy.history.HistoryServer$.createSecurityManager(HistoryServer.scala:333)
        at org.apache.spark.deploy.history.HistoryServer$.main(HistoryServer.scala:294)
        at org.apache.spark.deploy.history.HistoryServer.main(HistoryServer.scala)

        at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1847)
        at org.apache.hadoop.security.UserGroupInformation.createLoginUser(UserGroupInformation.java:710)
        at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:660)
        at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:571)
        at org.apache.spark.util.Utils$.$anonfun$getCurrentUserName$1(Utils.scala:2476)
        at scala.Option.getOrElse(Option.scala:189)
        at org.apache.spark.util.Utils$.getCurrentUserName(Utils.scala:2476)
        at org.apache.spark.SecurityManager.<init>(SecurityManager.scala:79)
        at org.apache.spark.deploy.history.HistoryServer$.createSecurityManager(HistoryServer.scala:333)
        at org.apache.spark.deploy.history.HistoryServer$.main(HistoryServer.scala:294)
        at org.apache.spark.deploy.history.HistoryServer.main(HistoryServer.scala)
Caused by: javax.security.auth.login.LoginException: java.lang.NullPointerException: invalid null input: name
        at jdk.security.auth/com.sun.security.auth.UnixPrincipal.<init>(Unknown Source)
        at jdk.security.auth/com.sun.security.auth.module.UnixLoginModule.login(Unknown Source)
        at java.base/javax.security.auth.login.LoginContext.invoke(Unknown Source)
        at java.base/javax.security.auth.login.LoginContext$4.run(Unknown Source)
        at java.base/javax.security.auth.login.LoginContext$4.run(Unknown Source)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at java.base/javax.security.auth.login.LoginContext.invokePriv(Unknown Source)
        at java.base/javax.security.auth.login.LoginContext.login(Unknown Source)
        at org.apache.hadoop.security.UserGroupInformation$HadoopLoginContext.login(UserGroupInformation.java:1926)
        at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1837)
        at org.apache.hadoop.security.UserGroupInformation.createLoginUser(UserGroupInformation.java:710)
        at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:660)
        at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:571)
        at org.apache.spark.util.Utils$.$anonfun$getCurrentUserName$1(Utils.scala:2476)
        at scala.Option.getOrElse(Option.scala:189)
        at org.apache.spark.util.Utils$.getCurrentUserName(Utils.scala:2476)
        at org.apache.spark.SecurityManager.<init>(SecurityManager.scala:79)
        at org.apache.spark.deploy.history.HistoryServer$.createSecurityManager(HistoryServer.scala:333)
        at org.apache.spark.deploy.history.HistoryServer$.main(HistoryServer.scala:294)
        at org.apache.spark.deploy.history.HistoryServer.main(HistoryServer.scala)

        at java.base/javax.security.auth.login.LoginContext.invoke(Unknown Source)
        at java.base/javax.security.auth.login.LoginContext$4.run(Unknown Source)
        at java.base/javax.security.auth.login.LoginContext$4.run(Unknown Source)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at java.base/javax.security.auth.login.LoginContext.invokePriv(Unknown Source)
        at java.base/javax.security.auth.login.LoginContext.login(Unknown Source)
        at org.apache.hadoop.security.UserGroupInformation$HadoopLoginContext.login(UserGroupInformation.java:1926)
        at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1837)
        ... 10 more

This works for me:
FROM gcr.io/spark-operator/spark:v3.1.1-hadoop3

USER root

ADD https://xxx.com/artifactory/apixio-spark/org/apache/hadoop/hadoop-aws/2.7.4/hadoop-aws-2.7.4.jar $SPARK_HOME/jars/
ADD https://xxx.com/artifactory/apixio-spark/com/amazonaws/aws-java-sdk-bundle/<IP_ADDRESS>/aws-java-sdk-<IP_ADDRESS>.jar $SPARK_HOME/jars/

RUN groupadd -g 185 spark && \
    useradd -u 185 -g 185 spark

USER 185
ENTRYPOINT bash /opt/spark/sbin/spark-daemon.sh start org.apache.spark.deploy.history.HistoryServer 1

@jdonnelly-apixio @indranilr  thank for your solution.
In fact, I also use deployment to deploy hiss just like yours.
But what I want to say is that we can deploy Spark history server ourselves  is completely different from being included in this helm repo.
I think we should simply change the vaules.yaml to make his work well.

@nonpool yep, kind of agreed. it would be useful if the spark-operator supported the deployment of common services like spark history server, a hive metastore, prometheus server, etc.
can a helm chart install other helm charts (if not, not sure it would make sense to duplicate helm install functionality for stuff like prometheus server inside of the spark-operator helm chart and the official prometheus-community helm chart should probably be used instead)? not sure what best practices would be from a helm stand point... maybe just some additional documentation that shows or links to how to install some common useful additional services would be a good start

Hi @stephbat @indranilr @nonpool @jdonnelly-apixio @haolixu
I am as of now confused how to install Spark History Server on kubernetes after successful installation of the Spark Operator on Kubernetes?

@chetkhatri Here is my personal experience:

Build an image with the following Dockerfile:

ARG SPARK_IMAGE=gcr.io/spark-operator/spark:v3.1.1-hadoop3
FROM ${SPARK_IMAGE}

USER root

# Setup dependencies for Google Cloud Storage access.
RUN rm $SPARK_HOME/jars/guava-*.jar
ADD https://repo1.maven.org/maven2/com/google/guava/guava/27.0-jre/guava-27.0-jre.jar $SPARK_HOME/jars
RUN chmod 644 $SPARK_HOME/jars/guava-27.0-jre.jar
# Add the connector jar needed to access Google Cloud Storage using the Hadoop FileSystem API.
ADD https://storage.googleapis.com/hadoop-lib/gcs/gcs-connector-latest-hadoop3.jar $SPARK_HOME/jars
RUN chmod 644 $SPARK_HOME/jars/gcs-connector-latest-hadoop3.jar

USER ${spark_uid}

ENTRYPOINT ${SPARK_HOME}/sbin/start-history-server.sh

I've added some stuff related to GCP. Feel free to add your own for AWS/Azure whatever...
2. Create a new chart with helm create
3. Change the following in templates/deployment.yaml:

spec.template.spec.containers[0].ports.containerPort -> 18080
spec.template.spec.containers[0].env ->

env:
            - name: SPARK_NO_DAEMONIZE
              value: "false"
            - name: SPARK_CONF_DIR
              value: {{ .Values.config.sparkConfDirectory }}
            - name: SPARK_HISTORY_OPTS
              value: "-Dspark.history.fs.logDirectory={{ .Values.config.historyLogDirectory }}"

Add config section to values.yaml

config:
  sparkConfDirectory: "/opt/spark/conf"
  historyLogDirectory: ""

Fill values.yaml sections according to your needs (services, ingresses etc...)
Install your chart
Done
I hope I didn't forget anything

This works for me:
FROM gcr.io/spark-operator/spark:v3.1.1-hadoop3

USER root

ADD https://xxx.com/artifactory/apixio-spark/org/apache/hadoop/hadoop-aws/2.7.4/hadoop-aws-2.7.4.jar $SPARK_HOME/jars/
ADD https://xxx.com/artifactory/apixio-spark/com/amazonaws/aws-java-sdk-bundle/<IP_ADDRESS>/aws-java-sdk-<IP_ADDRESS>.jar $SPARK_HOME/jars/

RUN groupadd -g 185 spark && \
    useradd -u 185 -g 185 spark

USER 185
ENTRYPOINT bash /opt/spark/sbin/spark-daemon.sh start org.apache.spark.deploy.history.HistoryServer 1

awesome,this works， but i have no idea how does it works

Hi everyone, I've one question:
are you also able to collect driver/executor stdout/stderr logs that you can see through kubectl logs?

Has anyone tried to do this more recently? I'm trying with the apache/spark:3.5.0 container image and have lost a day to dependency hell. A Spark History helm chart in this repo or documentation on how to get this up and running would be very welcome indeed.

What is your securityContext set to on your deployment for the history server? In my case, I had the same issue as @jdonnelly-apixio here, but the real issue was that my securityContext was set to run as a different user.
The fix, in my case, was simply:
securityContext:
  runAsUser: 185

As previously it was set to run as user 1000. This fixed the issue for me.

Bump to keep alive as this would be useful.

/lifecycle frozen


Make sure we throw when an instance of an unmapped derived type is use

Regardless of whether inheritance is being mapped in the model, it is always possible that the runtime type of an instance passed to the context is actually an unmapped derived type. In EF6 we explicitly disallowed this, except that we made a special exception for proxies. The fact that we made this exception shows that sometimes using an unmapped derived type can be useful, so it maybe useful to allow this in EF7. However, doing so means that code cannot rely on a certain type, which may result in additional complexity and/or subtle bugs.

@smitpatel work with @ajcvickers  on this one

Make  we test what happens when you try to map a base type

Currently when an instance of unmapped derived type is used, we are not able to find the mapped EntityType in the model and we throw following exception
Microsoft.Data.Entity.Metadata.ModelItemNotFoundException: The entity type 'DerivedProduct' was not found. Ensure that the entity type has been added to the model.

Which looks good enough. Though there are no tests to cover this.
Clearing flags for re-triage.

@smitpatel can you add a test for this


Exported resources

This pull request allows rspec-puppet module spec tests to be written for classes which contain the exported resource collector <<| |>> by defining let(:exported_resources) arrays of resource hashes. It builds off of @rodjek's previous work, with some modifications to remove monkey-patching and work around rspec bugs.
This currently implements exporting resources for importing. It does not have the ability to check that resources were exported by the catalogue compile because the temporary database is wiped after each test case (but rspec-puppet caches the catalogue instead of compiling for each test case).
There is currently no way to inject virtual resources into a catalogue compile, so this functionality has not been carried on from the original commit by @rodjek.
This change is backwards compatible. Adds 26 spec tests and all tests pass.

@hunner @rodjek anybody willing to pick this back up, is there an interest in being able to mock this up for anybody else?


NA-0: Update raml-parser version to 1.0.14

As mentioned by @brentryan, please bump the version. Also, please add an entry to the CHANGELOG


Avoid an error in integration test because of a locale different of en-US

It aims to fix #13987.

/botio integrationtest

Thank you for improving this!


flutter 3.3  release package crash when enter delete key

build a  release package
input sth at EditableText,enter delete key

Logs
io.flutter.plugin.editing.a is mapping io.flutter.plugin.editing.InputConnectionAdaptor .
java.lang.AbstractMethodError: abstract method "boolean io.flutter.plugin.editing.a$a.a(android.view.KeyEvent)"
        at io.flutter.plugin.editing.a.sendKeyEvent(SourceFile:1)
        at com.android.internal.view.IInputConnectionWrapper.executeMessage(IInputConnectionWrapper.java:438)
        at com.android.internal.view.IInputConnectionWrapper$MyHandler.handleMessage(IInputConnectionWrapper.java:93)
        at android.os.Handler.dispatchMessage(Handler.java:106)
        at android.os.Looper.loop(Looper.java:236)
        at android.app.ActivityThread.main(ActivityThread.java:8060)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:656)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:967)

[✓] Flutter (Channel stable, 3.3.0, on macOS 12.5.1 21G83 darwin-arm, locale zh-Hans-CN)
    • Flutter version 3.3.0 on channel stable at /Users/moneky/codesoft/flutter-mac
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision ffccd96b62 (6 days ago), 2022-08-29 17:28:57 -0700
    • Engine revision 5e9e0e0aa8
    • Dart version 2.18.0
    • DevTools version 2.15.0
    • Pub download mirror https://pub.flutter-io.cn
    • Flutter download mirror https://storage.flutter-io.cn
                                                                                Checking Android licenses is taking an unexpectedly long time...[✓] Android toolchain - develop for Android devices (Android SDK version 32.1.0-rc1)
    • Android SDK at /Users/moneky/Library/Android/sdk
    • Platform android-32, build-tools 32.1.0-rc1
    • ANDROID_HOME = /Users/moneky/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 13.4.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 13F100
    • CocoaPods version 1.11.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)

[✓] Connected device (3 available)
    • Redmi K20 Pro (mobile)      • 90e28d73      • android-arm64  • Android 11 (API 30)
    • sdk gphone64 arm64 (mobile) • emulator-5554 • android-arm64  • Android 12 (API 32) (emulator)
    • Chrome (web)                • chrome        • web-javascript • Google Chrome 105.0.5195.52

[✓] HTTP Host Availability
    • All required HTTP hosts are available

• No issues found!

Hi @lzhAndroid, please provide a complete minimal reproducible code sample that reproduces the issue so that we can investigate this.
Thank you