please add more activation functions

System information

TensorFlow version (you are using): 2.0b1
Are you willing to contribute it (Yes/No): Yes

Describe the feature and the current behavior/state.
activation functions are easy to add and quite handy
Will this change the current api? How?
something like this:
import tensorflow as tf

K = tf.keras
B, L = K.backend, K.layers

RRELU_MIN, RRELU_MAX = 0.123, 0.314
HARD_MIN, HARD_MAX = -1., 1.
SOFT_ARGMAX_BETA = 1e10

fn_string = ["swish", "gaussian", "sin", "cos", "hard_tanh", "lisht", 'rrelu'] # etc etc
activators = [L.Activation(fn) for fn_string in fn_strings]

def clean_activation(activation):
    if callable(activation):
        return activation
    elif activation == 'soft_argmax':
        fn = soft_argmax
    elif activation == 'gaussian':
        fn = gaussian
    elif activation == 'swish':
        fn = swish
    elif activation == 'lisht':
        fn = lisht
    elif activation == 'sin':
        fn = tf.math.sin
    elif activation == 'cos':
        fn = tf.math.cos
    else:
        fn = activation
    return fn

def swish(x):
    """
    Searching for Activation Functions
    https://arxiv.org/abs/1710.05941
    """
    return (B.sigmoid(x) * x)

def soft_argmax(x, beta=SOFT_ARGMAX_BETA):
    """
    https://stackoverflow.com/questions/46926809/getting-around-tf-argmax-which-is-not-differentiable
    https://lucehe.github.io/differentiable-argmax/
    """
    x_range = tf.range(x.shape.as_list()[-1], dtype=x.dtype)
    return tf.math.reduce_sum(
        tf.nn.softmax(x * beta) * x_range, axis=-1)

def gaussian(x):
    return B.exp(-B.pow(x, 2))

def hard_tanh(x, min=HARD_MIN, max=HARD_MAX):
    if x > max:
        return max
    elif x < min:
        return min
    else:
        return x

def lisht(x):
    """
    LiSHT: Non-Parametric Linearly Scaled Hyperbolic Tangent
    https://github.com/swalpa/LiSHT
    """
    return (B.tanh(x) * x)

def rrelu(x, min=RRELU_MIN, max=RRELU_MAX):
    return x if x >= 0 else tf.random.uniform(min, max) * x

def tanhshrink(x):
    return x - B.tanh(x)

def hardshrink(x, min=HARD_MIN, max=HARD_MAX):
    if x > max:
        return x
    elif x < min:
        return min
    else:
        return 0

Who will benefit with this feature?
anybody who likes to play around
it would also be cool if we could set activations on a per-neuron basis instead of a per-layer basis. or per-channel, potentially... shrug!
Any Other info.
might be better suited for addons... cc @seanpmorgan

/cc @seanpmorgan for extended ops.

We would be happy to include most of these in Addons.

@bionicles Can you open a PR in addons repo here and close the issue here. Thanks!

@jvishnuvardhan @yongtang @seanpmorgan done, closed, here is updated code just for future searchers:
from tensorflow_addons.activations import sparsemax
import tensorflow as tf

K = tf.keras

B, L = K.backend, K.layers

RRELU_MIN, RRELU_MAX = 0.123, 0.314
HARD_MIN, HARD_MAX = -1., 1.
SOFT_ARGMAX_BETA = 1e10
FN = 'relu'

def swish(x):
    """
    Searching for Activation Functions
    https://arxiv.org/abs/1710.05941
    """
    return (B.sigmoid(x) * x)

def soft_argmax(x, beta=SOFT_ARGMAX_BETA):
    """
    https://stackoverflow.com/questions/46926809/getting-around-tf-argmax-which-is-not-differentiable
    https://lucehe.github.io/differentiable-argmax/
    """
    x_range = tf.range(x.shape.as_list()[-1], dtype=x.dtype)
    return tf.math.reduce_sum(
        tf.nn.softmax(x * beta) * x_range, axis=-1)

def gaussian(x):
    return B.exp(-B.pow(x, 2))

def hard_tanh(x, min=HARD_MIN, max=HARD_MAX):
    if x > max:
        return max
    elif x < min:
        return min
    else:
        return x

def hard_lisht(x, min=HARD_MIN, max=HARD_MAX):
    if x < min or x > max:
        return max
    else:
        return tf.math.abs(x)

def lisht(x):
    """
    LiSHT: Non-Parametric Linearly Scaled Hyperbolic Tangent
    https://github.com/swalpa/LiSHT
    """
    return (B.tanh(x) * x)

def rrelu(x, min=RRELU_MIN, max=RRELU_MAX):
    return x if x >= 0 else tf.random.uniform(min, max) * x

def tanh_shrink(x):
    return x - B.tanh(x)

def hard_shrink(x, min=HARD_MIN, max=HARD_MAX):
    if x > max:
        return x
    elif x < min:
        return min
    else:
        return 0

FUNCTION_LOOKUP = {
    'soft_argmax': soft_argmax,
    'log_softmax': tf.nn.log_softmax,
    'sparsemax': sparsemax,
    'hard_lisht': hard_lisht,
    'hard_shrink': hard_shrink,
    'tanh_shrink': tanh_shrink,
    'hard_lisht': hard_lisht,
    'hard_tanh': hard_tanh,
    'gaussian': gaussian,
    'swish': swish,
    'lisht': lisht,
    'rrelu': rrelu,
    'lrelu': tf.nn.leaky_relu,
    'crelu': tf.nn.crelu,
    'relu6': tf.nn.relu6,
    'sin': tf.math.sin,
    'cos': tf.math.cos,
}

def clean_activation(activation):
    if callable(activation):
        return activation
    else:
        fn = activation
    return fn

def use_fn(fn):
    fn = clean_activation(fn)
    return L.Activation(fn)

Why not add the trainable version of Swish as well as the non-trainable version? keras-contrib has the code for the trainable version right here: keras-contrib.layers.avanced_activations.swish.Swish. Adding the trainable version also gives access to the beta scaling factor, which makes Swish much more flexible.


Add link to decorators proposal

Q
A

Bug fix?
no

Breaking change?
no

New feature?
no

Deprecations?
no

Spec compliancy?
no

Tests added/pass?
no

Fixed tickets

License
MIT

👍  thanks!


Footnotes have bad contrast

I noticed that footnote numbers are very hard to read. Could those numbers use a different color?
Light mode

Dark mode

FTR: This is caused by a:link/a:visited having higher specificity than a naked a.

This is fixed and will be in the next beta build.


[Bug]: Double scroll bar in the website

Is there an existing issue for this?

[X] I have searched the existing issues

Describe the bug
A double scroll bar appears on the page due to overflow issues, likely caused by an extra layer or component setting an unintended scroll behavior. This creates a secondary scroll bar, which affects the overall UX by adding unnecessary space and redundant scrolling.
https://github.com/user-attachments/assets/487cf780-6a94-4ab5-9211-91c50cddae14
Expected behavior
Only one scroll bar should appear, allowing users to scroll through the page without redundancy.
Add ScreenShots/Videos

On which device are you experiencing this bug?
Chrome, Windows
Record

[X] I have read the Contributing Guidelines
[X] I'm a GSSOC'24 Extended contributor
[X] I'm a Hacktoberfest'24  contributor
[X] I have starred the repository

Kindly assign me this task under Hacktoberfest2024, Thanku.

I have requested for PR. Kindly review it.

As per Rules i Requested first, please @Shane-3 try another one all the best


Create unirv.txt

Register of Universidade de Rio Verde
http://www.unirv.edu.br/
http://www.unirv.edu.br/graduacao_curso.php?id=42

@Peronbombom unirv.txt: this request has an incorrect file path format, please ensure that you are familiar with     this repository rules located at the bottom of this page and change     your request accordingly. If you don't know how to change request, close it and open a new one.     If you are not able to make the pull request according the instructions, please send us an email to<EMAIL_ADDRESS>Please mention this request URL in the message.

@Peronbombom unirv.txt: this request has an incorrect file path format, please ensure that you are familiar with     this repository rules located at the bottom of this page and change     your request accordingly. If you don't know how to change request, close it and open a new one.     If you are not able to make the pull request according the instructions, please send us an email to<EMAIL_ADDRESS>Please mention this request URL in the message.

@Peronbombom unirv.txt: this request has an incorrect file path format, please ensure that you are familiar with     this repository rules located at the bottom of this page and change     your request accordingly. If you don't know how to change request, close it and open a new one.     If you are not able to make the pull request according the instructions, please send us an email to<EMAIL_ADDRESS>Please mention this request URL in the message.

@Peronbombom unirv.txt: this request has an incorrect file path format, please ensure that you are familiar with     this repository rules located at the bottom of this page and change     your request accordingly. If you don't know how to change request, close it and open a new one.     If you are not able to make the pull request according the instructions, please send us an email to<EMAIL_ADDRESS>Please mention this request URL in the message.

@Peronbombom Pull request merged. Thank you! You did not name the file in the request correctly, thus I  did it for you. Please wait at least two hours after this message is sent, before start requesting the licenses at https://jetbrains.com/student.

OK! Thank you for correcting the name for me! I'm already waiting for the acception of my studant license request.
Thank you again!

De: Philip Torchinsky<EMAIL_ADDRESS>Enviado: segunda-feira, 11 de maio de 2020 08:01
Para: JetBrains/swot<EMAIL_ADDRESS>Cc: Peronbombom<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Assunto: Re: [JetBrains/swot] Create unirv.txt (#8255)
@Peronbombomhttps://github.com/Peronbombom Pull request merged. Thank you! You did not name the file in the request correctly, thus I did it for you. Please wait at least two hours after this message is sent, before start requesting the licenses at https://jetbrains.com/student.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/JetBrains/swot/pull/8255#issuecomment-626603513, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AKLQGVXRSENAR63JG56HCITRQ7EIRANCNFSM4M3YR6DA.


Is it possible to add custom operators with custom evaluation functions?

Discussed in https://github.com/react-querybuilder/react-querybuilder/discussions/829

Originally posted by Lemour-sudo December 10, 2024
Is it possible to add custom operators with custom evaluation functions:
<QueryBuilder
    operators={[
        {
            name: 'strlen',
            label: 'strlen',
            value: 'strlen',
            evaluate: (value: string) => value.length,
        },
        {
            name: 'isOdd',
            label: 'isEven',
            value: 'strlen',
            evaluate: (value: number) => value % 2 === 0,
        },
    ]}
    fields={fields}
    query={query}
    onQueryChange={setQuery}
/>
```</div>

https://github.com/react-querybuilder/react-querybuilder/discussions/829


Update Jersey to 1.19

Found via http://stackoverflow.com/questions/22917462/java-8-lambda-expression-within-rest-service-not-working
Jersey has to be updated to 1.19 to support Java8: https://java.net/jira/browse/JERSEY-2429

[x] direct dependency
[ ] check transitive dependencies and file CQs
[ ] update about.html files

Double check that property(ServerProperties.WADL_FEATURE_DISABLE, true); is in place.
public class Application extends ResourceConfig {

WADL is currently not disabled. A WADL description can be retrieved by calling http://localhost:8080/winery/application.wadl


Broken NOVA asset in the app

What

Broken NOVA asset in the app. I believe there's a question mark, from memory

@teolemon It was indeed a question mark.
We recently upgraded the flutter_svg package, and for some reason this specific svg file is not properly rendered with the latest version.
It is not a question of applied color on top, as in that case we don't use color.
I'm about to PR a rollback to the previous version of flutter_svg.

I'm reopening this issue, since it does not seem solved for me (is my local version overriden by the remote one ?)

I've tried with the current version of the app on Android and it's OK.

As there is no news, I close this issue.
Please reopen it if necessary


[network-data] check and validate received network data before registering

This commit adds check in Leader::RegisterNetworkData()to validate
the received network data and ensure it only contains entries matching
the RLOC16 of the device registering the network data. It updates
RlocLookup() method to add a new mode to allow or not allow any
entries that do not match the given RLOC16.

Decided to break out the original change in the first version of this PR which added a RlocMatch helper method to simplify the RLOC checks.
Also noticed that in  Leader::RemoveRloc(PrefixTlv &aPrefix, HasRouteTlv &aHasRoute, uint16_t aRloc16, bool aExactMatch) when aExatcMatch is false, if there are multiple matching entries, only the first one will be removed (due to break out of loop). Will address this in a separate PR.

Rloc16 would be updated per SRV_DATA.ntf as in
https://github.com/openthread/openthread/blob/master/src/core/thread/network_data_local.cpp#L356
Are there any scenario that causes RLOC16 in SRV_DATA.ntf not matching the sender's short address?


Login and Signup features (connecting with backend)

add features to the login and signup buttons, create new jsx pages.

Done bro


Note importing process follow-ups

Some things to fix came up when testing #375:

There are some minor issues to revisit but I think overall it's working as expected and we can tackle them in a follow up PR:

note should show whether it is ignored or not (easier for the user of the CLI to track)
if I export a committed note as partial, then I consume it and I import it in another client, the note goes from Expected > => Committed but never gets marked as consumed
- also, every sync says:
  2024-06-28T18:57:40.207197Z  WARN miden_client::client::sync: Current partial MMR already contains the requested data   State synced to block 167   New public notes: 0   Tracked notes updated: 1   Tracked notes consumed: 0   Tracked accounts updated: 0   Commited transactions: 0
- ... but no notes got actually updated
- this only happens for partial notes, since full notes are never ignored and id notes can't be imported (if they're private)
I tried exporting a public committed note with export type id, then consumed the note, then imported it in another client and it failed.

Originally posted by @mFragaBA in https://github.com/0xPolygonMiden/miden-client/pull/375#pullrequestreview-2148792711
However, some of this might depend on how we rework the note importing workflow.

if I export a committed note as partial, then I consume it and I import it in another client, the note goes from Expected > => Committed but never gets marked as consumed

The main reason for this is that currently we can sync before importing the note and we might miss the nullifier update (the SyncStateResponse contains the nullifiers between request.block_num + 1 and response.block_header.block_num), so once we sync and then import the note we can't go back. This issue will happen for both ignored and non ignored notes.
I don't think there's a good way to solve this issue right now. We could:

if an ignored note gets committed during an ignored notes sync, we could do a sync request to the node between the block the note was included in and the current chain tip (we wouldn't actually update the client state as a regular sync, we'd just update the nullifiers) and if the nullifier is there, we'd change the note status to consumed

The issue would still persist for non ignored notes though.

when importing NoteFile::NoteDetails(...), check if the note is already committed (and set the status accordingly) and if so do the trick from above (and set the status accordingly).

however, we'll end up with a more complex algorithm, and the logic is scattered around multiple places. Proper documentation would be crucial

If we were to have an endpoint to validate whether a list of notes have been consumed / nullifiers are present in the nullifier SMT, we could add a single step after both the normal sync and the ignored notes sync where we see if any of those got consumed.

cc @bobbinth

If we were to have an endpoint to validate whether a list of notes have been consumed / nullifiers are present in the nullifier SMT, we could add a single step after both the normal sync and the ignored notes sync where we see if any of those got consumed.

We do have a CheckNullifiers endpoint in the node - thought, it requires sending full nullifiers and returns back authentication paths for the nullifiers found in the node. So, this is a bit more heaving then needed and also reduced privacy. Maybe it is worth adding an endpoint to the node to return a list of nullifiers based on some prefix and block bounds. Could you create an issue for this in miden-node?
More generally, I think that maybe the root cause of the issues is that we are trying to do too much with the import note endpoint. Specifically, we try to cover 3 separate use cases:

Importing notes which already have inclusions proofs. By itself, handling these notes should be pretty straight-forward:
a. If the note is "in the past" as compared to where the client currently is, immediately request the block header with the corresponding MMR path. Also, check if the nullifier for this note exists. Here, we need to be careful not to check for exact nullifiers as it may allow correlating nullifiers with notes created in the block.
b. If the note is "in the future", request the block header for this note after the next sync (assuming we don't get it naturally). Also, add the nullifier prefix to the list of prefixes to be requested during the next sync.
Importing notes by ID. Also and easy case to handle: we immediately make a request to the node and if we get the note back, we treat it in the same way as case 1 above. If the note does not exist yet, we error out.
Importing "expected" notes. I think this is the most complicated case to handle because: (1) we don't know if the note is in the past or in the future, and (2) we don't know how long we should "wait" for the note. Also, the strategies would be different for notes which match existing client tags vs. notes that don't (i.e., tracked vs. untracked notes).

I'm not entirely happy with how we handle case 3, but I also don't have great suggestions here. The issues I'm seeing:

For tracked notes, it is not clear how to handle notes that are "in the past". Currently, we leave this up to the user, but I think there should be better strategy here.
For untracked notes, we could handle them similar to note IDs (i.e., make an immediate request to the node), but if the note is not in the chain yet, not clear how long we should keep trying to get it. I think we also leave this up to the user, and similarly, I think there could be a better strategy here.

I think one way of simplifying the nullifier flow could be to make the sync endpoint return nullifier data regardless of the [request.block_num, response.block_num] block range. I don't think there's much penalty for this and nullifiers would then be handled by the sync on all cases; although it could be more confusing than useful if we can just move that data to a different endpoint.

I think one way of simplifying the nullifier flow could be to make the sync endpoint return nullifier data regardless of the [request.block_num, response.block_num] block range. I don't think there's much penalty for this and nullifiers would then be handled by the sync on all cases; although it could be more confusing than useful if we can just move that data to a different endpoint.

I think this would introduce quite a bit of overhead:

In the node, we'd need to search through the entire set of nullifiers to find the relevant ones. While the nullifiers will be indexed, the number of nullifiers will be pretty large so even that may become quite expensive. With the current approach the node could shard the database (e.g., each shard could contain 2 weeks worth of blocks) and then searching could be done only in relevant shards.
The sync_state response would contain a lot of duplicate data. Basically, for every 16-bit nullifier prefix, we'd get a lot of the same nullifiers for every request. In fact, I'd think that the vast majority of the returned nullifiers would be the same across multiple requests.

Following up on https://github.com/0xPolygonMiden/miden-client/issues/405#issuecomment-2212513725, I think we should first try to solve case "3a" - i.e., tracked expected notes - as it is probably by far the more likely one to happen in practice.
When we import such a note, ideally, we'd like to know the following:

Has this note already been recorded on-chain.
Has this note already been consumed.

I think we have the following options for getting this info:

Make the requests to the node using the exact note ID and nullifier. This would guarantee that we get the most accurate info, but has pretty bad implications for privacy. Specifically:
a. Using the exact note ID leaks info that the user is potentially interested in a specific note. Given that note sender info is public, this may allow linking sender and recipient.
b. Using the exact note nullifier by itself does not leak much info (or at least not much additional info as nullifiers are sent to the node when users consume notes in transactions anyway) - but making a request by note ID and very close to it checking a nullifier for a given note will allow the node to link note IDs and nullifiers, and this should be avoided.
c. Separately, now that I think of it, we could probably increase the nullifier prefix that we send to the network (e.g., during state sync etc.) quite a bit - e.g., could be 32 bits or more and it shouldn't impact privacy too much.
Re-sync the client starting at some point in the past. This would basically simulate the note being "in the future" for the client. Since the note is already tracked, we won't have to modify much for the state sync requests. The are two drawbacks of this approach:
a. It is not clear from which point we should re-sync. Though, this could be a client-specific setting (e.g., re-sync last week, or last month, or last year).
b. We may get a lot of extra unrelated data (i.e., the data we've already received about account updates and nullifiers).
Have some specialized endpoints for handling this situation. This would be similar to what we are discussing in https://github.com/0xPolygonMiden/miden-node/issues/404.

To summarize some of the above:

Making requests by exact note IDs is probably not a good idea and we should try to avoid this (this applies to private notes only).
Making requests for exact nullifiers may be OK, but we should be careful not to correlate requests for notes with request for nullifies. In this context, it may make sense to use nullifier prefixes (16 bits for now, and maybe increasing later).

Thinking more about the approach 3 above, we could do something like this:
To check if a note has been consumed, use a specialized endpoint which would take a set of 16-bit nullifier prefixes and would return a list of nullifiers matching these prefixes regardless of when these nullifiers have been generated.
To check if a note has been already recorded on chain, we use a specialized endpoint which could like like a "light sync" (basically, state sync but only for notes). We could call this endpoint SyncNotes (or something like that). This endpoint would take a list of note tags, and would return a list of notes found in a given period. Then, in the client, we'll make the "look-back" period configurable (and default it to something like 1 month) - though, the configuration would probably be defined by the CLI rather than client.

One other option is to make sure that expected notes always come with some lower bound on when they could be added to the chain. For example, NoteFile could be defined like so:
pub enum NoteFile {
    NoteId(NoteId),
    NoteDetails { details: NoteDetails, after_block_num: u32, tag: Option<Tag> },
    NoteWithProof(Note, NoteInclusionProof),
}

The after_block_num field in the NoteDetails variant would specify the block number after which the note is expected to be added to the chain. Then, based on this field the client could either defer retrieving of the note to the SyncState (if after_block_num is greater than the last block the client is aware of) or do the SyncNotes starting with the after_block_num block.
One thing to keep in mind though: after_block_num is more like a "hint" since there is no guaranteed that the creator of the NoteFile fills it in correctly. As such, we should make sure that it is not set too far in the past (or at least show some kind of a warning if that's the case).

With the addition of the CheckNullifiersByPrefix endpoint the problem of "Has this note already been consumed?" is solved as we can ask for recorded nullifiers without giving away it's information. I've implemented a solution for the two easier cases.
For the NoteFile::NoteDetails the question of "Has this note already been recorded on-chain?" remains. I like the idea of the light sync (SyncNotes) that with the addition of the after_block_num field we can use to know if a note is committed. Once the endpoint is impelmented I can get to work with this.
Small question: Given that the after_block_num is more of a "hint". What should happen if it states that the note is from the future but it's actually is from the past? I think this case is a bit of an extreme but we should be aware of it.
This will make the import_note more complex that we originally anticipated, we should probably also look to tackle #393 at the same time and find a way to make it more legible and clear.

Given that the after_block_num is more of a "hint". What should happen if it states that the note is from the future but it's actually is from the past? I think this case is a bit of an extreme but we should be aware of it.

I think the other case here is that the not was committed in the past but not as far back as after_block_num suggested (e.g., client is currently at block 20, after_block_num = 10 but the note was committed in block 5).
I think this case could be handled separately. Specifically, we should probably do something about "stale" expected notes - i.e., notes which we have been expecting "for a while". For example, if an expected note didn't get committed within a week, we try to get it by ID. But this would be a separate issue (i.e., something about "handling of stale notes").

The after_block_num field in the NoteDetails variant would specify the block number after which the note is expected to be added to the chain. Then, based on this field the client could either defer retrieving of the note to the SyncState (if after_block_num is greater than the last block the client is aware of) or do the SyncNotes starting with the after_block_num block.

You might have been referring to tracked notes here, but we can't always assume that if an expected note is in the future, the normal SyncState will be responsible of retrieving it, no? It depends entirely on whether the correct tags are requested.

You might have been referring to tracked notes here, but we can't always assume that if an expected note is in the future, the normal SyncState will be responsible of retrieving it, no? It depends entirely on whether the correct tags are requested.

Yes, this is specifically what I called the case 3a (tracked expected notes). I haven't thought through the case 3b (untracked expected notes) - but also these should be quite rare.

Importing NoteWithProof or NoteId notes which are already consumed is addressed in #449
Importing NoteDetails which is committed in the past is addressed in #472


CodeQL: Double escaping or unescaping at utils/readfiles.js:20

Title: Double escaping or unescaping
File: utils/readfiles.js:20
Tool: CodeQL
Rule ID: js/double-escaping
Query: View source

@guatedude2, we applied your fix for #7 #8 and #9, but we still get CodeQL alerts, and this new one about double escaping.
We did a simple test of buildFilter with the current directory . .
The old code returns true correctly:
> buildFilter(".")
/^\/?\.$/i
> buildFilter(".").test(".")
true

The new code returns false incorrectly:
> buildFilter(".")
/^\/?\\\.$/i
> buildFilter(".").test(".")
false

so I tested this out with the unit tests and seems to be working:
  it('creates a filter RegExp given a filter string', () => {
    const result = buildFilter('.');

    expect(result).toEqual(/^\/?\.$/i);
  });

passes

Actually, we had missed one of your commits. Our test above is returning true now. Thanks


emacs not opening on certain Docker container

# emacs --version
GNU Emacs 24.5.1
Copyright (C) 2015 Free Software Foundation, Inc.
GNU Emacs comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of Emacs
under the terms of the GNU General Public License.
For more information about these matters, see the file named COPYING.

root@8ccc5902e5d5:~# emacs -nw
emacs: Terminal type "dumb" is not powerful enough to run Emacs.
It lacks the ability to position the cursor.
If that is not the actual type of terminal you have,
use the Bourne shell command `TERM=... export TERM' (C-shell:
`setenv TERM ...') to specify the correct type.  It may be necessary
to do `unset TERMINFO' (C-shell: `unsetenv TERMINFO') as well.

I see the same on the shell on tmux and non-tmux environment.

Tried some but didn't work (SO thread).
root@aad20f11e086:/# export VISUAL=emacsclient
root@aad20f11e086:/# emacs -nw
emacs: Terminal type "dumb" is not powerful enough to run Emacs.
It lacks the ability to position the cursor.
If that is not the actual type of terminal you have,
use the Bourne shell command `TERM=... export TERM' (C-shell:
`setenv TERM ...') to specify the correct type.  It may be necessary
to do `unset TERMINFO' (C-shell: `unsetenv TERMINFO') as well.

root@aad20f11e086:/# unset TERMINFO
root@aad20f11e086:/# emacs -nw
emacs: Terminal type "dumb" is not powerful enough to run Emacs.
It lacks the ability to position the cursor.
If that is not the actual type of terminal you have,
use the Bourne shell command `TERM=... export TERM' (C-shell:
`setenv TERM ...') to specify the correct type.  It may be necessary
to do `unset TERMINFO' (C-shell: `unsetenv TERMINFO') as well.

It's not just emacs.
# nano
Error opening terminal: ....

But vim opens the editor, although it shows the following message in the beginning for a second:
root@aad20f11e086:~# vi

E558: Terminal entry not found in terminfo
'...' not known. Available builtin terminals are:
    builtin_amiga
    builtin_beos-ansi
    builtin_ansi
    builtin_pcansi
    builtin_win32
    builtin_vt320
    builtin_vt52
    builtin_xterm
    builtin_iris-ansi
    builtin_debug
    builtin_dumb
defaulting to 'ansi'


upload and dropzone cross communication and some finer communication

First, thanks for an excellent package.
When I have a uploader and a dropzone on the same page, I notice there is a little bit of communication between them already. For instance, I think when I upload a bunch of files in the uploader, the dropzone displays the percentage complete.
However, when I drop files into the dropzone, I wish that the files appeared in the list of the uploader as they were uploading and afterward to show their status. As it stands, the dropzone is a bit of blackhole. Did it succeed? Who knows? Is it possible for them to share list, statuses, and percentages?

Images Video and Audio

        Upload all images, video and audio files here.
        {{> upload_bootstrap fileTypes='.jpg,.png,.mp4,.wav,.mp3'
            multiple=true formData=specificFormData }}

Or drag and drop files on the target area

        {{> dropzone contentType='images' fileTypes='.jpg,.png,.mp4,.wav,.mp3'
            multiple=true formData=specificFormData }}

Once you upload your files, you don't need to do it again unless the files are changed
        or updated.

formData=specificFormData is just left there from copying from the examples.
Also, I wish I could change the text that appears in the dropzone easily.
And lastly, at one point during development when I was uploading files, the percentage displayed in the dropzone said "NaN%". I haven't tried to replicate that since, but was curious if there was something I could do to prevent it.

Wmodes, can you please prepare a small test app for me with your scenario. This looks like a pile of bugs I need to address ;/

I am getting the "NaN%" now when trying to upload images to my site. Not sure why it started, but it happened when I switched from modulus to digitalocean. I used meteor up for deployment to DO.

found issue #40 and will try that out. Sorry if mine was a repost

This shall be fixed in latest.


fix cluster names in circuit breaker docs

from old style cluster names to newer ones. Also updated outlier detection spec to remove deprecated http stuff

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: rshriram
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: sdake
Assign the PR to them by writing /assign @sdake in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

content/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

/lgtm

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: geeknoid, rshriram
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

content/OWNERS [geeknoid]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


Fix broken tests on master.

This commit fixes a few issues with the code related to:

parsing: some purl-type specific handling is required to deal with things such
as case
string conversion: certain fields need to be percent-encoded

go get -u github.com/golang/lint/golint
package github.com/golang/lint/golint: code in directory /home/travis/gopath/src/github.com/golang/lint/golint expects import "golang.org/x/lint/golint"
make: *** [lint] Error 1
The command "make lint" exited with 2.

Unrelated failure.

Rebase is needed :pray:

Rebased and updated Travis to use Go 1.12. golint still appears to fail in Travis, but the tests are green.


deprecate_disable: support optional replacement parameter

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[x] Have you written new tests for your changes? Here's an example.

[x] Have you successfully run brew typecheck with your changes locally?
[x] Have you successfully run brew tests with your changes locally?

Addresses https://github.com/Homebrew/brew/issues/18294.
A replacement may be specified like so:
deprecate! date: "2024-01-10", because: :repo_removed, replacement: "ffmpeg"

which yields something like
$ HOMEBREW_NO_INSTALL_FROM_API=1 brew info alac
==> alac: stable 0.2.0 (bottled)
Basic decoder for Apple Lossless Audio Codec files (ALAC)
https://web.archive.org/web/20150319040222/craz.net/programs/itunes/alac.html
Deprecated because it has a removed upstream repository! It will be disabled on 2025-01-10.
Consider replacing it with:
    brew install ffmpeg
Not installed
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/a/alac.rb
License: MIT
==> Analytics
install: 8 (30 days), 34 (90 days), 110 (365 days)
install-on-request: 8 (30 days), 34 (90 days), 110 (365 days)
build-error: 0 (30 days)

and, for disabled:
disable! date: "2024-08-03", because: :unmaintained, replacement: "gcalcli"

which yields something like
$ HOMEBREW_NO_INSTALL_FROM_API=1 brew info gcal
==> gcal: stable 4.1 (bottled)
Program for calculating and printing calendars
https://www.gnu.org/software/gcal/
Disabled because it is not maintained upstream! It was disabled on 2024-08-03.
Consider replacing it with:
    brew install gcalcli
Not installed
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/g/gcal.rb
License: GPL-3.0-or-later
==> Dependencies
Build: texinfo ✔
==> Analytics
install: 0 (30 days), 2 (90 days), 179 (365 days)
install-on-request: 0 (30 days), 2 (90 days), 179 (365 days)
build-error: 0 (30 days)

Replacements are added as an optional keyword argument. The message and behavior when no replacement is provided should remain the same.
I made an effort to include the replacement in the API JSON as well, but I'm not too familiar. Would appreciate if someone can confirm that the changes made there are sufficient.
The replacement is just a string for now, which is shown as an argument to the brew install help text. Note that there is no distinction between the string argument as a formula or cask; there is no validation on the value of the string.
I am aware that https://github.com/Homebrew/brew/pull/18661 is open and this may need to be refactored after that is merged (or vice versa).

I made an effort to include the replacement in the API JSON as well

I'd suggesst this be an array in the API JSON, in case we want to add support for multiple replacements to the DSL.

I'd suggesst this be an array in the API JSON, in case we want to add support for multiple replacements to the DSL.

I would very much like us to avoid specifying multiple replacements for a single formula (and therefore making this an array).
It's much less useful to say "to replace this: go check out multiple other projects and decide" than "to replace this: we have decided X is the best option".

Changed the replacement message to use a heredoc and reduced indentation to two spaces. Note that the heredoc adds a trailing newline compared to the previous inline string.


Lynx openssl

Change lynx depends

Ah, I would try to do a git fetch; git rebase; git push --force

thanks lol


Regarding .experiment file

The .experiment file is locked in the folder, what might be the reason.

Hello @BakingBrains , which kind of experiment have you launched, NAS or HPO? This may be because of experiment unexpected termination when .experiment file is read or written. You can del this file safely.

closing the issue as the answer had been provided, feel free to reopen if the answer does not work well for you, thanks.


Vulkan: Enabling SS Reflections in Environment causes artifacts

Godot version:
7fc9d4c
OS/device including version:
Windows 10 / Nvidia GeForce GTX 1080 Ti (457.30) / Vulkan
Issue description:
Enabling SS Reflections in Environment causes artifacts in editor viewport.
Also these errors are getting spammed in console:

Error:

ERROR: Attempted to use an unused shader variant (shader is null),
at: (./servers/rendering/renderer_rd/pipeline_cache_rd.h:70)
ERROR: Condition "!pipeline" is true.
at: draw_list_bind_render_pipeline (drivers\vulkan\rendering_device_vulkan.cpp:6112)
ERROR: The vertex format used to create the pipeline does not match the vertex format bound.
at: (drivers\vulkan\rendering_device_vulkan.cpp:6305)
ERROR: Attempted to use an unused shader variant (shader is null),
at: (./servers/rendering/renderer_rd/pipeline_cache_rd.h:70)
ERROR: The vertex format used to create the pipeline does not match the vertex format bound.
at: (drivers\vulkan\rendering_device_vulkan.cpp:6305)
ERROR: Source and Destination textures must be the same format.
at: (drivers\vulkan\rendering_device_vulkan.cpp:2805)
ERROR: Uniforms supplied for set (3):
Set: 1 Binding: 0 Type: CombinedSampler Length: 1
Set: 1 Binding: 1 Type: CombinedSampler Length: 1
are not the same format as required by the pipeline shader. Pipeline shader requires the following bindings:
Set: 0 Binding: 0 Type: Image Length: 1
Set: 0 Binding: 1 Type: Image Length: 1
Set: 1 Binding: 0 Type: Image Length: 1
Set: 1 Binding: 1 Type: Image Length: 1
Set: 2 Binding: 0 Type: Image Length: 1
Set: 3 Binding: 0 Type: CombinedSampler Length: 1
at: (drivers\vulkan\rendering_device_vulkan.cpp:6755)

Video
Steps to reproduce:
Enable SS Reflections in Environment in WorldEnvironment node.
Minimal reproduction project:
SSRTest.zip

Duplicate of https://github.com/godotengine/godot/issues/40779
Can you please post the above information as a response to that issue so that it is all tracked in one place?

Duplicate of https://github.com/godotengine/godot/issues/40779
Can you please post the above information as a response to that issue so that it is all tracked in one place?

Oh, sorry, didn't find that issue when i was searching.
Will re-post there.

Oh, sorry, didn't find that issue when i was searching.
Will re-post there.

Thanks!

Thanks!


Incorrect display of file names in Windows 10

os: Microsoft Windows [Version 10.0.19044.1706]
lsd --version: lsd 0.22.0
echo $TERM: xterm-256color
echo $LS_COLORS: rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;4
4:ex=01;32:.tar=01;31:.tgz=01;31:.arc=01;31:.arj=01;31:.taz=01;31:.lha=01;31:.lz4=01;31:.lzh=01;31:.lzma=01;31:.tlz=01;31:.txz=01;31:.tzo=
01;31:.t7z=01;31:.zip=01;31:.z=01;31:.dz=01;31:.gz=01;31:.lrz=01;31:.lz=01;31:.lzo=01;31:.xz=01;31:.zst=01;31:.tzst=01;31:.bz2=01;31:.bz=
01;31:.tbz=01;31:.tbz2=01;31:.tz=01;31:.deb=01;31:.rpm=01;31:.jar=01;31:.war=01;31:.ear=01;31:.sar=01;31:.rar=01;31:.alz=01;31:.ace=01;31:
.zoo=01;31:.cpio=01;31:.7z=01;31:.rz=01;31:.cab=01;31:.wim=01;31:.swm=01;31:.dwm=01;31:.esd=01;31:.jpg=01;35:.jpeg=01;35:.mjpg=01;35:.mjp
eg=01;35:.gif=01;35:.bmp=01;35:.pbm=01;35:.pgm=01;35:.ppm=01;35:.tga=01;35:.xbm=01;35:.xpm=01;35:.tif=01;35:.tiff=01;35:.png=01;35:.svg=01
;35:.svgz=01;35:.mng=01;35:.pcx=01;35:.mov=01;35:.mpg=01;35:.mpeg=01;35:.m2v=01;35:.mkv=01;35:.webm=01;35:.webp=01;35:.ogm=01;35:.mp4=01;3
5:.m4v=01;35:.mp4v=01;35:.vob=01;35:.qt=01;35:.nuv=01;35:.wmv=01;35:.asf=01;35:.rm=01;35:.rmvb=01;35:.flc=01;35:.avi=01;35:.fli=01;35:.fl
v=01;35:.gl=01;35:.dl=01;35:.xcf=01;35:.xwd=01;35:.yuv=01;35:.cgm=01;35:.emf=01;35:.ogv=01;35:.ogx=01;35:.aac=00;36:.au=00;36:.flac=00;36:
.m4a=00;36:.mid=00;36:.midi=00;36:.mka=00;36:.mp3=00;36:.mpc=00;36:.ogg=00;36:.ra=00;36:.wav=00;36:.oga=00;36:.opus=00;36:.spx=00;36:.xsp
f=00;36:

Additional info:

My terminal emulator: Alacritty with Hack NF font.
Alacritty config: https://gist.github.com/MeShootIn/288a06e700582cdc993708968a49814c

Expected behavior
If applicable, add the output of the classic ls command (\ls -la) in order to show the buggy file/directory.
Сolored correct file names with indents and line breaks should be displayed.
Actual behavior
If the application panics run the command with the trace (RUST_BACKTRACE=1 lsd ...).
In case of graphical errors, add a screenshot if possible.
In Alacritty:

In cmd (colored, but without icons (because of uninstalled font for cmd)):

This looks odd, it looks like the ansi escape codes are not working properly. Could you double check the TERM value set in alacrity.

This looks odd, it looks like the ansi escape codes are not working properly. Could you double check the TERM value set in alacrity.

Could you try if echo -e "\x1b[38;5;4mfolder-test\x1b[0m" gives you the same output. If not, it is probably some issue with the terminal, terminfo or $TERM.

The reason why ls is displaying colors but lsd is not is because ls uses 16 colors and lsd uses 256 by default.

Could you try if echo -e "\x1b[38;5;4mfolder-test\x1b[0m" gives you the same output. If not, it is probably some issue with the terminal, terminfo or $TERM.

How about any of '\x1B[38;5;4m\x1B[1mdev\x1B[0m' or '\033[38;5;4m\033[1mdev\033[0m' or '\033[38;5;4m\033[1mdev\033[0m'.  If I am not mistaken the first one is gonna cause problems.
In any case, I found related issues in alacritty (https://github.com/alacritty/alacritty/issues/4794)

These two lines below are the same:

'\033[38;5;4m\033[1mdev\033[0m' or '\033[38;5;4m\033[1mdev\033[0m'
However, both examples print correctly:

My bad, the last one was supposed to be \e[38;5;4m\e[1mdev\e[0m, but I guess that should work. lsd actually uses the first one mentioned there, but if that own its own is working  I am not entirely sure what is going on. Could you try lsd --icon never and see if that is still causing issues?

My bad, the last one was supposed to be \e[38;5;4m\e[1mdev\e[0m, but I guess that should work. lsd actually uses the first one mentioned there, but if that own its own is working I am not entirely sure what is going on. Could you try lsd --icon never and see if that is still causing issues?

I don't think this is an lsd issue as it works in other terminal emulators and the ansi escape codes is pretty much standard. I am starting to believe this is more related to the alacritty issue that I linked before. In any case, maybe give it a shot with TERM=screen, TERM=""(empty), TERM=screen-256color.


Only 1 node started when 5 requested

I tried deploying a 5 node cluster, but only 1 node is being started.
apiVersion: cloud.crate.io/v1
kind: CrateDB
spec:
  cluster:
    imageRegistry: crate
    name: crate-dev
    version: 5.2.5
  nodes:
    data:
      - name: crate-cluster
        replicas: 5
        resources:
          disk:
            count: 1
            size: 128GiB
            storageClass: ebs-storage-class
          heapRatio: 0.25
          limits:
            cpu: 2
            memory: 15Gi

Hi @robd003, could you share the output of the below please?
$ kubectl -n $YOUR_NAMESPACE get sts -o yaml
$ kubectl -n $YOUR_NAMESPACE get pods -o yaml


Director images and events are out of date

is the events here referencing the Notion events? if so, let's make that a separate issue so this PR is more lighter


What defines machine inactivity?

I have users with this policy set.  However, the time limit passes and the screen does not lock.  What determines when the inactivity period starts?

Hello @mbenford21, can you tell us which article your question is about? It will help us route this to the correct writer. Thanks!

Hello, @mbenford21  thank you for providing feedback.
The issues section of this repository is intended for product documentation issues only.
Please consider opening a product support ticket by the following link below for your problem so that it gets resolved quickly.
Windows 10 Support
Thank you.


chore: improve ci test reports with gotestsum

What kind of change does this PR introduce?
Bug fix, feature, docs update, ...
What is the current behavior?
Please link any relevant issues here.
What is the new behavior?
Feel free to include screenshots if it includes visual changes.
Additional context
Add any other context or screenshots.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.03%) to 58.295%

Totals

Change from base Build<PHONE_NUMBER>:
-0.03%

Covered Lines:
6005

Relevant Lines:
10301

💛  - Coveralls


problems with custom objective function for  mean absolute percentage error(MAPE)

Hi,everyone.
I have some problems when implement custom objective functions for mean absolute percentage error(MAPE) .
the evaluation metric is defined as follow:
`
def evalmape(preds,dtrain):
labels = dtrain.get_label()

error = sum(abs(preds-labels)*1.0/(labels))*1.0/len(labels)

return 'error',error`

the objective function is:
`
def mapeobj(preds,dtrain):
labels = dtrain.get_label()

preds = preds

grad = np.sign(preds-labels)*1.0/labels

hess = 0*np.ones(len(preds))

return grad,hess

`
and my parameters' setting is:
`
param = {
            'bst:max_depth':6,

            'min_child_weight':1,

            'bst:eta':0.01,

            'subsample':1.0,

            'silent':0,

            'objective':'reg:linear',

            'tree_method':'exact',

            'lambda':'1'

}`
but i just found that the tree didn't split at all and the prediction value stay 0.5 all the time.
all my labels is larger than 1.
than I found that there is a line in xgboost\src\tree\param.h，
if (sum_hess < min_child_weight) return 0.0;
I guess it may because the hess in my objective function is 0,so I set the parameter min_child_weight = 0.
The results shows that the tree start split now,but the MAPE Swings back and forth between the 1 and 1000.
But the min MAPE abviously should be smaller than 0.
Is there any problems in my setting?
And I also wonder that if xgboost can optimal the loss function such as MAPE?
Has anyone tried that?
I would be grateful if you can help me.Thank you.

The scores on leaves are calculated by dividing sum_hess.
https://github.com/dmlc/xgboost/blob/master/src/tree/param.h#L242
In this objective, scores and grad tends to be very large.(It may be negative)

Greater lambda
Lesser learning_rate

Both prevent that the scores are too far.
But I don't think these are effective.

Set max_delta_step > 0

@bloom77 : please see #1428 and #603

I opened an issue explaining the case https://github.com/dmlc/xgboost/issues/1825

@tqchen Is it possible to give an example of how that would work for the MAPE metric? Are we supposed to find the function that approximates the upper bound of MAPE?
I'm trying to use the Mean Squared Percentage Error to give it a derivative:
def mspe(y, dtrain):
    yhat = dtrain.get_label()
    grad = 2.0/yhat * (y * 1.0 / yhat - 1)
    hess = 2.0/(yhat**2)
    return grad, hess

and then I run it with
params={'eta': 0.05, 'max_depth': 7, 'colsample_bytree': 1, 'subsample': 1, 'min_child_weight':12}
clf = xgb.train(params, xg_train, num_boost_round=500,
                early_stopping_rounds=10, evals=[(xg_train, 'train'), (xg_test, 'test')],
                 obj= mspe
               )

but same as @bloom77 the predictions are stuck at 0.5, probably because the hessian is too small (assuming I did the derivatives correctly..)

yes, use an upper bound. In most cases, for example, set second order gradient to be bigger than zero could do, you can try to plot the curve and see how well the function approximates the upper bound

@zweicoder
why not consider using the "autograd" package in-place of computing your own gradient & hessian?
def mspe(y, y_pred):
from autograd import grad
from autograd import hessian
mape = lambda y, y_pred: autograd.numpy.sum(autograd.numpy.abs(y_pred - y)*1.0/(y))*1.0/len(y)
grad = grad(mape)
hessian = hessian(mape)
return grad, hess

@vatsan autograd is very slowly in computation when dealing with moderately sized data. In my case it is unusable (taking hessian of 5000 elements would already take a few minutes)

@zweicoder
True, i observed it while running it on my own task. It was very slow & XGBoost crashed eventually.

Hey, mape is too small to converge , 100* grad and 100hess  it will work better
label = t.get_label()
grad = -100((label-y)/label)
hess = 100/(label)
return grad,hess

@alanleemachinelearning
thanks,may I ask why the grad and hess can express like this and work fine?
the derivation of  grad and hess should like this:
grad =sign(preds-labels)/labels
hess =zeros(len(preds))

@alanleemachinelearning
when I use your method,the train err and val err are all decrease initially,
but then increase both,
they can't converge


Upload image and preview

Hi everybody,
Already I wanted to say a big thanx for your work.
Then I 'd like to know if it is possible with your tool to upload an image in a form with a preview.
Thank you in advance.

Create a directive and it to ng-admin :
http://stackoverflow.com/questions/18754943/preview-image-before-uploading-angularjs

Someone have an example plz?


Example as demo

I did not know why the example was not used as the full demo, so I moved all the latest changes there in to the example dir, with further

factoring
proper way to handle URL
a known free TURN server credential
flutter format .

@truongsinh I do n’t think we need to put the complete example in the plugin, because most people will directly rely on the flutter-webrtc plugin to develop their own products. Similar to the p2p sample, the data channel needs to connect to a specific server, which will make people mistake this It is a plug-in bundled with a specific server.
The plug-in itself only requires the most basic API testing and loopback support, and the other is to make the plug-in code as simple as possible.

OK thanks, then if I have some ideas to improve the demo, I will do in the other repo instead.

@cloudwebrtc maybe we could create the wiki page where collect such type of additional demos when users commit them to some public repo?
@truongsinh I am a bit curious, but what additional features of flutter-webrtc usage does your demo add to compare to flutter-webrtc-demo? Maybe you could create PR for that project?

@cloudwebrtc maybe we could create the wiki page where collect such type of additional demos when users commit them to some public repo?

This is a good idea.


[http] introduce legacy codec with no diff checker

https://github.com/envoyproxy/envoy/pull/10591 without the diff format checker
Commit Description:
This just introduces plumbing and clones codecs for a legacy versions.
There is no changes to the codecs at this point (the files are just duplicated and the namespace Legacy is used). Users can set which codecs are used at runtime via the feature new_codec_behavior. This will be the case for O(4-6 weeks) while the change to new no-exception codecs are pushed (see issue linked) to the new codecs.
Risk Level: Medium. No-op for behavior, but high because it branches and splits codecs.
Testing:
The H/1 test is parametrized for both new and legacy codec
The H/2 test is built twice, in new and legacy mode (legacy mode is triggered with a --runtime-disable-for-test flag)
Integration tests run over all their combinations of upstream/downstream legacy and new codec versions in compile_time_options.
Release Notes: Added release note
Part of #10878
Signed-off-by: Asra Ali<EMAIL_ADDRESS>

clang_tidy issues are pre-existing. Since this is a rollforward of an approved PR I will not be looking for all approvals again.


⚠ AGEG website (redirect) has degraded performance

In 79e3a6a, AGEG website (redirect) (https://ageg.ca/) experienced degraded performance:

HTTP code: 200
Response time: 3630 ms

Resolved: AGEG website (redirect) performance has improved in 57842b0.


docker-firefox needs privileged mode to run

Current Behavior
Running jlesage/docker-firefox image in TrueNAS Scale kubernetes cluster. My previous docker-firefox version (v23.03.1) deployed in that environment was running nicely without 'Privileged' mode permission but now we I tried to deploy the latest docker-firefox version (v23.05.2) it requires 'Privileged' mode to be set to be started and running without restarting continuously. I have seccomp profile defined for the pod in both releases so I suspect issue being elsewhere. I have attached 'kubectl logs' which prints container logs for both cases. Main difference what I can see that xvnc will not run in failed case.
Expected Behavior
docker-firefox pod starts and runs without enabling 'Privileged' mode.
Steps To Reproduce
Deploy docker-firefox container (v23.05.2) in TrueNAS Scale using its custom-app helm chart which automatically generates deployment manifest for the container and also injects configurations like seccomp profile to pod template. Try to run container without 'Privileged' mode, it starts but enters cycling pod restart.
Environment

OS: TrueNAS Scale
OS version: 22.12.2
CPU: AMD Ryzen 7 1700
Kubernetes (K3s) version: v1.25.3+k3s-9afcd6b9-dirty (bundled with TrueNAS)

Container creation
See Container inspect which shows Kubernetes deployment manifest generated when creating pod thru GUI.

**Succesful container execution - Container log with 'Privileged' mode**
# kubectl logs -n ix-firefox firefox-custom-app-798586cd97-znb4w
[init        ] container is starting...
[cont-env    ] loading container environment variables...
[cont-env    ] APP_NAME: loading...
[cont-env    ] APP_VERSION: loading...
[cont-env    ] DISPLAY: executing...
[cont-env    ] DISPLAY: terminated successfully.
[cont-env    ] DISPLAY: loading...
[cont-env    ] DOCKER_IMAGE_PLATFORM: loading...
[cont-env    ] DOCKER_IMAGE_VERSION: loading...
[cont-env    ] GTK2_RC_FILES: executing...
[cont-env    ] GTK2_RC_FILES: terminated successfully.
[cont-env    ] GTK2_RC_FILES: loading...
[cont-env    ] GTK_THEME: executing...
[cont-env    ] GTK_THEME: terminated successfully.
[cont-env    ] GTK_THEME: loading...
[cont-env    ] HOME: loading...
[cont-env    ] QT_STYLE_OVERRIDE: executing...
[cont-env    ] QT_STYLE_OVERRIDE: terminated successfully.
[cont-env    ] QT_STYLE_OVERRIDE: loading...
[cont-env    ] TAKE_CONFIG_OWNERSHIP: loading...
[cont-env    ] XDG_CACHE_HOME: loading...
[cont-env    ] XDG_CONFIG_HOME: loading...
[cont-env    ] XDG_DATA_HOME: loading...
[cont-env    ] XDG_RUNTIME_DIR: loading...
[cont-env    ] XDG_STATE_HOME: loading...
[cont-env    ] container environment variables initialized.
[cont-secrets] loading container secrets...
[cont-secrets] container secrets loaded.
[cont-init   ] executing container initialization scripts...
[cont-init   ] 10-certs.sh: executing...
[cont-init   ] 10-certs.sh: terminated successfully.
[cont-init   ] 10-check-app-niceness.sh: executing...
[cont-init   ] 10-check-app-niceness.sh: terminated successfully.
[cont-init   ] 10-cjk-font.sh: executing...
[cont-init   ] 10-cjk-font.sh: installing CJK font...
[cont-init   ] 10-cjk-font.sh: fetch http://dl-cdn.alpinelinux.org/alpine/edge/testing/x86_64/APKINDEX.tar.gz
[cont-init   ] 10-cjk-font.sh: fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64/APKINDEX.tar.gz
[cont-init   ] 10-cjk-font.sh: fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/community/x86_64/APKINDEX.tar.gz
[cont-init   ] 10-cjk-font.sh: (1/1) Installing font-wqy-zenhei (0.9.45-r2)
[cont-init   ] 10-cjk-font.sh: Executing fontconfig-2.14.2-r3.trigger
[cont-init   ] 10-cjk-font.sh: Executing mkfontscale-1.2.2-r3.trigger
[cont-init   ] 10-cjk-font.sh: OK: 546 MiB in 146 packages
[cont-init   ] 10-cjk-font.sh: terminated successfully.
[cont-init   ] 10-clean-logmonitor-states.sh: executing...
[cont-init   ] 10-clean-logmonitor-states.sh: terminated successfully.
[cont-init   ] 10-clean-tmp-dir.sh: executing...
[cont-init   ] 10-clean-tmp-dir.sh: terminated successfully.
[cont-init   ] 10-fontconfig-cache-dir.sh: executing...
[cont-init   ] 10-fontconfig-cache-dir.sh: terminated successfully.
[cont-init   ] 10-init-users.sh: executing...
[cont-init   ] 10-init-users.sh: terminated successfully.
[cont-init   ] 10-nginx.sh: executing...
[cont-init   ] 10-nginx.sh: terminated successfully.
[cont-init   ] 10-openbox.sh: executing...
[cont-init   ] 10-openbox.sh: terminated successfully.
[cont-init   ] 10-set-tmp-dir-perms.sh: executing...
[cont-init   ] 10-set-tmp-dir-perms.sh: terminated successfully.
[cont-init   ] 10-vnc-password.sh: executing...
[cont-init   ] 10-vnc-password.sh: terminated successfully.
[cont-init   ] 10-web-data.sh: executing...
[cont-init   ] 10-web-data.sh: terminated successfully.
[cont-init   ] 10-x11-unix.sh: executing...
[cont-init   ] 10-x11-unix.sh: terminated successfully.
[cont-init   ] 10-xdg-runtime-dir.sh: executing...
[cont-init   ] 10-xdg-runtime-dir.sh: terminated successfully.
[cont-init   ] 15-install-pkgs.sh: executing...
[cont-init   ] 15-install-pkgs.sh: terminated successfully.
[cont-init   ] 55-check-snd.sh: executing...
[cont-init   ] 55-check-snd.sh: sound device group 29.
[cont-init   ] 55-check-snd.sh: terminated successfully.
[cont-init   ] 55-firefox.sh: executing...
[cont-init   ] 55-firefox.sh: terminated successfully.
[cont-init   ] 56-firefox-set-prefs-from-env.sh: executing...
[cont-init   ] 56-firefox-set-prefs-from-env.sh: terminated successfully.
[cont-init   ] 85-take-config-ownership.sh: executing...
[cont-init   ] 85-take-config-ownership.sh: terminated successfully.
[cont-init   ] 89-info.sh: executing...
    ╭――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――╮
    │                                                                      │
    │ Application:           Firefox                                       │
    │ Application Version:   113.0.2-r1                                    │
    │ Docker Image Version:  23.05.2                                       │
    │ Docker Image Platform: linux/amd64                                   │
    │                                                                      │
    ╰――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――╯
[cont-init   ] 89-info.sh: terminated successfully.
[cont-init   ] all container initialization scripts executed.
[init        ] giving control to process supervisor.
[supervisor  ] loading services...
[supervisor  ] loading service 'default'...
[supervisor  ] loading service 'logmonitor'...
[supervisor  ] service 'logmonitor' is disabled.
[supervisor  ] loading service 'logrotate'...
[supervisor  ] loading service 'app'...
[supervisor  ] loading service 'gui'...
[supervisor  ] loading service 'openbox'...
[supervisor  ] loading service 'xvnc'...
[supervisor  ] loading service 'nginx'...
[supervisor  ] loading service 'certsmonitor'...
[supervisor  ] service 'certsmonitor' is disabled.
[supervisor  ] all services loaded.
[supervisor  ] starting services...
[supervisor  ] starting service 'logrotate'...
[supervisor  ] started service 'logrotate'.
[logrotate   ] reading config file /opt/base/etc/logrotate.conf
[logrotate   ] including /etc/cont-logrotate.d
[logrotate   ] reading config file nginx
[logrotate   ] acquired lock on state file /config/xdg/state/logrotate/logrotate.status
[logrotate   ] Reading state from file: /config/xdg/state/logrotate/logrotate.status
[logrotate   ] Allocating hash table for state file, size 64 entries
[logrotate   ] Creating new state
[logrotate   ] Creating new state
[logrotate   ] Handling 1 logs
[logrotate   ] rotating pattern: /config/log/nginx/access.log /config/log/nginx/error.log  monthly (6 rotations)
[logrotate   ] empty log files are not rotated, old logs are removed
[logrotate   ] considering log /config/log/nginx/access.log
[logrotate   ]   log /config/log/nginx/access.log does not exist -- skipping
[logrotate   ] considering log /config/log/nginx/error.log
[logrotate   ]   log /config/log/nginx/error.log does not exist -- skipping
[logrotate   ] not running postrotate script, since no logs were rotated
[supervisor  ] starting service 'xvnc'...
[supervisor  ] started service 'xvnc'.
[xvnc        ] Wed Jun  7 19:31:28 2023
[xvnc        ]  Config:      set rfbport(Int) to 5900
[xvnc        ]  Config:      set UseIPv6(Bool) to no(0)
[xvnc        ]  Config:      set rfbunixpath(String) to /tmp/vnc.sock
[xvnc        ]  Config:      set rfbunixmode(Int) to 0660
[xvnc        ]  Config:      set SecurityTypes(String) to None
[xvnc        ]  Config:      set desktop(String) to Firefox
[xvnc        ] Xvnc TigerVNC 1.13.1 - built May 16 2023 17:54:26
[xvnc        ] Copyright (C) 1999-2022 TigerVNC Team and many others (see README.rst)
[xvnc        ] See https://www.tigervnc.org for information on TigerVNC.
[xvnc        ] Underlying X server release 12014000
[xvnc        ]  vncext:      VNC extension running!
[xvnc        ]  Config:      set immutable AllowOverride
[xvnc        ]  vncext:      Listening for VNC connections on /tmp/vnc.sock (mode 0660)
[xvnc        ]  vncext:      Listening for VNC connections on all interface(s), port 5900
[xvnc        ]  VNCServerST: creating single-threaded server Firefox
[xvnc        ]  vncext:      created VNC server for screen 0
[supervisor  ] waiting for service 'xvnc' to be ready...
[supervisor  ] starting service 'openbox'...
[supervisor  ] started service 'openbox'.
[xvnc        ]  Selection:   Selection owner change for WM_S0
[supervisor  ] waiting for service 'openbox' to be ready...
[supervisor  ] starting service 'nginx'...
[supervisor  ] started service 'nginx'.
[nginx       ] Listening for HTTP connections on port 5800.
[supervisor  ] waiting for service 'nginx' to be ready...
[supervisor  ] starting service 'app'...
[supervisor  ] started service 'app'.
[app         ] Mozilla Firefox 113.0.2
[supervisor  ] all services started.
[supervisor  ] service 'logrotate' exited (with status 0).
[supervisor  ] waiting termination of logger thread of service 'logrotate'...
[supervisor  ] logger thread of service 'logrotate' successfully terminated.

Container inspect
--- This deployment manifest has 'privileged: true', which is only difference to failed case ---
# kubectl get deploy -n ix-firefox firefox-custom-app -o yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "49"
    meta.helm.sh/release-name: firefox
    meta.helm.sh/release-namespace: ix-firefox
  creationTimestamp: "2023-06-04T17:00:59Z"
  generation: 57
  labels:
    app: custom-app-8.0.17
    app.kubernetes.io/instance: firefox
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: custom-app
    app.kubernetes.io/version: 0.20.4145
    helm-revision: "38"
    helm.sh/chart: custom-app-8.0.17
    release: firefox
  name: firefox-custom-app
  namespace: ix-firefox
  resourceVersion: "27395729"
  uid: 1b057a77-45f5-4a05-b5fb-3f4b3efded33
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 3
  selector:
    matchLabels:
      app.kubernetes.io/instance: firefox
      app.kubernetes.io/name: custom-app
      pod.name: main
  strategy:
    type: Recreate
  template:
    metadata:
      annotations:
        rollme: buVN0
      creationTimestamp: null
      labels:
        app: custom-app-8.0.17
        app.kubernetes.io/instance: firefox
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/name: custom-app
        app.kubernetes.io/version: 0.20.4145
        helm-revision: "38"
        helm.sh/chart: custom-app-8.0.17
        pod.name: main
        release: firefox
    spec:
      automountServiceAccountToken: false
      containers:
      - env:
        - name: TZ
          value: Europe/Stockholm
        - name: UMASK
          value: "0022"
        - name: UMASK_SET
          value: "0022"
        - name: NVIDIA_VISIBLE_DEVICES
          value: void
        - name: PUID
          value: "1012"
        - name: USER_ID
          value: "1012"
        - name: UID
          value: "1012"
        - name: PGID
          value: "1012"
        - name: GROUP_ID
          value: "1012"
        - name: GID
          value: "1012"
        - name: DISPLAY_WIDTH
          value: "1920"
        - name: DISPLAY_HEIGHT
          value: "1080"
        - name: DARK_MODE
          value: "1"
        - name: ENABLE_CJK_FONT
          value: "1"
        - name: CONTAINER_DEBUG
          value: "1"
        image: docker.io/jlesage/firefox:v23.05.2
        imagePullPolicy: IfNotPresent
        name: firefox-custom-app
        ports:
        - containerPort: 5800
          name: main
          protocol: TCP
        resources:
          limits:
            cpu: "1"
            memory: 1Gi
          requests:
            cpu: 10m
            memory: 50Mi
        securityContext:
          allowPrivilegeEscalation: true
          capabilities:
            add:
            - CHOWN
            - SETUID
            - SETGID
            - FOWNER
            - DAC_OVERRIDE
            drop:
            - ALL
          privileged: true
          readOnlyRootFilesystem: false
          runAsGroup: 1012
          runAsNonRoot: false
          runAsUser: 0
          seccompProfile:
            type: RuntimeDefault
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /dev/shm
          name: devshm
        - mountPath: /config
          name: persist-list-0
        - mountPath: /downloads
          name: persist-list-1
        - mountPath: /shared
          name: shared
        - mountPath: /tmp
          name: tmp
        - mountPath: /var/logs
          name: varlogs
        - mountPath: /var/run
          name: varrun
      dnsConfig:
        options:
        - name: ndots
          value: "1"
      dnsPolicy: ClusterFirst
      enableServiceLinks: false
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext:
        fsGroup: 1012
        fsGroupChangePolicy: OnRootMismatch
        supplementalGroups:
        - 568
      serviceAccount: default
      serviceAccountName: default
      shareProcessNamespace: false
      terminationGracePeriodSeconds: 60
      volumes:
      - emptyDir:
          medium: Memory
        name: devshm
      - name: persist-list-0
        persistentVolumeClaim:
          claimName: firefox-custom-app-persist-list-0
      - hostPath:
          path: /mnt/nas-tank/seedbox
          type: ""
        name: persist-list-1
      - emptyDir: {}
        name: shared
      - emptyDir: {}
        name: tmp
      - emptyDir: {}
        name: varlogs
      - emptyDir:
          medium: Memory
        name: varrun
status:
  availableReplicas: 1
  conditions:
  - lastTransitionTime: "2023-06-04T17:00:59Z"
    lastUpdateTime: "2023-06-07T17:31:20Z"
    message: ReplicaSet "firefox-custom-app-798586cd97" has successfully progressed.
    reason: NewReplicaSetAvailable
    status: "True"
    type: Progressing
  - lastTransitionTime: "2023-06-07T17:31:24Z"
    lastUpdateTime: "2023-06-07T17:31:24Z"
    message: Deployment has minimum availability.
    reason: MinimumReplicasAvailable
    status: "True"
    type: Available
  observedGeneration: 57
  readyReplicas: 1
  replicas: 1
  updatedReplicas: 1

Anything else?
No response

Do you get the same failure every time you (re)start the container ?
Is v23.03.1 the last version that work fine ?

Yes, I get same failure always when trying run it with setting 'privileged: false'.
I had been using v23.03.1 until this point and that was working fine and did not need 'Privileged' mode.

Can you try other versions (https://github.com/jlesage/docker-firefox/releases) to see the latest one that works?
I don't see any change that would explain the sudden change of behaviour.

Sure, off course. Will probably do it tomorrow as it is pretty late for my timezone. What comes my mind is that perhaps nothing has really changed from docker-firefox side but perhaps they have tighten up security in TrueNAS / Truecharts side. So what I am wondering if docker-firefox container is e.g. getting sufficient Linux capabilities from Kubernetes.
This is what is currently given for containers
       securityContext:           allowPrivilegeEscalation: true           capabilities:             add:             - CHOWN             - SETUID             - SETGID             - FOWNER             - DAC_OVERRIDE             drop:             - ALL           privileged: true           readOnlyRootFilesystem: false           runAsGroup: 1012           runAsNonRoot: false           runAsUser: 0           seccompProfile:             type: RuntimeDefault

Made some testing and docker-firefox also failed with older releases, even with v23.03.1 which worked for me previously. However I made finding that if I add all 'Capabilities' listed for Baseline profile then docker-firefox container starts and run successfully. So with more relax capabilities also v23.05.2 starts and run.
These profiles are described in Kubernetes documentations under Pod Security Standards
My guess is that TruenNAS/Truechart folks may have tighten up security (which is good thing so kudos for them) when introducing their new common helm chart + custom-app chart which are used to automatically generate deployment manifest for docker-firefox and this caused issue.

Some further testing with Linux capabilities to better understand what exact capability set docker-firefox really needs to launch and run.
Truecharts are assigning following capabilities for every pod and deny rest (I did not try to remove any of capabilities from this list, even that is very easy thing to do).

CAP_CHOWN
CAP_SETUID
CAP_SETGID
CAP_FOWNER
CAP_DAC_OVERRIDE

However this is not sufficient for docker-firefox which requires two additional Linux capabilities to launch and run

CAP_KILL
CAP_NET_BIND_SERVICE

So it looks like there is no issue in docker-firefox itself which works like charm on Kubernetes when it gets sufficient Linux capabilities. However I would be grateful if you could consider to document which Linux capabilities docker-firefox really need to launch and run?

However I would be grateful if you could consider to document which Linux capabilities docker-firefox really need to launch and run?

Documenting the minimal set of capabilities might be difficult (I guess we need to try to see what is needed/missing), especially considering that they might change after updates.  In general, the container is tested to work properly with capabilities provided by Docker.

Fully understood that this is probably difficult to document and may change in the future. I however wanted to make you aware that above seven Linux capabilities were needed to able to launch and run docker-firefox (v23.05.2) successfully on Kubernetes (others were dropped).
Kubernetes lists three different security profiles, including Linux capabilities in their documentation under Pod Security Standards (wondering if Docker has similar documentation?) From there I took 'Baseline' profile as input for my test.
When considering documentation, it is perhaps sufficient just to refer 'Pod Security Standards' Baseline profile as it provides superset of capabilities which docker-firefox needs. This is because Kubernetes documentation considers capabilities listed under Baseline profile "safe" i.e. preventing known privilege escalations. What do you think?
I will remove 'Bug' label from this issue now as this is definitely not bug in docker-firefox but merely Kubernetes configuration issue. But I will be more than happy to test things (as I have Kubernetes setup with docker-firefox) and also exchange ideas how to document this if you think it is worth to do :)

Oh, seems that I am not able to remove 'Bug' tag from this issue, my apologies.


As a User I want to see the ASM Mammalian Diversity Database appear as a data source

As a User I want to see the ASM Mammalian Diversity Database appear as a data source in https://resolver.globalnames.org and related services .
It appears that the https://mammaldiversity.org resource is being used among mammal researchers.
I was able to download attached dump mammal.json.gz using curl 'https://mammaldiversity.org/species-account/api.php?q=*' -H 'User-Agent:' | gzip > mammal.json.gz .
Note that the User-Agent was somehow needed.
See https://github.com/globalbioticinteractions/globalbioticinteractions/issues/446 .
FYI @tigerhawkvok @n8upham
mammal.json.gz

@jhpoelen just saw it, will add it in the next few days

@jhpoelen, got the data to resolver.globalnames.org
I am making updates and new imports of data these few weeks, when I am done I'll move the data to index.globalnames.org as well and publish the data dump

@dimus excellent  - please also note that @n8upham et al. are starting an easier to use mammal diversity database at https://github.com/mammaldiversity/mammaldiversity.github.io/tree/master/_data . However, this is still very much a work in progress, so I'll let @n8upham , a mammal diversity database curator, chime in on whether to use this version.

I just did a search on Enhydra lutris via resolver.globalnames.org, and found a match with a MDD taxon of same name with taxon id 549 . However, in MDD, the taxon id https://mammaldiversity.github.io/#7750 (or https://mammaldiversity.org/species-account/species-id=7750) is associated with the taxon. How did you populate the taxon ids?

I just did a search on Enhydra lutris via resolver.globalnames.org, and found a match with a MDD taxon of same name with taxon id 549
hm, I guess I should use their internal_id instead of id

    "1721": {
      "id": "549",
      "species_authority": "Linnaeus",
      "canonical_sciname": "Enhydra lutris",
      "image": "species_photos/2bccdb0a8b47b4e59a0b56c800790173.jpg",
      "notes": "",
      "authority_year": "{\"1758\":1758}",
      "parens_auth_genus": "0",
      "internal_id": "7750",
      "major_type": "placentalia",
      "genus": "enhydra",
      "taxon_author": "",
      "parens_auth_species": "0",
      "genus_authority": "Linnaeus",
      "deprecated_scientific": "",
      "subspecies": "",
      "simple_linnean_group": "eutheria",
      "linnean_order": "carnivora",
      "source": "iucn",
      "species": "lutris",
      "taxon_credit": "",
      "taxon_credit_date": "",
      "common_name": "Sea Otter",
      "major_subtype": "boreoeutheria",
      "linnean_family": "mustelidae",
      "simple_linnean_subgroup": "weasels / otters",
      "citation": "",
      "image_credit": "J G Hall 1975",
      "image_license": "{\"CC BY-NC 4.0\":\"https:\\/\\/creativecommons.org\\/licenses\\/by-nc\\/4.0\\/legalcode\"}",
      "entry": "",
      "common_name_source": "iucn",
      "image_caption": "Side view, in kelp bed.",
      "species_authority_citation": "",
      "genus_authority_citation": "",
      "simple_linnean_group_alt": "placentalia",
      "linnean_subfamily": "lutrinae",
      "linnean_tribe": "",
      "juniorSynonym": "0",
      "synonymOf": "",
      "TaxonomyNotes": "",
      "TaxonomyNotes_Citation": "",
      "extinct": "0",
      "geo_distribution": "marine",
      "authority_fullcitation": "linneaus, c von. 1758. systema naturae per regna tria naturae: secundum classes, ordines, genera, species, cum characteribus, differentiis, synonymis, locis. (10th ed.). stockholm: laurentius salvius.",
      "authority_link": "https://biodiversitylibrary.org/page/25033865",
      "MDDv1": "1",
      "newSppSinceMSW3": "0",
      "IfNew_category": "",
      "IfNew_evidenceCitation": "",
      "IfNew_evidenceAuthors": "",
      "IfNew_evidenceYear": "",
      "IfNew_evidenceLink": "",
      "IfNew_nameCitation": "",
      "IfNew_nameAuthors": "",
      "IfNew_nameYear": "",
      "IfNew_nameLink": "",
      "genusTransfersinceMSW3": "0",
      "IfTransfer_oldSciName": "NA",
      "IfTransfer_oldIUCN_ID": "",
      "IfTransfer_evidenceCitation": "",
      "dwc": {
        "scientificName": "Enhydra lutris",
        "genus": "enhydra",
        "subspecificEpithet": "",
        "order": "carnivora",
        "specificEpithet": "lutris",
        "vernacularName": "Sea Otter",
        "family": "mustelidae",
        "namePublishedIn": "",
        "higherClassification": {
          "cohort": "eutheria",
          "magnaorder": "placentalia",
          "superorder": "boreoeutheria",
          "list": "eutheria|placentalia|boreoeutheria"
        },
        "scientificNameAuthorship": {
          "genus": "Linnaeus, 1758",
          "species": "Linnaeus, 1758"
        },
        "taxonRank": "species",
        "class": "mammalia",
        "taxonomicStatus": "accepted",
        "dcterms:bibliographicCitation": "Enhydra lutris (ASM Species Account Database #7750) fetched 2020-05-24T18:03:01+0000",
        "dcterms:language": "en",
        "dcterms:modified": "2020-05-24T18:03:01+0000",
        "dcterms:license": "https://creativecommons.org/licenses/by-nc/4.0/legalcode"
      }
    },

Hi @dimus and @jhpoelen -- Thanks for flagging this.  Yes, we are working on both a new web infrastructure for hosting the updated taxonomy, the CSV for which will always be located here as "mdd.csv" : https://github.com/mammaldiversity/mammaldiversity.github.io/blob/master/_data/mdd.csv
So rather than setting up the GlobalNames resolver to point to our old API, I'd suggest setting things up to point to this file location, which should be constant.

I am very interested in obscure outdated names that are some kind of synonyms to the currently accepted ones. Will your database include such information?
@dimus, yes we are collecting & curating large amount of synonym data as part of this ASM effort, but its not currently ready for prime-time, and so no visible via this CSV download.  Our next full version of the taxonomy (v2) will contain this information though

@jhpoelen I "hackishly" changed ID value  from id to internal_id.
I am desperately need to get verification speed to >= 1000 names/sec on average, so I am starting a new project where I will try to do outlink generation data a bit more precise
https://github.com/gnames/gnames/blob/68d9b5046df7c68cf95eece5e1d0988b1aa611d5/protob/protob.proto#L70
IDs in databases is quite a mess, and are just plain missing totally or partially sometimes. So I think I got to output that takes care of the inconsistencies
        {
          "data_source_id": 184,
          "data_source_title": "ASM Mammal Diversity Database",
          "gni_uuid": "557bc7f6-673b-58d1-8cfb-2832050d898a",
          "name_string": "Enhydra lutris Linnaeus, 1758",
          "canonical_form": "Enhydra lutris",
          "classification_path": "Animalia|Chordata|Mammalia|Carnivora|Mustelidae|Enhydra|Enhydra lutris",
          "classification_path_ranks": "kingdom|phylum|class|order|family|genus|",
          "classification_path_ids": "",
          "taxon_id": "7750",
          "edit_distance": 0,
          "imported_at": "2020-05-25T21:26:16Z",
          "match_type": 2,
          "match_value": "Exact match by canonical form",
          "prescore": "3|0|0",
          "score": 0.988
        }
      ]
    }

@n8upham it is great that you are going to have a permalink for the data! I can see that format of your CSV did not stabilize yet, so I think it makes sense for me to wait for some time until you feel it is not going to change anymore.
I do have a few questions/suggestions/requests about the CSV:

blank field.  I would suggest to remove the BOM character instead.

fields names. Would it be possible to have fields consistent in capitalization. I think most common for CSV files is so-called PascalCase when words are concatenated and each word is capitalized like in ScientificName. A camelCase is common for JSON outputs scientificName, and, to my knowledge, snake_case is common for variables in some languages like Ruby scientific_name.

metadata. Would be great to also have a URL to some metadata, with version and release date.

utf-8, I think you did change encoding to UTF-8 already, thats great!

vernacular names. Would be great to have them back

synonyms. I want them :)

Closing this one for now, we can continue talking...

Thanks for those tips @dimus, and for your interest in our project -- definitely all those things are in progress, we just aren't quite ready for primetime yet as I said.  We did, however, get the BOM character issue solved.  I'll work to standardize some PascalCase names too (I hadn't heard of those specific names for this before, cool!)
Metadata will be coming with our Zenodo DOIs on the official releases, and both vernacular names (ported in from HMW1-9) and synonyms will be coming back with our v2.0 of the MDD, likely in the next 1-2 months.  All the best
--n

great to know @n8upham, and I am happy I could help a little bit. Great to know about Zenodo DOI! I have a few command-line tools that might be of some help for quality control. They work on windows pretty well:
https://gitlab.com/gogna/gnparser
https://github.com/gnames/gnverify


Strains missing associated genes

Examples of strains that should have genes listed:
https://eric.dictybase.dev/stockcenter/strains/DBS0351367
https://eric.dictybase.dev/stockcenter/strains/DBS0351356
https://eric.dictybase.dev/stockcenter/strains/DBS0351312
https://eric.dictybase.dev/stockcenter/strains/DBS0351283
https://eric.dictybase.dev/stockcenter/strains/DBS0351270

Closing in favor of #78


⚠ Bofrost.at has degraded performance

In fedf35d, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1174 ms

Resolved: Bofrost.at performance has improved in 68076a3.


Branchless ipv4 is even faster

I finally had time to check and @aqrit is correct, doing a full branchless ipv4 parsing is faster. It is an interesting case because going branchless improves both the number of instructions (a good reduction) and the number instructions retired per cycle (a more modest gain, but appreciable nonetheless).
It is effectively a simplification of the prior code. I recommend that it'd be merged.

Thanks for reviewing @lemire. And, of course, thanks @aqrit! A very nice addition indeed.

@k0ekk0ek This should get you close to 3 GB/s for IPv4 parsing... It is probably now fast enough not to be a bottleneck.

I think you're right :slightly_smiling_face:


refactor permit

Resolves #???
PR Checklist

[ ] Tests
[ ] Documentation
[ ] Changelog

this was just a demo example


answers at users level

Whats the best way to implement answers up to user level? basically, i want to see every user action, who is clicking what button, and log all activity for every user. In the Answers events dashboard, we cant filter out events based on users and the time they occured. Any ideas?

Answers doesn't do that so you'd have to use something like Mixpanel or KissMetrics


Collectfast and Django 1.7+

The way Collectfast is added to INSTALLED_APPS (here) may be problematic for Django 1.7+, see this thread for more information: https://github.com/antonagestam/collectfast/issues/49#issuecomment-73836059
I'd suggest moving it to common.py and adding COLLECTFAST_ENABLED = False to local.py if Collectfast really isn't wanted there. However I don't really see why you'd want it installed on production only but there might be reasons to that that I don't know of.

Collectfast was added to production only because S3 config were introduced only in production.py. We didn't wanted to have Collectfast as common requirement.
I don't think there is any harm moving it to common.py with proper commented doc, given your explanation above.
@pydanny thoughts?

@antonagestam we have taken care of this in #199. Thanks to @ka7eh!

:+1:


Readonly filegroup causes new database deployment to fail

When deploying a database with a readonly filegroup, the deployment fails.  I think the deploy should set the reaonly status at the end of the deploy

@bdquig -- thank you for your feedback.  Are you following a particular docs.microsoft.com article?

No particular article. We were trying to deploy databases to a new server and our deploy fails.

@bdquig -- thank you for clarifying.  This repo is solely for documentation issues, and this appears to be product-related.
Please consider these resources:

SQL Server on Microsoft Q&A
DBA Stack Exchange
Stack Overflow

You are welcome to raise this issue here:

SQL Server Feedback

I am closing this issue now.  You are welcome to @ mention me for any followup.  We hope to hear from you again.


Add ovobot xtron

A programmable microcomputer that can be used for making MakeCode Arcade games, controlling Scratch sprites and creating works from electronics to robots.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: tsunyi sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

is it possible to order it?

Sorry, I have misunderstood. I will email you later.

@Jaqster ?


SimpleITK <IP_ADDRESS> causes a segfault when reading a truncated JPEG file from scikit-image

At scikit-image, we have observed CI failures (which I am also able to reproduce locally) when running our test suite with SimpleITK <IP_ADDRESS>. The specific failing test case involves reading in a truncated JPEG file and is expected to produce a RuntimeError.
A minimal example to reproducing the segfault outside of the test suite is
from skimage import io
from skimage._shared import testing

io.use_plugin('simpleitk')
truncated_jpg_filename = testing.fetch('data/truncated.jpg')
io.imread(truncated_jpg_filename)

Links to the CI logs with the segfaults are given at https://github.com/scikit-image/scikit-image/issues/6349. relevant section of the traceback:
Current thread 0x00007fdd7a93e740 (most recent call first):
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/SimpleITK/SimpleITK.py", line 8015 in Execute
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/SimpleITK/extra.py", line 346 in ReadImage
  File "/home/runner/work/scikit-image/scikit-image/skimage/io/_plugins/simpleitk_plugin.py", line 15 in imread
  File "/home/runner/work/scikit-image/scikit-image/skimage/io/manage_plugins.py", line 207 in call_plugin
  File "/home/runner/work/scikit-image/scikit-image/skimage/io/_io.py", line 53 in imread
  File "/home/runner/work/scikit-image/scikit-image/skimage/io/tests/test_simpleitk.py", line 48 in test_imread_truncated_jpg

We do not see this segfault with prior versions of SimpleITK or when testing locally with 2.2.0rc2 or the latest development binary.

This reported issue appears to be with with Python 3.8 with the manylinux 2014 x86_64 wheel. Is the issue reproducible on other platforms or wheels?

I also see it locally on a linux system with either Python 3.9 or 3.10. The specific wheels used are:
SimpleITK-<IP_ADDRESS>-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
SimpleITK-<IP_ADDRESS>-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

have not tried on other platforms yet

On my intel Mac with Python 3.8, I got the following:
>>> io.imread(truncated_jpg_filename)
Premature end of JPEG file
Bogus Huffman table definition
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/blowekamp/miniconda3/envs/skimage/lib/python3.8/site-packages/skimage/io/_io.py", line 53, in imread
    img = call_plugin('imread', fname, plugin=plugin, **plugin_args)
  File "/Users/blowekamp/miniconda3/envs/skimage/lib/python3.8/site-packages/skimage/io/manage_plugins.py", line 207, in call_plugin
    return func(*args, **kwargs)
  File "/Users/blowekamp/miniconda3/envs/skimage/lib/python3.8/site-packages/skimage/io/_plugins/simpleitk_plugin.py", line 15, in imread
    sitk_img = sitk.ReadImage(fname)
  File "/Users/blowekamp/miniconda3/envs/skimage/lib/python3.8/site-packages/SimpleITK/extra.py", line 346, in ReadImage
    return reader.Execute()
  File "/Users/blowekamp/miniconda3/envs/skimage/lib/python3.8/site-packages/SimpleITK/SimpleITK.py", line 8015, in Execute
    return _SimpleITK.ImageFileReader_Execute(self)
RuntimeError: Exception thrown in SimpleITK ImageFileReader_Execute: /Users/runner/work/1/sitk-build/ITK/Modules/IO/JPEG/src/itkJPEGImageIO.cxx:309:
ITK ERROR: JPEGImageIO(0x7f970e219e30): Error JPEGImageIO could not open file: /Users/blowekamp/Library/Caches/scikit-image/0.19.2/data/truncated.jpg

I am assuming that is how the test is suppose to fail.

I was able to reproduce the segmentation fault on Linux too with SimpleITK-<IP_ADDRESS>-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl. Interestingly when I tried the older manylinux 2010 wheel: SimpleITK-<IP_ADDRESS>-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl there was no segmentation fault.
I am suspicious that there is some conflicting C++ symbol similar to the issues linked in this change:
https://github.com/SimpleITK/SimpleITK/commit/ada61a499c68c41999cb442b5e79c328410e2d47

The manylinux2014 wheels were not in 2.1.1 but adding with <IP_ADDRESS>.
The manylinux 2014 wheel work in 2.2rc2 and the current nightly 2.2.0rc2.post68 wheels for x86_64 python 3.8. However, this above test still seg faults for 2.2rc1.
Can you please confirm that the manylinux2010 wheels work?
We are considering yanking or deleting just the manylinux2014 wheels from <IP_ADDRESS> while leaving the manylinux2010 wheels.

Thanks, I can confirm that the SimpleITK-<IP_ADDRESS>-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl works fine in local testing. I guess if you pull the manylinux2014 wheels then we won't even have to worry about updating the pinning in scikit-image. Thanks for the quick response on this!

The manylinux2014 wheels have been yanked. Which seems to resolve the failing test.
The underlying issue was in ITK <v5.3. It was reported in InsightSoftwareConsortium/ITK#2928 and fixed in PR InsightSoftwareConsortium/ITK#2933. We are anxiously awaiting using ITK v5.3.0 in SimpleITK 2.2 to contain this fix.

Thanks, I will go ahead and close as the broken wheels were removed and a fix has already been made for ITK 5.3.


Update docs to promote useOnyx usage over withOnyx

Details

Based on this Slack 🧵 discussion, this PR is updating the docs to promote the usage of the useOnyx hook over the withOnyx HOC which is deprecated and should not be used anymore, except if absolutely necessary for example if using React class components where hooks cannot be used.
cc @roryabraham @fabioh8010
Related Issues

GH_LINK
Automated Tests

Manual Tests

Author Checklist

[x] I linked the correct issue in the ### Related Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android / native
[x] Android / Chrome
[x] iOS / native
[x] iOS / Safari
[x] MacOS / Chrome / Safari
[x] MacOS / Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] If we are not using the full Onyx data that we loaded, I've added the proper selector in order to ensure the component only re-renders when the data it is using changes
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[x] I have checked off every checkbox in the PR author checklist, including those that don't apply to this PR.

Screenshots/Videos

Android: Native

Android: mWeb Chrome

iOS: Native

iOS: mWeb Safari

MacOS: Chrome / Safari

MacOS: Desktop

I have read the CLA Document and I hereby sign the CLA

@fabioh8010 I think I addressed all the change requests, and some extra docs UI/UX related things like paragraph spacing for better readability and the markdown deprecation warning template for better notice/visualisation of the deprecated HOC.
Please take a look when you get a chance and let me know if there's anything else you'd do / write differently.