API node update endpoint grants elevated permissions to users

Summary
To a limited extent, the API makes it possible to circumvent normal access restrictions, and let users with limited permissions modify a project
Steps to reproduce

As User A, create an OSF project, with visibility set to "private" (the default)
Go to the Sharing tab and give Read+Write permissions to User B.
In a separate browser (new session), log in as User B. Confirm that you can reach the OSF page for the project, but do not have the option to make the project public.
As User B, visit the API page for the newly created node (URL of the form http://localhost:8000/v2/nodes/<project_id>/)
Confirm that data can be seen.
Using the DRF web view,at the bottom of the page, change "public" from false to true.
Use the DRF to send a PATCH request. (Make sure to remove the {data: } wrapping the request body)
As User A, refresh the project page. The toggle should now appear as public.

Expected result
The API would enforce fine-grained permissions consistent with the current front-end behavior
Actual result
The API allows users to modify a project in ways they normally could not.
Here is the view as Sam Pull, a Read+Write contributor. Note the lack of visibility toggle + log entry saying he made the change.

Further work
Further investigation is needed to determine how far permissions escalation could go. (eg can read-only users also make a project public?) At the moment I'm having some issues with caching that make this difficult to test. (The DRF does not seem to be immediately reflecting when I log out of the OSF, or use the front-end to change an instance)

I think we should consider disabling any admin-level functionality from the API for the moment.
Also, test driven debugging would be super-handy in this instance.

Per @brianjgeiger , these tests might be coordinated with #69 . I'll get to this Mon-Tues if no one else has yet.

Okay, I've made public read_only as of 2c99d61f9bb2322e1717eec4ea1d1322177e3410 and @pattisdr is revising tests to make sure that all works properly here.


ILB is not working after deploying K8S 1.9.3

Is this a request for help?:
YES

Is this an ISSUE or FEATURE REQUEST? (choose one):
ISSUE
What version of acs-engine?:
0.13.1

Orchestrator and version (e.g. Kubernetes, DC/OS, Swarm)
1.9.3
What happened:
We regenerated the cluster in order to upgrade our acs-engine + Kubernetes versions and deployed the NGINX with a ILB service and we loose connection from a paired VNET.
What you expected to happen:
ILB needs to be working.
How to reproduce it (as minimally and precisely as possible):
Using two vnets (one for the cluster and one for the api management) and deploy the cluster on 1 of the vnets.
Anything else we need to know:
No

There are known ILB issues with v1.9.3, this has been fixed in v1.9.6 (acs-engine v0.14.4)


Raw HTML displayed on demo site https://juliacomputing.github.io/MultiDocumenter.jl/

Going to https://juliacomputing.github.io/MultiDocumenter.jl/ I get redirected to https://juliacomputing.github.io/MultiDocumenter.jl/inf/stable/ which shows some raw HTML, which I think is meant to insert the Infiltrator.jl logo.

In contrast this is how it look at https://juliadebug.github.io/Infiltrator.jl/stable/

This was an issue in Infiltrator's docs, which got fixed in 1.7 or 1.8. The page at https://juliacomputing.github.io/MultiDocumenter.jl/stable/ didn't get


chore(package): update npm-pkgbuild to version 3.1.1

:tada: This PR is included in version 1.1.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Text selection color is ugly

I find the text highlight color highly unsatisfactory; it reminds me of Windows 95's default theme. Can we change it to a nicer blue? I recommend #b9eaff or something similar.

Personally I find the current color just fine.

Agreed, and calling the existing colour "ugly" is highly subjective anyway; closing as WONTFIX?


hello i m running this code in jupyter notebook bt it is giving error plz can u tell me how i shld debug this i tried all ways for this

error                                     Traceback (most recent call last)
 in
1 main_img = cv2.imread(test_img_path)
----> 2 img = cv2.cvtColor(main_img, cv2.COLOR_BGR2RGB)
3 plt.imshow(img)
error: OpenCV(3.4.1) C:\Miniconda3\conda-bld\opencv-suite_1533128839831\work\modules\imgproc\src\color.cpp:11115: error: (-215) scn == 3 || scn == 4 in function cv::cvtColor

Hi,
Can you provide a proper documentation of the exact problem along with a supporting picture. Also do check if you are able to load main_img (ie, line 1). If not, then there is an issue with test img path


Rewrite equals methods

A while ago on IRC, @davidfestal pointed out to me that ceylon.ast’s source-gen’d equals methods are silly: Instead of, for instance,
if (exists arguments) {
    if (exists arguments_ = that.arguments) {
        return arguments == arguments_ && name == that.name;
    } else {
        return false;
    }
} else {
    if (!(that.arguments exists)) {
        return name == that.name;
    } else {
        return false;
    }
}

it would be much better to have
if (exists arguments) {
    if (exists arguments_ = that.arguments) {
        if (arguments != arguments_) {
            return false;
        }
    } else {
        return false;
    }
} else if (that.arguments exists) {
    return false;
}
return name == that.name;

Thus, if a node has several optional children, those are checked sequentially, not (as they are now) nested. For nodes with lots of optional children, like ClassDefinition, this would remove a lot of duplicate code. (It also makes it easier to update the equals method later, e. g. for #78.)
I should try to write a small program that rewrites the equals methods like this, possibly even using ceylon.ast to analyze the class (we need to get the types of the parameters).

possibly even using ceylon.ast to analyze the class (we need to get the types of the parameters)

Not actually necessary, I think. It’s probably much easier to just take the first return statement in the equals method – that’s the one for the branch where all optional attributes exist – and then analyze its expression, which will be a conjunction of equality tests between either x == that.x (non-optional) or x == x_ (optional) – in other words, I can simply check for the trailing underscore to see if it’s optional or not, which is all the information I need.

Done. Comparison here: 748 lines shorter!


--filter-labels isn't properly filtering CRD

What steps did you take:
[A clear and concise description steps and any files that can be used to reproduce the problem.]
:; kapp deploy -a someapp -y -f foo.yaml --filter-labels=a.b.c/bar=qaz

What happened:
[A small description of the result]
kapp complained about not finding custom resource types for several CRDs. the CRDs should have been filtered out (read: excluded from consideration) by the --filter-labels flag, but kapp was still processing them somehow.
What did you expect:
[A description of what was expected]
expected kapp to ignore all resources in foo.yaml that do not have labels matching the specified filter
Anything else you would like to add:
[Additional information that will assist in solving the issue.]
Environment:

kapp version (use kapp --version): 0.42.0
OS (e.g. from /etc/os-release):
Kubernetes version (use kubectl version)

Vote on this request
This is an invitation to the community to vote on issues, to help us prioritize our backlog. Use the "smiley face" up to the right of this comment to vote.
👍 "I would like to see this addressed as soon as possible"
👎 "There are other more important things to focus on right now"
We are also happy to receive and review Pull Requests if you want to help working on this issue.

xref #272

Hey @jdef ,
Could share your config please.
Thanks

Also, please try kapp deploy -a someapp -y -f foo.yaml --filter-labels=a.b.c/bar!=qaz to exclude.
(the "not equal" to)

Sorry I cannot share my yaml or copy paste the exact error messages. These nodes are airgapped, and copy/paste isn't possible. Here's a "by hand" re-typed version of the error message, redacted appropriately.
Target cluster 'https://xxxxxxx:port/' (nodes: xxxxx)

kapp: Error: Expected to find kind 'constraints.gatekeeper.sh/v1beta1/CustomType', but did not:
- Kubernetes API server did not have matching apiVersion + kind
- no matching CRD was found in given configuration

Consider the yaml configuration:
---
kind: CustomType
apiVersion: constraints.gatekeeper.sh/v1beta1
metadata:
  labels:
    a.b.c/bar=boo
  ...

---
kind: ConfigMap
metadata:
  labels:
    a.b.c/bar=qaz

My expectation is that --filter-labels a.b.c/bar=qaz would mask the CustomType resource so that kapp deploy ... would only consider the ConfigMap resource.

@jdef confirmed as a challenge with following setup
kind: Foo        # unknown kind to the cluster
apiVersion: v1
metadata:
  name: foo
  labels:
    kind-lbl: foo
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: cm
  labels:
    kind-lbl: cm

kapp deploy -a test -f <(pbpaste) --filter-labels kind-lbl=cm results in the same error as mentioned above due to Foo kind not being known to kapp (and to the cluster).
reason: filtering currently is done after preparation (internal step to fill in namespace, label resources, etc). one challenge i see with moving filtering to be done earlier is that namespace filter wont properly work. not yet sure how i feel about this.
alternative solution: can use carvel.dev/ytt to "shape" your configuration before passing it onto kapp. you can use ytt overlays to remove any resource with particular label (or without label, etc.).
additional note: we are working on a PR that actually makes it possible for gatekeeper constraints and gatekeeper installation itself to live in the same "kapp deploy" (https://github.com/vmware-tanzu/carvel-kapp/pull/319/files).

I'm actually trying to replace our existing preprocessing w/ the --filter-labels flag :)  So something like ytt isn't really a step forward at this point.
Happy to hear about #319 , we are interested in that too. In this case, our automation is trying to deploy something completely unrelated to gatekeeper - it just so happens that this is the "type not found" bump that we hit first.

I'm actually trying to replace our existing preprocessing w/ the --filter-labels flag

I would like to understand more about what kind of preprocessing you are trying to do.
In case you just want kapp to ignore the CustomResources and you are comfortable doing it without the labels, then you can use the kapp.k14s.io/noop annotation.
However, you would have to provide the basic details of the CustomeResourceDefinition for the missing CustomTypes.
So, for the yaml you had provided you would have to do something like
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: customtype.constraints.gatekeeper.sh
  annotations:
    kapp.k14s.io/noop: ""
spec:
  group: constraints.gatekeeper.sh
  versions:
    - name: v1beta1
  names:
    kind: CustomType
---
kind: CustomType
apiVersion: constraints.gatekeeper.sh/v1beta1
metadata:
  name: foo
  annotations:
    kapp.k14s.io/noop: ""

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: bar
  namespace: kapp-test

And then kapp deploy -a test -f <(pbpaste) would only deploy the config map and ignore the other resources.
(This does seem a bit too much but just thought of sharing as a possible workaround)


fix: implement appDir studio loading

It's not supported to import the sanity package in RSC routes, it only works if you're in a use client route.
Thus client config for projectId, dataset etc needs to be in a tiny file that can be safely imported in both server-only and client-only environments.

Ah, wow, did not expect a PR! Amazing. This works a lot better, but something still seems to be wrong with the code inputs. They render like this:

This works a lot better, but something still seems to be wrong with the code inputs.

Fixed in 3.0.1.


Generate sha256 message digest for built binaries

Relates to #306
This work generates files that contain the sha256 message digest for each binary. I'm not sure how you go about populating artifacts to the github release but would be able to do it if someone could point me in the right direction.

I think the best approach right now would be to produce a text file with the hashes for each artifact, since we manually upload all artifacts to GitHub. We should probably add a release pipeline to Bazel CI, and might explore something like SLSA in the future.


[Admin] Cross link between availability and common API pages

Related to https://github.com/OfficeDev/office-js-docs-reference/issues/739.
Also includes some formatting cleanup.

@Rick-Kirkham / @AlexJerabek : My take on the changes we can make now.

Docs Build status updates of commit c366a41:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/overview/office-add-in-availability.md
:white_check_mark:Succeeded
View

docs/reference/requirement-sets/office-add-in-requirement-sets.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit 3050606:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/overview/office-add-in-availability.md
:white_check_mark:Succeeded
View

docs/reference/requirement-sets/office-add-in-requirement-sets.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel


MVS not present

https://github.com/libuv/libuv/blob/04289fa326b790c1a4abb236d1f9d913bacfc8c6/src/unix/thread.c#L702

It is just a comment, it doesn't have to be exact, just useful


Monomorphize all geometry primitives to use f64

Why is this better?

The generics buy the ability to instantiate code with f32 (f32 and f64 are the only float types in Rust). However, queries in global coordinates (anything in ECEF, Web Mercator tile, S2 cell) require f64 precision anyway.
De facto everything is always f64 (see iterator.rs, including the query and point coordinate type), and changing it would require a bunch of refactoring anyway. This just makes it official :)
Code becomes nicer to read and maintain: Fewer trait bounds everywhere, no more nalgebra::convert() etc.
More things can be constants instead of run-time values, like std::f64::consts::PI instead of S::pi().
Compile times should improve a bit.

meow @feuerste

Btw, should I monomorphize the SAT code as well? Probably yes, right?

Yes

Thanks for reviewing!


BUG - Guided Answers shows a branch which does not exist

Related Feature
Feature request: Initial Release
Description
Guided Answers shows a branch which does not exist
Steps to Reproduce
Steps to reproduce the behavior:
Part 1

Launch GA web application
https://ga.support.sap.com/dtp/viewer/index.html#/tree/3046/actions/45995:45996:50742
// Here you will notice that only 1 branch is there: Cloud Foundry

Part 2

Now in VSCode, launch Guided Answers extension by SAP
Navigate to the same exact location where you are in Part 1 above.
i.e. SAP Fiori tools->Deployment->SAP Business Appplication Studio
Click on ABAP on premise

Expected results
ABAP on premise is not shown
Actual results
ABAP on premise is shown (See screenshots below)
Screenshots
View in Guided Answers web application:

View in VSCode extension:

Version/Components/Environment
Guided Answers extension version: https://github.com/SAP/guided-answers-extension/releases/download/sap-guided-answers-extension%400.4.0/sap-guided-answers-extension-0.4.0.vsix
OS:

[X] Mac OS
[ ] Windows
[ ] Other

Root Cause Analysis
Problem
{describe the problem}
Fix
{describe the fix}
Why was it missed
{Some explanation why this issue might have been missed during normal development/testing cycle}
How can we avoid this
{if we don’t want to see this type of issues anymore what we should do to prevent}

Hi @theStiggySpd,
Here is the API call response for the node 50742: https://ga.support.sap.com/dtp/api/nodes/50742
{
  NODE_ID: 50742,
  TITLE: "SAP Business Application Studio",
  BODY: "
In SAP Business Application Studio, what is the target environment you are deploying to?
",
  QUESTION: "I'm deloying to...",
  EDGES: [
    {
      LABEL: "Cloud Foundry",
      TARGET_NODE: 51205,
      ORD: 1
    },{
      LABEL: "ABAP On Premise",
      TARGET_NODE: 51206,
      ORD: 2
    }
  ]
}

I clearly returns two EDGES.
Perhaps a server caching issue? Was there recently a change to the node?
@JonWiens, can you help here?

ok i found the reason. So, in editor, we had added the outcome as ABAP on premise but never filled in the details.
Guided Answers web application in that scenario does not shows the outcome, where extension via API which still shows valid outcome (even though content is empty).
I removed this empty outcome and issue went away. We should still handle this gracefully. Lets see what @JonWiens suggests?

@theStiggySpd - When you preview a Guided Answer with an outcome that has no details it is simply just not shown. Should we not follow this approach in the extension? If there is no content to show why show the option?

@carwelsh: while I believe we could address this issue on extension side, I strongly recommend that the API should return only valid EDGES. @JonWiens what do you think?
Reason
If we as consumer would address this, we need to load all EDGES (nodes behind buttons) and validate them, which might have a negative impact on performance. Just think of a scenario where we have many options, like this one:
https://ga.support.sap.com/dtp/viewer/index.html#/tree/1910/actions/24709:34441

In this case, we would need to perform 13 (!) additional requests just to check if all of the options (EDGES) are valid nodes.

@Klaus-Keller @JonWiens - Yes I agree this would be a better approach.

@carwelsh and @ritarora, do you know if this is still an issue or has this been fixed on API side? Could you please retest?

@carwelsh, I think we've tried to reproduce this in a meeting but have not been able to, so I guess this is solved on API side.


Connection string option to create empty for on Open new collection?

When a new collection if opened, LiteDB does not write a file until the first document is written.
If think this is fine by the way.
However, in my application I want to determine whether a collection has been opened (even if it has no documents). I solved this problem by writing an empty placeholder file myself, and this seems to work OK. I do have slight concerned, however, that in some future version of LiteDB it may fail to open a collection where there is an empty placeholder file because it is expecting some header information of some kind (perhaps my concern is not valid).
Would it be possible to add an a connection string option to force the creation of an initial file?
(I thought the "Initial Size" option would do this, but it doesn't apparently. It appears to default to a value based on page size anyway.)

Hi @kuiperzone, another way to solve this problem is use EnsureIndex in empty collection. This will create collection with no documents and adding index definition.
It's complicated call this when you "open" and collection because just GetCollection do not use any datafile operation. To make a clean solution it's better expose this method in LiteEngine:
https://github.com/mbdavid/LiteDB/blob/master/LiteDB/Engine/LiteEngine.cs#L152


Accessibility Fixes

Fixes #133

Codecov Report

Merging #273 (5e291fa) into master (5331e94) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #273   +/-   ##
=======================================
  Coverage   64.74%   64.74%
=======================================
  Files          74       74
  Lines        1713     1713
  Branches      242      242
=======================================
  Hits         1109     1109
  Misses        410      410
  Partials      194      194

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5331e94...5e291fa. Read the comment docs.


Is it possible to "pause progress" in Mercury?

Hi,
We are currently having some issues with Mercury and fork()/clone() system calls due to fds and the internal epoll() set in the parent being duplicated in the new process. Since we are not calling exec() immediately after fork() we end up with an inconsistent state of the copied Mercury library in the child process.
A possible solution to our problem that we are currently exploring is to intercept the fork()/clone() system calls and, when one does arrive, "pause" Mercury from making progress, execute the actual fork() and resume Mercury progress in the parent and reinitialize the library in the child. I was wondering whether stopping our progress thread would be enough to suspend Mercury progress completely or if there's further progress made in the NA layer.

Hmm I think I would need to think about all the implications of doing that. If you pause the progress thread, would you still be making HG calls? It is currently still possible for some of those calls to call into non-blocking progress.

Our current progress thread does the following (feel free to point out any issues with it):
    void
    progress_thread() {

        assert(m_hg_class);
        assert(m_hg_context);

        hg_return ret;
        unsigned int actual_count;

        while(!m_shutdown) {
            do {
                ret = ::HG_Trigger(m_hg_context,
                                 0,
                                 1,
                                 &actual_count);

                DEBUG4("HG_Trigger(context={}, timeout={}, "
                              "max_count={}, actual_count={}) = {}",
                              fmt::ptr(m_hg_context), 0, 1, actual_count,
                              ::HG_Error_to_string(ret));

            } while((ret == HG_SUCCESS) &&
                    (actual_count != 0) &&
                    !m_shutdown);

            if(!m_shutdown) {
                ret = ::HG_Progress(m_hg_context, 100);

                DEBUG4("HG_Progress(context={}, timeout={}) = {}",
                              fmt::ptr(m_hg_context), 100,
                              ::HG_Error_to_string(ret));

                if(ret != HG_SUCCESS && ret != HG_TIMEOUT) {
                    FATAL("Unexpected return code {} from HG_Progress: "
                                 "{}", ret, ::HG_Error_to_string(ret));
                }
            }
        }
    }

It would certainly be possible to stop the progress thread in a similar way as it's done with m_shutdown, which would stop the thread from calling HG_Progress(). From my (admittedly limited) understanding the Mercury code, this function eventually calls epoll_wait(), which tells the kernel to deliver any pending requests to the calling process, so stopping the parent process from doing this would leave us in a "safe state" where requests are processed by the kernel, but not yet delivered to any process, which would allow us to fork() the process safely.
I'm not sure, however, about what happens at the NA layer: are there further calls to epoll_wait() (or accept() or any similar calls) done at the NA layer? If that were the case, it might not be sufficient to stop our progress thread to fork() the process safely.

If you stop the progress thread, there are no other epoll_wait() calls so you should be safe on this side. I'm wondering if you will not encounter other issues though when using libfabric etc.


How can we change the libbpf  log level ?

If you play with:
https://github.com/aquasecurity/libbpfgo/blob/main/libbpfgo.go#L36
(and return vfprintf() in all cases, you will get all the debug information from libbpf during libbpfgo execution as well.
I think we're missing an automated way of setting different loglevels though.

This is a duplicate of https://github.com/aquasecurity/libbpfgo/issues/55


Exception cloning elements

This issue only happens with certain HTML, but if you run the following snippet of code that tries to call cloneNode(true) it will throw a NamespaceError.
var jsdom = require('jsdom');

jsdom.env('http://www.ksl.com/?nid=148&sid=30067208', function(errors, window) {
  window.document.body.cloneNode(true);
});

I know it's some element in the DOM tree causing the exception, but I haven't narrowed it down to which exactly is causing it to throw. Anyway, I don't know that there's any condition where cloning a chunk of the DOM is supposed the throw an exception.
I don't know if it's related but at trying to clone with the following URL will throw a different exception. It throws InvalidCharacterError:
http://www.ksl.com/?sid=18035498

Nice find. This is almost certainly because we are using the normal constructors/etc. for creating elements or attributes, and those constructors throw such errors in the case of invalid names. However, browsers always use secret construction methods which allow invalid names while cloning. Should be easy to fix once we track down the offending line. PR welcome if you want to dig into it, or I can probably get around to it within a day or two.

Ok, I wrote up a script and tracked that first error down to an element in the page that looks like this:
<http://KSL.com>

It looks like an invalid element without a closing tag and there are two of them in the same sentence. It looks like the browser just ignores it completely.
For the second error (the InvalidCharacterError) it looks like it's caused by a weird snippet in the HTML at the end of one of the paragraphs of the article that looks like this:
<Home-Schooling</b>

Again, the browser just strips out that invalid HTML and continues on, leaving the rest of the text around it intact.
I don't know if it will be useful to you but here's the script I wrote to track those down:
var jsdom = require('jsdom');

// var url = 'http://www.ksl.com/?nid=148&sid=30067208';
var url = 'http://www.ksl.com/?sid=18035498';

jsdom.env(url, function(errors, window) {
  var errPath = [ window.document.body ];

  for(var i = 0; i < errPath.length; ++i) {
    var curEl = errPath[i],
        cNodes = curEl.childNodes;

    for(var j = 0; j < cNodes.length; ++j) {
      var cNode = cNodes[j];

      try {
        cNode.cloneNode(true);
      } catch(err) {
        errPath.push(cNode);
        break;
      }
    }
  }

  renderPath(errPath);
});

function renderPath(nodes) {
  for(var i = 0; i < nodes.length; ++i) {
    var node = nodes[i],
        spaces = Array(i + 1).join('  '),
        nodeAttrs = node.attributes,
        attrs = [];

    for(var j = 0; j < nodeAttrs.length; ++j) {
      attrs.push(nodeAttrs[j].name + '="' + nodeAttrs[j].value + '"');
    }

    console.log(`${spaces}<${node.tagName}${attrs.length ? ' ' + attrs.join(' ') : ''}>`);
  }
}

So, 5.5.0 is out with this fix! Sorry it took so long to get a release; normally we're pretty fast about this sort of thing.

It works great! Thanks for the fix!


Add optional support for Object Storage to GitLab.com ELT

Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/40
Originally created by @joshlambert on 2018-05-02 21:46:55

Currently our GitLab.com ELT writes the data into CSV files locally on disk, and then we have a CI job which picks them up and writes them into the data warehouse.
This works fine if the BizOps CI jobs and rake task are running in the same segment, but if you have these separated for security reasons you will need to figure out moving the files yourself.
It would be nice to add direct support to move these up to and down from object storage, to reduce the burden on end users.

View 8 previous comments from the original issue on GitLab


[DO NOT MERGE]Remove QA to content feature flag

Related PR 8427 in govuk-puppet

closing because https://github.com/alphagov/finder-frontend/pull/727 now includes 7a6fb41 which does this.


What's the meaning of "ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions"

Hello,
I try to use your package. I pass it a panda series of events. Those events are storms. I provide a time series of storm durations (in hours) and storm begin time (index of the series). There can be several storms at the same time and there are times without storms. Then I run:
model = EVA(pd.Series(durations, np.sort(dates))) # Dates are not important, durations are.
model.get_extremes(method="BM", block_size="10D")
model.plot_extremes()

which ends up with:
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (731,) + inhomogeneous part.

I believe you expect data at regular intervals (and so I must fill in some gaps). Correct ? I'll continue fiddling a bit. This issue is more about giving an error message one (novice) can understand.

More information. I understand the error comes out of pandas, not your code directly. Just for information, my data looks like:
model = EVA(pd.Series(durations, np.sort(dates)))
print(dates, dates.dtype)
print(durations, durations.dtype)

['2002-01-01T16:00:00.000000000' '2002-01-01T20:00:00.000000000'
 '2002-01-02T03:00:00.000000000' ... '2004-10-02T23:00:00.000000000'
 '2004-10-03T07:00:00.000000000' '2004-10-03T13:00:00.000000000'] datetime64[ns]
[20. 18.  7. ...  4.  5.  7.] float64

Full stack trace:
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Input In [115], in <module>
----> 1 model.get_extremes(method="BM", block_size="10D")
      2 model.plot_extremes()

File ~/.local/lib/python3.9/site-packages/pyextremes/eva.py:452, in EVA.get_extremes(self, method, extremes_type, **kwargs)
    450 message = f"for method='{method}' and extremes_type='{extremes_type}'"
    451 logger.debug("extracting extreme values %s", message)
--> 452 self.__extremes = get_extremes(
    453     method=method,
    454     ts=self.data,
    455     extremes_type=extremes_type,
    456     **kwargs,
    457 )
    458 self.__extremes_method = method
    459 self.__extremes_type = extremes_type

File ~/.local/lib/python3.9/site-packages/pyextremes/extremes/extremes.py:59, in get_extremes(ts, method, extremes_type, **kwargs)
     13 """
     14 Get extreme events from time series.
     15
   (...)
     56
     57 """
     58 if method == "BM":
---> 59     return get_extremes_block_maxima(
     60         ts=ts,
     61         extremes_type=extremes_type,
     62         **kwargs,
     63     )
     64 if method == "POT":
     65     return get_extremes_peaks_over_threshold(
     66         ts=ts,
     67         extremes_type=extremes_type,
     68         **kwargs,
     69     )

File ~/.local/lib/python3.9/site-packages/pyextremes/extremes/block_maxima.py:148, in get_extremes_block_maxima(ts, extremes_type, block_size, errors, min_last_block)
    137     warnings.warn(
    138         message=f"{empty_intervals} blocks contained no data",
    139         category=NoDataBlockWarning,
    140     )
    142 logger.debug(
    143     "successfully collected %d extreme events, found %s no-data blocks",
    144     len(extreme_values),
    145     empty_intervals,
    146 )
--> 148 return pd.Series(
    149     data=extreme_values,
    150     index=pd.Index(data=extreme_indices, name=ts.index.name or "date-time"),
    151     dtype=np.float64,
    152     name=ts.name or "extreme values",
    153 ).fillna(np.nanmean(extreme_values))

File ~/.local/lib/python3.9/site-packages/pandas/core/series.py:439, in Series.__init__(self, data, index, dtype, name, copy, fastpath)
    437         data = data.copy()
    438 else:
--> 439     data = sanitize_array(data, index, dtype, copy)
    441     manager = get_option("mode.data_manager")
    442     if manager == "block":

File ~/.local/lib/python3.9/site-packages/pandas/core/construction.py:570, in sanitize_array(data, index, dtype, copy, raise_cast_failure, allow_2d)
    567     data = list(data)
    569 if dtype is not None or len(data) == 0:
--> 570     subarr = _try_cast(data, dtype, copy, raise_cast_failure)
    571 else:
    572     subarr = maybe_convert_platform(data)

File ~/.local/lib/python3.9/site-packages/pandas/core/construction.py:760, in _try_cast(arr, dtype, copy, raise_cast_failure)
    755         subarr = maybe_cast_to_integer_array(arr, dtype)
    756     else:
    757         # 4 tests fail if we move this to a try/except/else; see
    758         #  test_constructor_compound_dtypes, test_constructor_cast_failure
    759         #  test_constructor_dict_cast2, test_loc_setitem_dtype
--> 760         subarr = np.array(arr, dtype=dtype, copy=copy)
    762 except (ValueError, TypeError):
    763     if raise_cast_failure:

ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (731,) + inhomogeneous part.

@wiz21b can you share your data so that I can reproduce your error? This is not meant to happen because EVA pre-processes the data during initialization - this may be a scenario I didn't account for.
Also data doesn't have to be at regular intervals.

what was the solution? I'm having the same issue

Cool thanks. I'm getting the same error as wiz21b when I run the get_extremes command. I also get the error when I try to execute some of the plot functions and POT functions. I haven't been able to figure out why.
I'm following exact tutorial case but using data from a different noaa station using the NOAA_COOPS function to download the data into a dataframe.
tide_gauge=noaa_coops.Station(8775237)
#https://api.tidesandcurrents.noaa.gov/api/prod/#products
df_water_levels=tide_gauge.get_data(
begin_date="20040406",
end_date="20220925",
product="water_level",
datum="NAVD",
units="english",
time_zone="LST")
I then normalize the dataset by adjusting for RSLR:
measured_rslr=5.54*0.00328084 #ft/yr
df_water_levles_corrected=df_water_levels['water_level'].copy().sort_index(ascending=True).astype(float).dropna()
df_water_levels_corrected=df_water_levels_corrected-(df_water_levels_corrected.index.array-pd.to_datetime("1992"))/pd.to_timedelta("365.2425D")*measured_rslr
am=pyextremes.EVA(df_water_levels_corrected)
Univariate Extreme Value Analysis
                                  Source Data

Data label:                   water_level      Size:                           1,408,862
Start:                         April 2004      End:                       September 2022
                                 Extreme Values

Extreme values have not been extracted
                                     Model

Model has not been fit to the extremes
Everything works up until this point and here is the command that results in the error:
am.get_extremes(method="BM", block_size="365.2425D",errors="ignore")
am.get_extremes(method="BM", block_size="365.2425D",errors="ignore")
C:\Users: NoDataBlockWarning: 1 blocks contained no data
warnings.warn(
Traceback (most recent call last):
Input In [94] in <cell line: 1>
am.get_extremes(method="BM", block_size="365.2425D",errors="ignore")
File ~.conda\envs\work\lib\site-packages\pyextremes\eva.py:452 in get_extremes
self.__extremes = get_extremes(
File ~.conda\envs\work\lib\site-packages\pyextremes\extremes\extremes.py:59 in get_extremes
return get_extremes_block_maxima(
File ~.conda\envs\work\lib\site-packages\pyextremes\extremes\block_maxima.py:148 in get_extremes_block_maxima
return pd.Series(
File ~.conda\envs\work\lib\site-packages\pandas\core\series.py:451 in init
data = sanitize_array(data, index, dtype, copy)
File ~.conda\envs\work\lib\site-packages\pandas\core\construction.py:594 in sanitize_array
subarr = _try_cast(data, dtype, copy, raise_cast_failure)
File ~.conda\envs\work\lib\site-packages\pandas\core\construction.py:784 in _try_cast
subarr = np.array(arr, dtype=dtype, copy=copy)
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (18,) + inhomogeneous part.

@coastalmodeler thank you for your input. In order to help you I'll need to be able to reproduce your error. Please provide the following:

OS
Python version
pyextremes version
numpy version
pandas version

In addition to that I'll need a complete code snippet which can be run as is. For example:
import pandas as pd
import pyextremes

data = pd.read_csv("data.csv")
model = pyextremes.EVA(data)
model.get_extremes()

And provide a link to your data.csv. You can also make a GitHub gist with jupyter notebook if that's what you prefer.

Thank you for your quick response. See info and code below.

OS: Microsoft Windows 10
Python Version: 3.8.13
pyextremes version: 2.2.4
numpy version: 1.21.5
pandas version: 1.4.3
noaa_coops version: 0.1.9

Here's the code. Note, there is no CSV file I'm using noaa-coops to download the data directly into python from the API. The noaa coops wrapper can be found here: https://pypi.org/project/noaa-coops/
import noaa_coops as nc
import pyextremes
import numpy as np
import pandas as pd

tide_gauge=nc.Station(8775237)

#https://api.tidesandcurrents.noaa.gov/api/prod/#products
df_water_levels=tide_gauge.get_data(
    begin_date="20040406",
    end_date="20220925",
    product="water_level",
    datum="NAVD",
    units="english",
    time_zone="LST")

measured_rslr=5.54*0.00328084

df_water_levels_corrected=df_water_levels['water_level'].copy().sort_index(ascending=True).astype(float).dropna()

df_water_levels_corrected=df_water_levels_corrected-(df_water_levels_corrected.index.array-pd.to_datetime("1992"))/pd.to_timedelta("365.2425D")*measured_rslr

am=pyextremes.EVA(df_water_levels_corrected)
am.get_extremes(method="BM",errors="ignore")

I think I found the issue. There were duplicate time steps in the NOAA dataset. Once I removed those the code works as intended. I'd guess that was the same problem @wiz21b was having. Thank you for your time.

@coastalmodeler thank you for posting your solution here, it was an issue with the EVA class not removing duplicates - I have included a fix in the latest release

Hi, just to add some information for anyone who might have a similar problem. I was getting the same error, even though I had hourly data. Upon a closer inspection, I noticed that the time step between the data (dt) was not exactly 1 hour, but the difference dt.max()-dt.min()=np.float64(1.1641532182693481e-10). By creating a new perfectly spaced index the problem was solved.

@MBendoni did you have this issue with the latest version? This is not supposed to happen, your data can be spaced at any intervals - pyextremes should be able to handle that. Can you please provide instructions to reproduce this error?

@georgebv the pyextreme version is the 2.3.2. To reproduce the error I think I should send you the netcdf file where the data are stored. In any case, here you have a snippet of code which gives to the error:
import netCDF4 as nc
import pandas as pd
import numpy as np
import pyextremes as pxtr

file_1 = 'myfile.nc'
ds_1 = nc.Dataset(file_1)
ref_date = np.datetime64('1990-01-01 00:00:00')
ny = 30
nd = ny*365*24
t_1 = ref_date + ds_1.variables['time'][0:nd].data.astype('timedelta64[D]')
var_name = 'wl'
v_1 = ds_1.variables[var_name][0:nd].data.flatten()
v_1[v_1<0]=0
s_1 = pd.Series(data=v_1, index=t_1)
ex_1 = pxtr.get_extremes(s_1, "POT", threshold=0.5, r="12h")

@MBendoni can you please write an example that doesn't rely on external data? You can generate random data inside the snippet.
https://numpy.org/doc/stable/reference/random/index.html


lang: revert to repr(packed) by default

Why the revert? I know packed should be an option for older accounts, but repr(C) seems like a reasonable default since repr(packed) can lead to unaligned references

@archseer 1. we merged the repr(C) change too quickly and it broke some things 2. repr(C) is bad UX and space-inefficient.
We can revert repr(packed) and we either
a) switch to repr(C) later but do it in a more thoughtful way that doesnt break things elsewhere
b) switch back to repr(packed) permanently and use raw pointers. add some helper macros/functions to abstract the raw pointers. better ux and more soace efficient so this is the option I like most

repr(C) is bad UX and space-inefficient.

It's space-inefficient because of alignment. You can get the same efficiency by properly arranging the struct (and adding padding as necessary).
Unaligned references will become a hard error in rust, meaning that if you lay your struct out incorrectly your code might not compile anymore in a future version: https://github.com/rust-lang/rust/issues/82523
#[zero_copy] requires careful thought and the user should guarantee to uphold https://docs.rs/bytemuck/latest/bytemuck/trait.Pod.html#safety

yes thats my point. you have to align it and add padding. so the user has to add unused space for alignment and unused padding if they have to use repr(c).
yes it will become a hard error. thats why Im saying we can stop using references and use raw pointers instead. those will still be fine to use

use raw pointers instead

with std::ptr::read_unaligned etc? In that case the type stops being zero_copy and is closer to borsh.
We're using zero_copy because that's what we want: a slightly less space efficient format with no overhead. It shouldn't be used lightly but bytemuck has built-in asserts that verify alignment nowadays.

just a raw pointer cast (what bytemuch does under the hood only that we dont cast it to a reference). no read_unaligned. as you said, that would be pointless

just a raw pointer cast (what bytemuch does under the hood only that we dont cast it to a reference)

That's potentially even worse, unaligned pointers are UB. (After all, what are references internally but "safe"/ compiler-validated pointers?)

Dereferencing (using the * operator on) dangling or unaligned pointers (see below)

https://doc.rust-lang.org/nomicon/what-unsafe-does.html?highlight=Unaligned#what-unsafe-rust-can-do
Alignment is an extra invariant the user has to be aware of but I'd prefer that to a bunch of custom unsafe code to work around it.

repr(packed)/repr(c, packed) aligns a struct to a single byte. we will not have unaligned pointers.
the only unsafe code we will have (and which the user wont see) is dereferncing the pointers which is safe.
all in all packed is much better ux with some unsafe complexities that are actually safe and can be abstracted.

though I would say that the rust docs arent clear on this.they say that a type using packed is aligned to a single byte but in the next paragraph they say that the type is unaligned.
Given that mainnet programs and our tests all work fine, Im inclined to think that what they call an unaligned pointer is a pointer that is pointing to an address % 1 byte != 0

upon further reading of the tracking issues, it does seem like all programs using packed are potentially causing UB and we've just been lucky
we should keep repr(C) then

@paul-schaaf should we add some docs about the correct patterns of using zero_copy. I've been running into some gotchas with it, and I doubt I'm the only one. Some examples of how to ensure the right alignment, etc would help a lot

@paul-schaaf should we add some docs about the correct patterns of using zero_copy. I've been running into some gotchas with it, and I doubt I'm the only one. Some examples of how to ensure the right alignment, etc would help a lot

Some examples would be awesome.

repr(c) by default is being kept. Closing.


Extensions recommendation based on installed CRD

This is draft implementation.
Todo:

[ ] Detect cluster change to update recommendation list
[ ] Add preference settings to disable recommendations

Current demo:

Fix: #772

@lstocchi @itowlson  Can you look on this?
I can prototype more with new vscode webview api.

cc @squillace

thanks for tagging me, @evidolob, this looks great. I like the experience, but we'll have to design how we detect and enable CRDs properly and fairly for all. I seem to recall we had a conversation like this. @itowlson do you recall?

@evidolob Apologies for the slow response on this one.  I think this is super interesting!  This feature really helps discoverability, and helps authors of CRD-based tools provide a truly end to end experience for their users.
I am not quite sure, however, what the user experience should be.  What you have will certainly do the job, but it doesn't yet feel consistent with the normal VS Code recommendation UX - the "there are recommendations for..." notification with a button that opens the Extensions tab on a marketplace search (which then brings in all the associated UI such as extension previews).  But I don't know if consistency with the VS Code recommendation UX is possible or practical, given that 1. we might want to present a number of specific extensions rather than a category search and 2. we don't want to have to reimplement a bunch of "don't show me again" style logic etc. that the built-in recommendation system provides for itself.
We also need to consider how to evolve the recommendations database.  I don't want to become responsible for evaluating, say, multiple Tekton extensions to decide which is "recommended."  Is our bar going to be "anyone can put in a PR recommendingy any extension and as long as it's not obviously evil then we'll give it our blessing"?  This feels like it risks our reputation (consider a malicious actor who creates a plausible extension, gets us to recommend it, and then modifies it to steal data).  A plain "search for extensions for CRD X" based on a keyword is more neutral than "we recommend" and would require no curation - but might provide correspondingly less value.
These are deep issues and I won't pretend that they're just nitpicks.  But this is a great first step in investigating those issues and I would be really eager to work out how we can address them and deliver this to users and CRD authors, because there is so much goodness here just waiting to be unlocked.

@itowlson @squillace
Sorry I was disrupted by other work.
I change a way how to recommended extensions shows. Now we have a CRD mapping to extension Tags and with recommendation I just perform search on marketplace with tahat tags:

@evidolob @itowlson What do we need to do to move this PR forward? @squillace

Hi @gorkem, I don't know, which is a fancy way of saying I think there aren't any blockers.
Two last questions:

What if the VS Code K8s extension were submitted to a publicly governed foundation? Would you as a user be comfy with the way we "suggest" things in that scenario?
What about non-marketplace extensions? Can someone add a GH repo-based suggestion?

Apologies for this late questioning, as we're all swamped but the ux is just lovely....

What if the VS Code K8s extension were submitted to a publicly governed foundation? Would you as a user be comfy with the way we "suggest" things in that scenario?

I think open governance would make this even better. Ultimately this would be a PR to the recommended-extensions.json and would go through the same rules as discussions as any other PRs. We are probably trusting the project maintainers with even more critical features on this process so I am not worried.

What about non-marketplace extensions? Can someone add a GH repo-based suggestion?

Depends on the installation experience. If there is a way to provide a similar installation experience for GH repo-based extensions or alternate marketplaces I would be open to supporting those methods and adding them. However I do not think this should become a way to show information that is not backed up with an experience.

Good responses! @itowlson whatchya think to that response? It's almost as if he's thought of everything.....

I agree with @gorkem's view on how the list would be maintained and I agree that this would align really well with an open governance structure.  And I don't think there's any need to support non-marketplace extensions at this point - let's build it and see who comes.
We would want to document the process for getting your extension onto the list, and we would want to have some sense of how we ensure the list remains decent quality (like what if a 'recommended' extension is turned to evil, or simply falls out of date and no longer works well with current versions).
@gorkem @evidolob this PR is still marked as draft - are we now at the stage where we feel it's ready for review?

To be clear, I don't think the documentation and process stuff is a blocker - to some extent we can work that out as we go.  I'd love to get this in if we feel is it ready to review.

I concur. I haven't DONE anything yet about concurring, but I continue to concur.

whenever you're ready, @gorkem, let us know.

Thanks @evidolob :pray:  @itowlson WDYT about it? Ready to be merged? We can then refine the documentation and discuss about "how to get an extension onto the list" in a new PR


How to add iScroll to fSelect

Hi
I'd like to add a smooth scroll script to fSelect e.g. http://iscrolljs.com/
How to do it?

Apologies, but we're not integrating with 3rd party libs at this time.


”Livezile”, ”Izvorălu de Jos”, ”Izvoru Aneștilor”, ”Petriș”, ”Ștefan Odobleja” are listed in the drop-down menu

Description:
When user writes ”Livezile”, ”Izvorălu de Jos”, ”Izvoru Aneștilor”, ”Petriș”, ”Ștefan Odobleja” the locations appear in the drop-down menu next to the comune and the county which is a part according to the law.
Precondition:
The website is up an running.

Step 1
Write in the search bar "Livezile"

Expected results
The location appears in the drop-down menu as "Sat Livezile, MEHEDINȚI (Livezile)".

Step 2
Press "x" button.

Expected results
The location was deleted from the search bar.

Step 3
Write in the search bar "Izvorălu de Jos"

Expected results
The location appears in the drop-down menu as "Sat Izvorălu de Jos, MEHEDINȚI (Livezile)".

Step 4
Press "x" button.

Expected results
The location was deleted from the search bar.

Step 5
Write in the search bar "Izvoru Aneștilor"

Expected results
The location appears in the drop-down menu as "Sat Izvoru Aneștilor, MEHEDINȚI (Livezile)".

Step 6
Press "x" button.

Expected results
The location was deleted from the search bar.

Step 7
Write in the search bar "Petriș"

Expected results
The location appears in the drop-down menu as "Sat Petriș, MEHEDINȚI (Livezile)".

Step 8
Press "x" button.

Expected results
The location was deleted from the search bar.

Step 9
Write in the search bar "Ștefan Odobleja"

Expected results
The location appears in the drop-down menu as "Sat Ștefan Odobleja, MEHEDINȚI (Livezile)".


Add more resource links for Git & GitHub

Hi,
Added more resources for learning Git & GitHub as suggested by @TehSebaXx - #54
Thanks.

Thanks @kukiron !


disconnecting google sheet from meeting-finder

I inherited a very old Dreamweaver website and our meetings are connected to meeting-finder via a google-sheet/doc.
I finally tracked down someone from the past who gave me access to the google-doc and I see the owner is Meeting-Finder with original creator Josh. R. who I've learned is part of Meeting Finder and not someone locally from the past.
I would like to disconnect my local google-sheet from meeting finder so that I can connect via the 12-Step-Meeting-Finder list in our new WordPress version of our website.
How do I do it?

hi @jlabenberg just let<EMAIL_ADDRESS>know and they will switch it for you

hi @joshreisner - than you so much for the quick response - I also need the google doc disconnected from my associated AREA website as well since they have access to is as well and they want it disconnected from them before they connect to my TSML version - will<EMAIL_ADDRESS>help me with that as well?

when you say "disconnected from" you mean the area is importing your google sheet data and displaying it on their site? if so, then the area webservant can do that on the area website by removing it at meetings > import & export


apps: Revert prometheus back to 2.19.1

What this PR does / why we need it:
The new prometheus version does not like running on NFS and it leads to issues with the wal.
fs_type=NFS_SUPER_MAGIC msg="This filesystem is not supported and may lead to data corruption and data loss. Please carefully read https://prometheus.io/docs/prometheus/latest/storage/ to learn more about supported filesystems."
...
msg="error tailing WAL" err="failed to find segment for index

Checklist:

[ ] Added relevant notes to WIP-CHANGELOG.md
[ ] Proper commit message prefix on all commits
[ ] Updated the public facing documentation
Is this changeset backwards compatible for existing clusters? Applying:

[ ] is completely transparent, will not impact the workload in any way.
[ ] requires running a migration script.
[ ] will create noticeable cluster degradation.
E.g. logs or metrics are not being collected or Kubernetes API server
will not be responding while upgrading.
[ ] requires draining and/or replacing nodes.
[ ] will change any APIs.
E.g. removes or changes any CK8S config options or Kubernetes APIs.
[ ] will break the cluster.
I.e. full cluster migration is required.

LGTM!
Any suggestion on how to solve this going forward? Should we introduce node local storage for prometheus in places where NFS was previously used? Or maybe just an emptydir?

Yeah, emptyDir seems like a reasonable solution in clusters where only NFS is available. You have to be a bit more aware of what nodes your running on to properly configure retention.
I was hoping that we could replace NFS with rook/ceph but that might be too wishful thinking ;)


IMDB builder imdb_search error: KeyError: 'data'

Version Number
1.20.0-nightly29 (Docker)
What branch are you on?
nightly
Describe the Bug
The following error happens when trying to use imdb_search:
|============================ Running Top Rated of the 1990s Collection =============================|
|                                                                                |
| Sync Mode: sync                                                                                |
|                                                                                |
| Builder: imdb_search: {'limit': 500, 'language': ['en'], 'release.after': '1990-01-01', 'release.before': '1999-12-31', 'sort_by': 'rating.desc', 'type': ['tv_series', 'tv_mini_series', 'tv_special'], 'votes.gte': 100} |
|                                                                                |
| Processing IMDb Search:                                                                            |
|     limit: 500                                                                                |
|     language: ['en']                                                                               |
|     release.after: 1990-01-01                                                                      |
|     release.before: 1999-12-31                                                                     |
|     sort_by: rating.desc                                                                           |
|     type: ['tv_series', 'tv_mini_series', 'tv_special']                                            |
|     votes.gte: 100                                                                                |
| Starting new HTTPS connection (1): api.graphql.imdb.com:443                                        |
| https://api.graphql.imdb.com:443 "POST / HTTP/1.1" 200 100                                         |
| Traceback (most recent call last):                                                                 |
|   File "//plex_meta_manager.py", line 775, in run_collection                                       |
|     builder.filter_and_save_items(builder.gather_ids(method, value))                               |
|                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                |
|   File "/modules/builder.py", line 2209, in gather_ids                                             |
|     ids = self.config.IMDb.get_imdb_ids(method, value, self.language)                              |
|           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                              |
|   File "/modules/imdb.py", line 572, in get_imdb_ids                                               |
|     return [(_i, "imdb") for _i in self._search(data)]                                             |
|                                    ^^^^^^^^^^^^^^^^^^                                              |
|   File "/modules/imdb.py", line 416, in _search                                                    |
|     total = response_json["data"]["advancedTitleSearch"]["total"]                                  |
|             ~~~~~~~~~~~~~^^^^^^^^                                                                  |
| KeyError: 'data'                                                                                |
|                                                                                |
| Unknown Error: 'data'                                                                              |
|                                                                                |
|====================================================================================================|

Relevant Collection/Overlay/Playlist Definition
## 0_debug.yml

templates:
  decade_top:
    collection_mode: hide
    collection_order: custom
    delete_not_scheduled: false
    limit: 250
    sort_title: "+3_<<num>>_<<collection_name>>"
    sync_mode: sync
    visible_home: false
    visible_library: false
    visible_shared: false
    imdb_search:
      language: <<language>>
      limit: 500
      release.after: <<release.after>>
      release.before: <<release.before>>
      sort_by: rating.desc
      type: <<type>>
      votes.gte: <<votes.gte>>

collections:
  "Top Rated of the 1990s":
    summary: "Top rated TV shows of the 1990s"
    file_poster: config/assets/3_decades_top/decades_top_1990s.png
    template: {
                    name: decade_top,
                    num: "T",
                    type: "tv_series,tv_mini_series,tv_special",
                    release.after: 1990-01-01,
                    release.before: 1999-12-31,
                    votes.gte: 100,
                    language: "en"
                    }

Logs
No response

I ran your exact YAML and had no issues:

I'm also having this issue, running the latest mater release. IMDB having issues again?

We're aware of this and have a potential solution that we have some early positive testing of - keep an eye out for a nightly update as soon as we can.

We're aware of this and have a potential solution that we have some early positive testing of - keep an eye out for a nightly update as soon as we can.

Any updates on this? I'm still getting the same 'data' error, unfortunately.

We're aware of this and have a potential solution that we have some early positive testing of - keep an eye out for a nightly update as soon as we can.

Any updates on this? I'm still getting the same 'data' error, unfortunately.

The latest nightly, released yesterday, should fix this once again.
This error will most likely reappear every time that imdb hash requires updating.

Just to add to this - we have deployed an interim solution in nightly that will allow us to deploy new hashes remotely without having to update nightly/PMM.
We will look towards a fully automated solution, but this will allow us to address the issue a lot quicker.

I'm still running into this problem on the nightly :(

I'm still running into this problem on the nightly :(

we'd need a log file

The latest release is newer than the nightly, so I'm testing that right now

The problem is fixed in 1.21.0 that was pushed a couple of hours ago, the nightly still does have the problem, I apologize for the report. Thanks everybody. Love this product.

You appear to have a bit of a misunderstanding and I don't think you're updating your nightly on a regular basis since this was fixed in nightly 2 weeks ago.
nightly is promoted to develop which in turn gets promoted to master - there is no scenario where master is ahead of nightly because of this pipeline.
nightly will always be on-par or ahead with master, but never behind it.
the final 1.21 build was pushed to nightly 6 hours prior to it going to master, but the fix you need was put into nightly 2 weeks ago - I suspect you're running an old version of nightly.
You should look on the wiki at how to keep PMM up to date.

Thanks for clarifying.
Before sending the report I pulled the latest nightly and when running that I got the error.
I even tried the nightly from linuxserver with no success.
I don't know why it still didn't work even though it was fixed 2 weeks ago.
But thanks anyways for the clarification.


move props related pages to separate section in sidebar

This PR adds new "Props" section to the "API" page sidebar which contains all props related pages earlier listed in section below.
The order of sections can be changed, it's only a proposal aiming to tidy up the sidebar.

@rachelnabors What do you think about those changes? If you don't like that idea I can close this PR.


Input Validation for Blocks

Blocks that accept text should validate or modify text so that spaces and capitalization don't break things. This can be by making changes to the inputs, or by abstracting them away in the backend. The latter is more friendly, but the former is easier as a short-term solution.

Take into account under not-allowed values inside predicate names, like -.

This has been completed for predicate and variable names in v1.6.17-alpha.


Exception using Python 3

Using Python 3.6.9 the following error occurred while running the script:
  File "dbcexplode.py", line 46
    except ValueError, e:
                     ^
SyntaxError: invalid syntax

However, it ran successfully using Python 2.7.17

I had the same issue.  Script works with Python 2.7 though, so I used conda to downgrade my Python for a 1-time run.

@brosl @strycker I just made an update to make this work for python3 as well as a shebang so you can now run it like
$ ./dbcexplode.py 20161003a.dbc

or
$ python3 ./dbcexplode.py 20161003a.dbc

Works now, thanks!


[Request]support to Custom Machinery mod

https://www.curseforge.com/minecraft/mc-mods/custom-machinery

I will try to implement

implementation done, now just wait for the 1.4.6 compilation | https://github.com/vindocel/Dark-Everywhere/commit/02a450e4b3a47f1ac7caf48449fa67075ff88e1d


Setting layer name shown under the layer control

Is it possible to set the layer name shown under the layer control? Currently, it seems the default layer name is empty. I tried using l.name = 'layer name', but it has no effect. It would be great if the da.leaflet.plot() has a name argument that can be set directly. Thanks.

Thanks for your suggestion @giswqs, I will try to work on it soon. Maybe we could use the DataArray name? Similarly, plotting a Dataset (#2) would use each array's name.

@davidbrochart Sounds good. So basically, users can set da.name, then your da.leaflet.plot() function automatically uses da.name as the layer name to show under the layer control. Am I correct?
da = xr.open_rasterio('/path/to/file')
da.name = 'layer name'
l = da.leaflet.plot(m, x_dim='x', y_dim='y')

Yes, that's the plan. But it could also be possible to set it as a keyword argument of the plot function in the case of a DataArray as you suggested.

Sounds great! Looking forward to it. I will be adding xarray_leaflet as a dependency for geemap. I plan to add a function (e.g., Map.add_raster(file_path, layer_name)) that wraps da.leaflet.plot() so that users can use one line of code to load local raster datasets to geemap.

Awesome!

I was able to customize the layer name. Here is the source code.

So it looks like setting the layer name works fine?

Yes, it works with geemap. But I would still suggest you add the layer_name argument to the plot() function so that non-GEE users can still use ipyleaflet to load rasters. Otherwise, when multiple local raster layers are added, only the last one added will be listed under the layer control.

I would still suggest you add the layer_name argument to the plot() function

I second that suggestion!
Regards

Sorry, I didn't have much time to work on xarray-leaflet lately. There will be a JupyterCon sprint on ipyleaflet, where we could also work on xarray-leaflet. It would be great if you could participate!

The layer name is now set to the DataArray name (see https://github.com/davidbrochart/xarray_leaflet/commit/66536e0f6be8de088d1594d2defdf27051c45ca5).


Update ckpt loading

PR#15 reads a set of ckpt file names from the index json file.
When OH downloads files from the hub instead of loading from a cache dir, get_repo_root() skips downloading the index json file. Thus the PR#15 fails to load file names.
This PR scans the path and returns a list of names that matches the pattern
Tested the models in cache dir:
Falcon-7b, Falcon-40b, Falcon-180b, Llama-70b
Tested the models downloaded from HF hub:
Falcon-7b, Falcon-40b, TinyLlama
Before submitting

[ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
[ ] Did you make sure to update the documentation with your changes?
[ ] Did you write any new necessary tests?

@schoi-habana , please also test downloaded ckpt for falcon-180b

@schoi-habana  - WIll this solve llama2 ; right now without your change we see below error:
pickle.UnpicklingError: Memo value not found at index 107
Loading 2 checkpoint shards:   0%|                                                                                | 0/2 [00:00<?, ?it/s]
Loading 2 checkpoint shards:   0%|                                                                                | 0/2 [00:00<?, ?it/s]Traceback (most recent call last):
File "/software/users/pkhanna/optimum-habana-fork/examples/text-generation/run_generation.py", line 626, in
main()
File "/software/users/pkhanna/optimum-habana-fork/examples/text-generation/run_generation.py", line 277, in main
model, tokenizer, generation_config = initialize_model(args, logger)
File "/software/users/pkhanna/optimum-habana-fork/examples/text-generation/utils.py", line 374, in initialize_model
else setup_distributed_model(args, model_dtype, model_kwargs, logger)
File "/software/users/pkhanna/optimum-habana-fork/examples/text-generation/utils.py", line 235, in setup_distributed_model
model = deepspeed.init_inference(model, **ds_inference_kwargs)
File "/usr/local/lib/python3.10/dist-packages/deepspeed/init.py", line 342, in init_inference
engine = InferenceEngine(model, config=ds_inference_config)
File "/usr/local/lib/python3.10/dist-packages/deepspeed/inference/engine.py", line 154, in init
self._apply_injection_policy(config, client_module)
File "/usr/local/lib/python3.10/dist-packages/deepspeed/inference/engine.py", line 417, in _apply_injection_policy
replace_transformer_layer(client_module, self.module, checkpoint, config, self.config)
File "/usr/local/lib/python3.10/dist-packages/deepspeed/module_inject/replace_module.py", line 333, in replace_transformer_layer
replaced_module = replace_module(model=model,
File "/usr/local/lib/python3.10/dist-packages/deepspeed/module_inject/replace_module.py", line 571, in replace_module
sd = torch.load(checkpoint, map_location='cpu')
File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 1026, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 1244, in _legacy_load
magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: Memo value not found at index 107
[2024-02-14 06:40:09,891] [INFO] [logging.py:96:log_dist] [Rank 0] DeepSpeed info: version=0.12.4+f4fa754c, git-hash=f4fa754c, git-branch=HEAD
[2024-02-14 06:40:09,892] [INFO] [logging.py:96:log_dist] [Rank 0] quantize_bits = 8 mlp_extra_grouping = False, quantize_groups = 1
Loading 2 checkpoint shards:   0%|                                                                                | 0/2 [00:00<?, ?it/s]Traceback (most recent call last):
File "/software/users/pkhanna/optimum-habana-fork/examples/text-generation/run_generation.py", line 626, in
main()
File "/software/users/pkhanna/optimum-habana-fork/examples/text-generation/run_generation.py", line 277, in main
model, tokenizer, generation_config = initialize_model(args, logger)
File "/software/users/pkhanna/optimum-habana-fork/examples/text-generation/utils.py", line 374, in initialize_model
else setup_distributed_model(args, model_dtype, model_kwargs, logger)
File "/software/users/pkhanna/optimum-habana-fork/examples/text-generation/utils.py", line 235, in setup_distributed_model
model = deepspeed.init_inference(model, **ds_inference_kwargs)
File "/usr/local/lib/python3.10/dist-packages/deepspeed/init.py", line 342, in init_inference
engine = InferenceEngine(model, config=ds_inference_config)
File "/usr/local/lib/python3.10/dist-packages/deepspeed/inference/engine.py", line 154, in init
self._apply_injection_policy(config, client_module)
File "/usr/local/lib/python3.10/dist-packages/deepspeed/inference/engine.py", line 417, in _apply_injection_policy
replace_transformer_layer(client_module, self.module, checkpoint, config, self.config)
File "/usr/local/lib/python3.10/dist-packages/deepspeed/module_inject/replace_module.py", line 333, in replace_transformer_layer
replaced_module = replace_module(model=model,
File "/usr/local/lib/python3.10/dist-packages/deepspeed/module_inject/replace_module.py", line 571, in replace_module
sd = torch.load(checkpoint, map_location='cpu')
File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 1026, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 1244, in _legacy_load
magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, 'x'.

Please tell some 70b command if it works:
python ../gaudi_spawn.py --use_deepspeed --world_size 8 run_generation.py --model_name_or_path /software/
data/llama_inference/Llama-2-70b-hf/ --use_hpu_graphs --use_kv_cache --max_input_tokens 128 --max_new_tokens 1024 --batch_size 16 --attn_softmax_bf16  --trim_logits --bf16 --reuse_cache --warmup 2 --n_iterations 2 --limit_hpu_graphs

@puneeshkhanna it may be related to DeepSpeed version, too. I'm aware of that error but couldn't repro in my test env. Which DS version is used in QA?

@puneeshkhanna DeepSpeed-fork git-hash=f4fa754c is not latest, there is a change you need to load safetensors.
Anyway this PR will load bins so no need for you to checkout later changes in DS-fork. Can you test this PR in your env?

https://github.com/huggingface/optimum-habana/pull/773


Image does not run

Using docker and in kubernetes the container keeps crashing
The only output is
Not setting JVM max heap size, already set
By setting the args to /run.sh in kubernetes, the container started as expected

What instructions are you following? https://github.com/pires/kubernetes-elasticsearch-cluster?

@pires I was having the same issue and yes I was following the instructions at https://github.com/pires/kubernetes-elasticsearch-cluster.

I've seen a few reports of this too. Might be related to my PR, but I tested running it via docker locally and didn't have any issues, but I guess I wasn't checking for run.sh being run. I wonder if the issue is that both of these scripts run exec.

I can confirm I'm seeing the same.  Running the statefulset example from the k-e-c repo. Going to investigate @chancez's suggestion about manually specifying command

I've been able to start the new images with the command
          command:
            - /entrypoint.sh
            - su-exec
            - elasticsearch
            - /elasticsearch/bin/elasticsearch

How was I able to run this then when updating the repo with the instructions for 6.0.0? 🤔

I spoke too soon
OVERWRITE_JVM_HEAPSIZE: true
NAMESPACE:
NODE_NAME:
MY_MEM_REQUEST:<PHONE_NUMBER>
MY_MEM_LIMIT:<PHONE_NUMBER>

and I'm still seeing
$ES_JAVA_OPTS set and JVM max heapsize already configured, but $OVERWRITE_JVM_HEAPSIZE=true, setting JVM max heap size
Setting JVM_HEAP_SIZE to 512M
[2017-12-07T16:13:53,164][INFO ][o.e.n.Node               ] [elasticsearch-master-69cc995bfb-cjtb8] initializing ...
[2017-12-07T16:13:53,460][INFO ][o.e.e.NodeEnvironment    ] [elasticsearch-master-69cc995bfb-cjtb8] using [1] data paths, mounts [[/data (/dev/xvda1)]], net usable_space [53.5gb], net total_space [62.8gb], types [ext4]
[2017-12-07T16:13:53,461][INFO ][o.e.e.NodeEnvironment    ] [elasticsearch-master-69cc995bfb-cjtb8] heap size [494.9mb], compressed ordinary object pointers [true]

I have the same problem running it on a fresh kubernetes-vagrant-coreos-cluster

@deedubs that's what it should look like. Are you saying you got it working without changing anything besides env vars?
FWIW, it seems CMD simply isn't running. If I just do docker run -it  quay.io/pires/docker-elasticsearch-kubernetes:6.0.0_jre8u151 /run.sh it does both the entrypoint and the /run.sh, so it doesn't seem like the scripts are interacting poorly, but rather the CMD isn't being executed for some reason.

@chancez Agreed. I missed where it was dividing the resource limit to set the heap.
https://github.com/pires/docker-elasticsearch-kubernetes/blob/master/entrypoint.sh#L46 is expecting an argument but we are currently not passing on https://github.com/pires/docker-elasticsearch-kubernetes/blob/master/Dockerfile#L15

@deedubs The argument should come from the base image which this is built on top of at https://github.com/pires/docker-elasticsearch/blob/master/Dockerfile#L72

Sorry for the delay but I've been really busy as of lately. I can reproduce and am working on a fix.

Unfortunately, I had to revert @chancez PR until I figure out what's going on. Releasing 6.0.1 as we speak.

@pires not sure if it'll help with your debugging but I have it running.  This is the full Statefull lSet https://gist.github.com/deedubs/c5bbbddf605794ca9a4609c23f94dc08

@deedubs you got it working by overriding the command. I don't want that and I really can't tell why the entrypoint doesn't call the cmd or why it returns an error. Even though they're in separate Dockerfiles, I believe it should just work. I don't have the free cycles to look more deeply into it now and so the quickest solution was to simply revert the commit that broke this.

Yeah exactly, not proposing it as a viable solution, just what I had to do
to move forward with 6.0(we had an external dep on 6.0).  I'll see if I can
investigate.
On Mon, Dec 11, 2017 at 9:58 AM Paulo Pires<EMAIL_ADDRESS>wrote:

@deedubs https://github.com/deedubs you got it working by overriding
the command. I don't want that and I really can't tell why the entrypoint
doesn't call the cmd or why it returns an error. Even though they're in
separate Dockerfiles, I believe it should just work. I don't have the free
cycles to look more deeply into it now and so the quickest solution was to
simply revert the commit that broke this.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pires/docker-elasticsearch-kubernetes/issues/54#issuecomment-350748295,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAnU3JDLY5jTYIUwDX-N4x8s8M8F6oxks5s_UMggaJpZM4Qzgbn
.

I wonder if this has to do with docker version.

We're on 1.12.6 FWIW

I test things with my kubernetes-vagrant-coreos-cluster repo and it runs CoreOS alpha which runs Docker 17.11.0, as I write this.


Document flags used to compile with system dependencies rather than internal ones

Distributions don't want the software they ship to use internal libraries when they can be used system wide. It seems dotnet can be built using several system dependencies, but they seem to be undocumented at the moment. It would make sense to document those flags at https://github.com/dotnet/source-build/blob/release/3.1/Documentation/packaging-installation.md.
Think of flags like -p:UseSystemLibunwind=true.

Seems something went wrong and I posted this twice. My mistake, closing.


Match statements for dialogue

Is your feature request related to a problem? Please describe.
In writing my dialogue, I frequently have dialogue that's based off of how many times you've talked to a character.
Currently the only way to write dialogue to work like this is by having long if/elif/else statements.
Describe the solution you'd like
Syntactic sugar over the if/else statement that lets you state the one variable you'd like to compare against at the start, followed by options for each of them, as an example.
match Chars.john_talk_count:
    0:
        John: Hey, nice to meet you!
    1:
        John: Hey, {{Chars.player_name}}, right? Good to see you again!
    2:
        John: Hey! I've already talked to you twice. Isn't that enough for now?
    _:
        John: {{Chars.player_name}}... please, we've talked {{Chars.john_talk_count}} times already... I need a break.
Chars.john_talk_count += 1

Which is way nicer and faster to write than:
if Chars.john_talk_count == 0:
    John: Hey, nice to meet you!
elif Chars.john_talk_count == 1:
    John: Hey, {{Chars.player_name}}, right? Good to see you again!
elif Chars.john_talk_count == 2:
    John: Hey! I've already talked to you twice. Isn't that enough for now?
else:
    John: {{Chars.player_name}}... please, we've talked {{Chars.john_talk_count}} times already... I need a break.
Chars.john_talk_count += 1

Describe alternatives you've considered
Just having a bunch of if statements that check the same property verbosely
Additional context
This would be a great addition for writing dialogue faster. It also means that if you want to change which variable is being checked, you only have to change the value in the match statement,
(please forgive me if this does exist, I'm new to using the plugin but I couldn't find this feature anywhere in the documentation)

I've been thinking of adding match statements but it's not a small task - it's definitely on the todo list but I'm not entirely sure how long before I'll get to it.

Noted, maybe I'll take a look at the parser and see if I can find a way to add them, save you a bit of work.

Hello,
We have an exciting opportunity for you! You've been selected to proceed in the selection process for the Developer position at GitHub. Congratulations on your achievement!
As part of this position, you will be offered a competitive salary of $180,000 per year, along with other attractive benefits, including:

Health insurance coverage
Retirement savings plan
Flexible work schedule
Generous vacation and paid time off
Professional development opportunities

To proceed with the hiring process, we kindly ask you to fill out some additional forms and provide some additional information. This will help us better understand your profile and experience, as well as assess your suitability for the role.
Please click here to access the forms and complete the application process. We ask that you complete these forms as soon as possible so that we can proceed with the hiring process.
Important: You have 24 hours to complete the application process.
If you have any questions or need further information, please don't hesitate to contact us.
Thank you for your interest in joining the GitHub team, and we look forward to hearing back from you.
Best regards,
GitHub Recruitment Team
jbog512, @sbhamad, @I-Want-ToBelieve, @ptcampbell, @lildev, @muhshahabipour, @milanig, @tomoyuen, @0x5457, @liajoy, @2-0-4, @dsimm, @orakle, @KrisjanisP, @setegonz, @martynski, @yadavrg, @TheReal1604, @raduchiriac, @gerasimsergey

Celestia Airdrop - Claim Your Free TIA Tokens Now!

Don't Miss Out on Your Share of 60 Million $TIA Tokens!
Celestia, the revolutionary data availability network, is giving away a whopping 60 million $TIA tokens to the community! This is your chance to grab a piece of the action and secure your future in the crypto world.
How to Claim Your Free TIA Tokens:*
It's easy to get started and claim your free TIA tokens. Follow these simple steps:

Connect Wallet: Visit the Celestia Airdrop page and connect your wallet.
Claim: Click the Claim button to receive your tokens.
Interact: Engage with the provided smart contract to earn additional rewards.

Why Wait? Start Earning Today!
With Celestia, the sky's the limit. Don't let this opportunity slip through your fingers. Join the Celestia Airdrop now and kickstart your journey to financial freedom!oberbichler, @vimota, @mateusortiz, @kirankunigiri, @Scosh, @brianruggieri, @RajuBhosale, @EvoluoGames, @LunarFlash, @o-vuong, @fjell-dev, @fontofonto, @yankees714, @kollayacine, @bianpratama, @KuangPF, @jonahisadev, @bankole7782, @farskid, @lnked


Fix to line style max offset calculation to properly account for modifiers.

Partial fix for issue also need change to batch connector.
itwinjs-core: https://github.com/iTwin/itwinjs-core/pull/6039
Previously padding only considered with line style definition width without considering modifiers which can both increase or decrease the overall width. There was a special case for continous with width, but this did not account for point symbol or stroke pattern width.
This also fixes keeping the line style name map up to date when new styles are inserted. This previously was done in the C++ method and has now been move to the _OnInsert method of the line style definition element.

Is this fix needed in the imodel02 repository? Presumably that is how the iModel in the issue was created.

Is this fix needed in the imodel02 repository? Presumably that is how the iModel in the issue was created.

Yes, I'm doing that PR now. It doesn't need the _OnInsert changes, just the max offset calc.


Scrobbling of tracks not possible

First off: thank you for this wonderful package: very happy to have found it. Am I correct in thinking the TrackScrobble class doesn't have the params necessary to do a scrobble? I'd be happy to help out.

Thank you very much! Thank you for the nice wishes! 😁
The problem with the TrackScrobble class is my inattention, the problem was that I accidentally missed implementing parameters and writing documentation for this class.
Therefore, in the new updated version and all subsequent versions (i.e. >=1.2.2) there is already a patch that solves this problem, BUT I'm not quite sure if it works, since I tried to figure out this particular method for an hour and a half, but I failed
In general, there are plans to release version 2.0.0, where the API and the amount of code is much smaller and lighter, but it's far from being done because the work is not yet complete

Thank you for your sincere wishes! It encourages me to keep working on the library. 😁
Most likely, the problem lies in my clumsy code that I added to version 1.2.2
If you manage to understand how the lastfmxpy library works, you will most likely be able to fix this misunderstanding and use this library fully, but today I see many bugs that could be fixed in version 2.0.0, which I do not have the strength to take on yet, I am too inexperienced in writing clean and good code


telegra.ph

Trust trading site
https://urlscan.io/result/96b871fd-ff91-49ee-9270-3546b7fe03eb#summary
address:  0x5F58fE612DB5B3E97FAf4A2E8f38B4AE295288Aa

Domain: xn--myetherwaet-0t8ea.com/#view-wallet-info
Reason: It is not the original myetherwallet. notice the unicode LL. came via a phishing email:

The Bee Token Team<EMAIL_ADDRESS>Mar 8 at 5:04 AM
BEE Token has now been distributed and is available to trade on several exchanges such as EtherDelta!
You must follow these instructions in order to access your BEE. If you do not, you will not be able to access your BEE at all nor trade it.
Instructions:

Load your wallet on MyEtherWallet https://myetherwaḻḻet.com/#view-wallet-info
Scroll down until you see Token Balances in the bottom right corner of your screen.
Click “Show All Tokens” and scroll down until you see BEE.
Select “Click to Load BEE”. Your BEE is now loaded into your address!
Once this is done you can instantly begin trading BEE which is currently enabled on EtherDelta, HitBTC and Kraken at 2.3x the ICO price.
Respectfully,
The Bee Token Team


Fix bad access to ok in CompletionRunner::run

Not sure why the sanitizers in CI didn't catch it, but it was causing bad access when running node_test on my MacBook.

Not sure why the sanitizers in CI didn't catch it, but it was causing bad access when running node_test on my MacBook.

Oh, thanks, my bad.
I would like our sanitizers in CI to catch this, but we currently don't have UBSAN enabled. So I've just tried to enabled it locally using GCC, but it seems not to report this at least on my setup (Ubuntu 20.04 gcc 11.1.0). Moreover, using Clang UBSAN doesn't work for us due to int128 multiply issue, so we're a bit out of luck here.


Add optional helperText for formfields

Use Bootstrap form-text class (and text-muted for styling) …
Add aria-described by to all applicable inputs

@MrCrayon This is the redo of pull request #4488. Not touching the for attribute on this one, and added bootstrap classes & aria

@emptynick What do you guys think of merging this one? The requested change was fixed some time ago. I use this in production. It's a feature addition that doesn't change any existing functionality.

Hey, sorry for the holdup.
Can't we simply include that text directly under the formfield in the edit/add view so it doesn't need to be repeated for every formfield?

@emptynick Yes, and this PR does exactly that. But the ARIA tag for accessibility is an attribute on the input/select/etc itself, so that has to be per-formfield. The aria attribute I’m using is aria-describedby and it only references the name of the single helper text p in add-edit (it does not repeat the actual helper text).


Update go dependencies

Update golang.org/x/crypto for CVE-2023-48795 and update other packages. go-git is not updated because it needs time to figure out why some tests fail.

No, don't backport

@wxiaoguang yup, sorry I applied the label thinking this was just the crypto changes and removed it when I saw the full extent of this PR


upload omnibus-base-andromeda.toml

replaced with https://github.com/Synthetixio/synthetix-deployments/pull/66
branches over forks!


Markdown[] ScriptError: Authorization is required to perform that action.

I cant get this tool to work at all

Can you share a (view-only) copy of a doc that causes this error, so we can investigate?

sure, let me see if i still have it

https://docs.google.com/document/d/1a5cVAS6SfQ3foQXE-sm3aM5d_OtU-TEEmFcrwJb75N8/edit?usp=sharing

Thanks -- will check it out.

I am able to convert a copy of your doc to Markdown and HTML successfully. Are you still having this problem? If so, please reopen.


Combine Face and Web Packages

Right now the packages are doing the same thing for server setup. The restructure involves making it so that the server can start whichever html is requested by an interaction and then work with it. This avoids having repeated code while allowing for an increasing number of custom html pages.

Finally we have decided to keep them separate. We can close this issue.


TableClient.createTable doesn't throw if the table already exists

Package Name: @azure/data-tables
Package Version: 13.0.0
Operating system: macOS
[X] nodejs

version: 14.16.0

[ ] browser

name/version:

[X] typescript

version: 4.4.4

Is the bug related to documentation in

[ ] README.md
[ ] source code documentation
[ ] SDK API docs on https://docs.microsoft.com

Describe the bug
Normally, I'd expect TableClient.createTable to throw if the table already exists. Azure documentation even suggests that an error should be thrown, since there's a Conflict 409 error with TableAlreadyExists.
To Reproduce

Run the following code:

async function testCreateDuplicate() {
    try {
        const client = TableClient.fromConnectionString("<CONNECTION_STRING>", "existingTable");
        await client.createTable({ onResponse: (response) => console.log("Response: " + response) });
        await client.createTable({ onResponse: (response) => console.log("Response: " + response) });
        console.error("This should throw!");
    } catch (error) {
        console.log(error);
    }
}

Expected behavior
An error to be thrown by the second call to createTable.
Screenshots
N/A
Additional context
createTable also doesn't seem to throw if the table is currently being deleted, which is another Conflict 409.

Hi @craxal, thanks for reporting this. Not throwing on table creation when the table already exists is a conscious design decision. We think that most of the time, users wouldn't really care if the table already existed or not, but only that the table is ready to interact with. Having to try/catch every time would be rough for the common use case.
Users should be able to check for the TableAlreadyExist error code if they need to. For example:
 const tableName = `oneTable`;
  await serviceClient.createTable(tableName, {
    onResponse: (r) => {
      if (r.status === 409) {
        console.log(`Table ${tableName} already exists`);
      }
    }
  });

With that said, there are 3 things that we need to fix in the library related to this scenario:

[ ] Document that createTable doesn't throw an error when the Table exists and also have samples on how to check for the error if needed
[ ] We should throw on any other errors such as TableBeingDeleted
[ ] createTable doesn't execute the onResponse callback when handling the TableAlreadyExists error. This needs to be fixed to allow users checking for TableAlreadyExists

Sent out #18914 to address the 3 issues noted above

Fix went in with PR #18914 the latest @dev version in npm should include the fix.


Crashes on viewDidLoad due to nil outlets

I have seen reports of crashes on viewDidLoad methods of the preference panes, like this:
Thread 0 Crashed:
0   HSTracker                            0x000000010b816eaf HSTracker.TrackersPreferences.viewDidLoad() -> () (TrackersPreferences.swift:0)
1   HSTracker                            0x000000010b816ef8 @objc HSTracker.TrackersPreferences.viewDidLoad() -> () (<compiler-generated>:0)
2   AppKit                               0x00007ff814505a92 -[NSViewController _sendViewDidLoad] + 86
3   AppKit                               0x00007ff8144eded2 -[NSViewController _loadViewIfRequired] + 384
4   AppKit                               0x00007ff8144edd16 -[NSViewController view] + 22
5   HSTracker                            0x000000010b99e2a3 Preferences.PreferencesTabViewController.(setWindowFrame in _1B0415CEB51F45D735E92B4C0504C08B)(for: __C.NSViewController, animated: Swift.Bool) -> () (PreferencesTabViewController.swift:193)
6   HSTracker                            0x000000010b99e7e3 closure #1 (__C.NSAnimationContext) -> () in Preferences.PreferencesTabViewController.transition(from: __C.NSViewController, to: __C.NSViewController, options: __C.NSViewControllerTransitionOptions, completionHandler: () -> ()?) -> () (PreferencesTabViewController.swift:169)
7   HSTracker                            0x000000010b99f089 partial apply forwarder for closure #1 (__C.NSAnimationContext) -> () in Preferences.PreferencesTabViewController.transition(from: __C.NSViewController, to: __C.NSViewController, options: __C.NSViewControllerTransitionOptions, completionHandler: () -> ()?) -> () (<compiler-generated>:0)
8   HSTracker                            0x000000010b99f0c1 partial apply forwarder for reabstraction thunk helper from @callee_guaranteed (@guaranteed __C.NSAnimationContext) -> () to @escaping @callee_guaranteed (@guaranteed __C.NSAnimationContext) -> () (<compiler-generated>:0)
9   HSTracker                            0x000000010b99e8e8 reabstraction thunk helper from @escaping @callee_guaranteed (@guaranteed __C.NSAnimationContext) -> () to @callee_unowned @convention(block) (@unowned __C.NSAnimationContext) -> () (<compiler-generated>:0)
10  AppKit                               0x00007ff8144b01ab +[NSAnimationContext runAnimationGroup:] + 54
11  HSTracker                            0x000000010b99dfed Preferences.PreferencesTabViewController.(animateTabTransition in _1B0415CEB51F45D735E92B4C0504C08B)(index: Swift.Int, animated: Swift.Bool) -> () (PreferencesTabViewController.swift:165)
12  HSTracker                            0x000000010b99da1f Preferences.PreferencesTabViewController.activateTab(index: Swift.Int, animated: Swift.Bool) -> () (PreferencesTabViewController.swift:85)
13  HSTracker                            0x000000010b9a45c3 Preferences.ToolbarItemStyleViewController.(toolbarItemSelected in _7A80F77958D94D970CD625FA09110540)(__C.NSToolbarItem) -> () (ToolbarItemStyleViewController.swift:0)
14  HSTracker                            0x000000010b9a461f @objc Preferences.ToolbarItemStyleViewController.(toolbarItemSelected in _7A80F77958D94D970CD625FA09110540)(__C.NSToolbarItem) -> () (<compiler-generated>:0)
15  AppKit                               0x00007ff8147f302f -[NSToolbarButton sendAction:to:] + 106
16  AppKit                               0x00007ff81479092e -[NSToolbarItemViewer mouseDown:] + 5172
17  AppKit                               0x00007ff814692bc1 -[NSWindow(NSEventRouting) _handleMouseDownEvent:isDelayedEvent:] + 4858
18  AppKit                               0x00007ff814606d7e -[NSWindow(NSEventRouting) _reallySendEvent:isDelayedEvent:] + 2581
19  AppKit                               0x00007ff81460614e -[NSWindow(NSEventRouting) sendEvent:] + 351
20  HSTracker                            0x000000010b9a5429 @objc Preferences.UserInteractionPausableWindow.sendEvent(__C.NSEvent) -> () (Utilities.swift:92)
21  AppKit                               0x00007ff814604524 -[NSApplication(NSEvent) sendEvent:] + 351
22  AppKit                               0x00007ff8148bd18b -[NSApplication _handleEvent:] + 64
23  AppKit                               0x00007ff814485d3e -[NSApplication run] + 622
24  AppKit                               0x00007ff814459c97 NSApplicationMain + 816
25  HSTracker                            0x000000010b762fe9 main (AppDelegate.swift:21)
26  ???                                  0x000000011616152e 0x0 + 0

Any idea what could be causing it? From decoding the addresses, the crash location is in the first reference to an IBOutlet. This same pattern has showed up on different preference panes as well;

@fmoraes74 This looks like the crash is happening in your view controllers, so it's out of scope of this package.
Found your dev forums thread, which is a much better place: https://developer.apple.com/forums/thread/711847

The crash is because the outlets are not connected, not because of the actual code. Is it possible there is no check or assert in the Preferences code to make sure the NIB was actually loaded? I don't see how this can be the HSTracker code when it works all the time. If connections were missing, it would always fail. So the only explanation is that sometimes the NIB doesn't load correctly and the implicit references are actually nil and fail at runtime.


Update Extension Listing

Need to update the Chrome and Firefox listings with more relevant screenshots, and maybe a video demonstrating some of the features. Might be able to put a pull request into _assets for now.
Chrome listing requirements

Firefox listing requirements

Some decent marketplace listings to reference...
https://chrome.google.com/webstore/detail/awesome-screenshot-screen/nlipoenfbbikpbjkfpfillcgkoblgpmj
https://chrome.google.com/webstore/detail/evernote-web-clipper/pioclpoplcdbaefihamjohnefbikjilc?hl=en
https://chrome.google.com/webstore/detail/ghostery-–-privacy-ad-blo/mlomiejdfkolichcflejclcbmpeaniij?hl=en
https://chrome.google.com/webstore/detail/honey/bmnlcjabgnpnenekpadlanbbkooimhnj?hl=en
https://chrome.google.com/webstore/detail/writer/hngnkmianenpifegfoggnkamjnffiobn?hl=en
https://chrome.google.com/webstore/detail/google-keep-chrome-extens/lpcaedmchfhocbbapmcbpinfpgnhiddi?hl=en
https://chrome.google.com/webstore/detail/adblock-plus-free-ad-bloc/cfhdojbkjhnklbpkdaibdccddilifddb

Some IBM specific type of marketplace listings.
apple store
ibm-hmc-mobile
ibm-client-vantage
ibm-enterprise-video-streaming


make translatable field shippingTime importable and exportable

this field shippingTime is by shopware default translatable.

but currently it is not possible to export or import shippingTimes translations.
With this pull request, the "shippng time" can also be used from the list of translatable fields.

Export and Reimport from Shipping Time translations is possible.

Hello,
thank you for creating this pull request.
I have opened an issue on our Issue Tracker for you. See the issue link: https://issues.shopware.com/issues/PT-12447
Please use this issue to track the state of your pull request.

thanks for your contribution @wrongspot 👍🎉💙


Attempt to return control status immediately is flawed

In an attempt to provide mitigation of #45 , there's a DNP3 Outstation option (WaitForCommandResponses) that's meant to allow immediate return of control status without waiting for the actual result:
auto future_results = PublishCommand(arCommand, aIndex);

	auto pConf = static_cast<DNP3PortConf*>(this->pConf.get());
	if (!pConf->pPointConf->WaitForCommandResponses)
	{
		return CommandStatus::SUCCESS;
	}

This doesn't actually work, because the d'tors of the futures block. This actually makes the situation worse than if the option isn't used, because the thread is actually blocked, whereas we'd normally call poll_one() (still a problem - see #45 )

Fixed - Futures are gone 36d234d1cb5c7f1e571b7f1069039e210db9f723