AttributeError from CommonGenericTaggedItemBase.tags_for() with current pre-release Django

I'm testing an existing, working, project with the very latest commit of Django and I'm getting an error from my custom TaggableManager that I don't get with Django 4.1. An AttributeError comes from CommonGenericTaggedItemBase.tags_for() when using extra_filters.
I have a Bookmark model that has BookmarkTags, though a TaggedBookmark:
from django.db import models
from taggit.models import GenericTaggedItemBase, TagBase

class BookmarkTag(TagBase):
    # custom methods here

class TaggedBookmark(GenericTaggedItemBase):
    tag = models.ForeignKey(
        BookmarkTag,
        on_delete=models.CASCADE,
        related_name="%(app_label)s_%(class)s_items",
    )

class Bookmark(models.Model):
    is_private = models.BooleanField(default=False)
    # other fields

    tags = TaggableManager(manager=_BookmarkTaggableManager, through=TaggedBookmark)

The _BookmarkTaggableManager used is like:
from django.db import models
from taggit.managers import _TaggableManager

class _BookmarkTaggableManager(_TaggableManager):

    def most_common(self):
        "Get the most common tags used on all non-private Bookmarks"

        extra_filters {"bookmark__is_private": False,}

        return (
            self.get_queryset(extra_filters)
            .annotate(num_times=models.Count(self.through.tag_relname()))
            .order_by("-num_times")
        )

Calling Bookmark.tags.most_common() has worked fine in Django 4.0 and earlier. But with the very latest Django code I get:
Traceback (most recent call last):
  File "/Users/phil/.local/share/virtualenvs/devproject-25a5mkZN/lib/python3.10/site-packages/taggit/managers.py", line 77, in get_queryset
    return self.instance._prefetched_objects_cache[self.prefetch_cache_name]
AttributeError: 'NoneType' object has no attribute '_prefetched_objects_cache'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/Users/phil/Projects/personal/django-ditto/ditto/pinboard/managers.py", line 36, in most_common
    self.get_queryset(extra_filters)
  File "/Users/phil/.local/share/virtualenvs/devproject-25a5mkZN/lib/python3.10/site-packages/taggit/managers.py", line 80, in get_queryset
    return self.through.tags_for(self.model, self.instance, **kwargs).order_by(
  File "/Users/phil/.local/share/virtualenvs/devproject-25a5mkZN/lib/python3.10/site-packages/taggit/models.py", line 161, in tags_for
    return cls.tag_model().objects.filter(**kwargs).distinct()
  File "/Users/phil/.local/share/virtualenvs/devproject-25a5mkZN/src/django/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/Users/phil/.local/share/virtualenvs/devproject-25a5mkZN/src/django/django/db/models/query.py", line 1395, in filter
    return self._filter_or_exclude(False, args, kwargs)
  File "/Users/phil/.local/share/virtualenvs/devproject-25a5mkZN/src/django/django/db/models/query.py", line 1413, in _filter_or_exclude
    clone._filter_or_exclude_inplace(negate, args, kwargs)
  File "/Users/phil/.local/share/virtualenvs/devproject-25a5mkZN/src/django/django/db/models/query.py", line 1420, in _filter_or_exclude_inplace
    self._query.add_q(Q(*args, **kwargs))
  File "/Users/phil/.local/share/virtualenvs/devproject-25a5mkZN/src/django/django/db/models/sql/query.py", line 1532, in add_q
    clause, _ = self._add_q(q_object, self.used_aliases)
  File "/Users/phil/.local/share/virtualenvs/devproject-25a5mkZN/src/django/django/db/models/sql/query.py", line 1562, in _add_q
    child_clause, needed_inner = self.build_filter(
  File "/Users/phil/.local/share/virtualenvs/devproject-25a5mkZN/src/django/django/db/models/sql/query.py", line 1470, in build_filter
    col = self._get_col(targets[0], join_info.final_field, alias)
  File "/Users/phil/.local/share/virtualenvs/devproject-25a5mkZN/src/django/django/db/models/sql/query.py", line 387, in _get_col
    return target.get_col(alias, field)
AttributeError: 'ManyToManyRel' object has no attribute 'get_col'

I don't think I'm doing anything particularly weird?
I can't see anything obvious in the backwards incompatible changes for Django 4.1.
The issue seems to be in CommonGenericTaggedItemBase.tags_for(). With Django 4.0 I can do this successfully:
extra_filters = {"bookmark__is_private": False}
TaggedBookmark.tags_for(Bookmark, **extra_filters)

But if I use (what will be) Django 4.1 I get the above error ending in AttributeError: 'ManyToManyRel' object has no attribute 'get_col'.
If I leave the **extra_filters off, there is no error.

I'd forgotten that there's a CommonGenericTaggedItemBase.most_common() method already, and that mine overrides that (I wrote this code a long time ago).
The django-taggit tests work fine with the djmain environments, so it's obviously a difference in my code, or else an error the existing tests don't pick up.
If I run:
tox -e py39-djmain -- tests.tests.TaggableManagerOfficialTestCase.test_most_common_extra_filters

and print the kwargs in CommonGenericTaggedItemBase.tags_for() before the method's final line, it outputs:
{
    'tagged_items__content_type__app_label': 'tests',
    'tagged_items__content_type__model': 'officialfood',
    'officialfood__name__in': ['pear', 'apple']
}

If I run one of my failing tests, and output kwargs at the same point in django-taggit, then it looks like:
{
    'pinboard_taggedbookmark_items__content_type__app_label': 'pinboard',
    'pinboard_taggedbookmark_items__content_type__model': 'bookmark',
    'bookmark__is_private': False
}

I guess the only difference I see here is that mine is using a through model?

If I remove my overridden _BookmarkTaggableManager.most_common() method, so that I'm using _TaggableManager's own one, and do this:
Bookmark.tags.most_common(extra_filters={"bookmark__is_private": False})

I get the same error. If I don't pass extra_filters in, then there's no error.

OK, this is now too much for anyone to read. Going to make a new issue with a minimal demonstration of the issue.


Implement JMX input

Maybe via https://www.npmjs.com/package/jmx (but didn't look into options deeply), though I see this depends on some node-java package and that node-java README mentions Python...

Evaluated and worked so far. Java installation and integration with node-java was a bit of trouble on Mac-OS-X https://github.com/joeferner/node-java/issues/90#issuecomment-45613235.
A handy tool to discover JMX beans and property names is https://github.com/sematext/jmxc :)

This should be a separate npm module/plugin, to avoid dependencies to "java".

An alternative to get JMX stats into Logagent could be jmxtrans combined with Logagent command plugin: https://github.com/jmxtrans/jmxtrans/wiki/StdoutWriter

Make some experiments with the last version of node-jmx and haven't trouble in installation on MacOs.
Developing ingestion for Cassandra metrics I found that node-jmx seems to store every event emitted. I need to investigate a little more to figure out if is a my implementation problem or something else.

Hi, I' dig a little in node-jmx but during experiments I went against some problems, infact using it in batch mode there was trouble in memory usage or too much event listener was added. I've forked library and remove client and node-jmx layers and use only a mbeanServerConnection with a minor refactoring.
At the moment APIs are a little rougth but works, as soon as is possible they will come more fluent.
now code look like
'use strict'
const mBeanconnection = require('./lib/adapters/mbeanServerConnectionNew')
const jmxServiceUrlBuilder = require("./lib/adapters/helpers/jmxServiceUrlBuilder"),
checkError = require("./lib/helpers/error").checkError

setInterval(readDatAll.bind(this), 1000)

var serviceOrHost = 'localhost'
var port = 7199
var protocol
var urlPath

var keyValuecqlMetrics = [
["org.apache.cassandra.metrics:type=CQL,name=PreparedStatementsEvicted","Count"],
["org.apache.cassandra.metrics:type=CQL,name=PreparedStatementsCount","Value"],
["org.apache.cassandra.metrics:type=CQL,name=PreparedStatementsExecuted","Count"],
["org.apache.cassandra.metrics:type=CQL,name=PreparedStatementsRatio","Value"],
["org.apache.cassandra.metrics:type=CQL,name=RegularStatementsExecuted","Count"]]

this.cqlMetrics = new Map(keyValuecqlMetrics)

this.url = jmxServiceUrlBuilder(serviceOrHost, port, protocol, urlPath);

function readDatAll(){
    var self = this
    self.cqlMetrics.forEach(function(value, key){
        connectBean(self.url, key,value);
    })
}

function connectBean(url,bean,measure)
{
    var self = this
    var mBean = new mBeanconnection()
    mBean.myconnect(url,function(){
        var self = this;
        mBean.queryMBeans(null, bean, function(instance) {
          instance.getObjectName(function(err, objectName) {
            if (checkError(err, self)) return;
            mBean.getAttribute(objectName,measure, function(data){
                console.log(bean + ' value ' + data)
            });
            mBean.close
            mBean = null
          });
        });
    });
}

At the moment it works for me but any suggestion are welcome ;-) Asap I'll send a Cassandra input metrics plugin

I think we can close this.  https://github.com/sematext/sematext-agent-java has JMX support.


"Favorite docs" - Add method for setting alternate Figma Files to load in the Design Switcher

Umesh notes that the existing feature to show branches allows easier navigation between docs in the Design Switcher, without needing to rebuild the app (so we can provide this guidance).


Homebrew installation fails

I tried installing see with homebrew on a mac and got:
==> Tapping guilhermeprokisch/see
Cloning into '/opt/homebrew/Library/Taps/guilhermeprokisch/homebrew-see'...
remote: Repository not found.
fatal: repository 'https://github.com/guilhermeprokisch/homebrew-see/' not found
Error: Failure while executing; `git clone https://github.com/guilhermeprokisch/homebrew-see /opt/homebrew/Library/Taps/guilhermeprokisch/homebrew-see --origin=origin --template= --config core.fsmonitor=false` exited with 128.
❯ brew install guilhermeprokisch/see/see
==> Tapping guilhermeprokisch/see
Cloning into '/opt/homebrew/Library/Taps/guilhermeprokisch/homebrew-see'...
remote: Repository not found.
fatal: repository 'https://github.com/guilhermeprokisch/homebrew-see/' not found
Error: Failure while executing; `git clone https://github.com/guilhermeprokisch/homebrew-see /opt/homebrew/Library/Taps/guilhermeprokisch/homebrew-see --origin=origin --template= --config core.fsmonitor=false` exited with 128.

It seems like homebrew-see repo does not exist.

hey @pwntester
I've just reinstall now and it's working fine: brew install guilhermeprokisch/see/see
It's the first time that you installed or you upgrading from the old namedsmd version?

Thanks for the quick response. Its the first time I try to install it and I get the same error when retried. I get a 404 for https://github.com/guilhermeprokisch/homebrew-see/ Is it a public repo?

God.. it was private :sweat_smile:
Can you try now?

@pwntester  let me know if it works.

Im AFK until sunday but will report how it goes

working fine for me using homebrew

It works now, thanks!


Add link to The Changelog episode

I wanted to link up the changelog episode that we did a few weeks ago. There's not really a logical place to put it on the website right now, so I just added it on the homepage right above the footer. I'm open to other ideas.
I also tried using the embeddable player from changelog, but I couldn't really figure out how to make it fit and look good.
@JasonEtco would you be interested in fiddling with this to see what feels best to you?

This looks nice already for the homepage but what do you think about putting it on the "Community" page instead? I think it makes sense to keep the homepage as straightforward as possible, plus this sounds like more of a community-ish thing.
Also, side-note, we should add @jbjonesjr's talk somewhere (its on Facebook right now, hopefully it'll land on YouTube at some point)

👍  to both suggestions

Cool, I'll move the changelog link over to there and add the talk in a separate PR 👍

@bkeepers I went ahead and adding the Universe talk in this PR, since the design would look different with just one. Here's what it looks like:

@bkeepers I've added the redirect, should be good to merge with one last 👍

Thanks for finishing this up!


[BUG] - Card provides wrong type for "as" prop

Describe the bug
The current as prop of the Card component is as?: keyof JSX.IntrinsicElements; but it should be As<any> to allow components to be passed.
This is how most other Next UI components do it.
For example, this would allow the Card component to be used as a Next.js Link like this:
import Link from "next/link"

<Card as={Link} />

Your Example Website or App
No response
Steps to Reproduce the Bug or Issue

Try to pass a component to the as prop.
Get a type error.

Expected behavior
No error.
Screenshots or Videos
No response
Operating System Version
macOS
Browser
Chrome

Hey @hmbrg  sorry for the delay please upgrade to V2 and test this again.


fix(cmake): fix bundled GRPC_LIBRARIES order

When building with the bundled grpc on big-endian platforms, thelibabsl_random_internal_platform.a needs to be after the other libabsl_random_internal_* libraries to avoid undefined references when linking.
Signed-off-by: Jonathan Albrecht<EMAIL_ADDRESS>
What type of PR is this?
/kind bug
Any specific area of the project related to this PR?
/area build
Does this PR require a change in the driver versions?
no
What this PR does / why we need it:
Fixes the build on s390x when using -DUSE_BUNDLED_DEPS=On which has an error like:
[ 98%] Linking CXX executable unit-test-libsinsp
/usr/bin/ld: ../../grpc-prefix/src/grpc/third_party/abseil-cpp/absl/random/libabsl_random_internal_randen_slow.a(randen_slow.cc.o): in function `absl::lts_20211102::random_internal::RandenSlow::GetKeys()':
randen_slow.cc:(.text+0x2): undefined reference to `absl::lts_20211102::random_internal::kRandenRoundKeysBE'

Which issue(s) this PR fixes:
none

Special notes for your reviewer:
Does this PR introduce a user-facing change?:
no
NONE

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: jonathan-albrecht-ibm
Once this PR has been reviewed and has the lgtm label, please assign mstemm for approval by writing /assign @mstemm in a comment. For more information see the Kubernetes Code Review Process.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Welcome @jonathan-albrecht-ibm! It looks like this is your first PR to falcosecurity/libs 🎉

Hi @jonathan-albrecht-ibm thank you for that!

LGTM label has been added.  Git tree hash: 652d63543fb7937db44926ede4a7a49700096f5a

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97, jonathan-albrecht-ibm
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Andreagit97]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97, FedeDP, jonathan-albrecht-ibm
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Andreagit97,FedeDP]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Thanks @Andreagit97 and @FedeDP !

/close
Closing and reopening to trigger the CI (ie. @poiana :angel: )

@leogr: Closed this PR.

In response to this:

/close
Closing and reopening to trigger the CI (ie. @poiana :angel: )

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/reopen

@leogr: Reopened this PR.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97, FedeDP, jonathan-albrecht-ibm, leogr
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Andreagit97,FedeDP,leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

/test build-and-test-modern-bpf-x86 😇 (bundled_deps)

@leogr: No presubmit jobs available for falcosecurity/libs@master

In response to this:

/test build-and-test-modern-bpf-x86 😇 (bundled_deps)

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/override build-and-test-modern-bpf-x86 😇 (bundled_deps)

I apologize. We have a problem with the CI preventing us from merging this PR. I'm investigating the issue.

N.B. I've manually merged this PR because the CI was not working.

Thanks  for merging this @leogr!


Update syntax.md

Only a little mistake: "[...] conformes aux spécifications ." ==> "[...]conformes aux spécifications." (the space before the dot has been removed).

Merci !


fail to install codis

Hello guys,
when I tried to execute bootstrap.sh and get the the error as below:
root@james:/opt/codis/codis# ./bootstrap.sh
downloading dependcies, it may take a few minutes...
cd /usr/local/codis/src/github.com/codegangsta/martini-contrib; git pull --ff-only
error: The requested URL returned error: 403 while accessing https://github.com/codegangsta/martini-contrib/info/refs
fatal: HTTP request failed
In fact I get a 404 error when I access "https://github.com/codegangsta/martini-contrib/info/refs".
Is there a full package without download external package? or how to fix it?
Thanks
James

Hi James,
Do you set your $GOPATH correctly? If you set it right and then use execute go get -d github.com/wandoulabs/codis, the directory of codis should be:
$GOPATH/src/github.com/wandoulabs/codis
not
/opt/codis/codis
In other words, you can not download codis directly.

Hi YangZhe,
Would you like to take a look at my error, and I show you the GOPATH and GOROOT.
root@james:/usr/local/codis# ./bootstrap.sh
downloading dependcies, it may take a few minutes...
cd /usr/local/codis/src/github.com/bsm/redeo; git pull --ff-only
error: Failed connect to github.com:443; Connection timed out while accessing https://github.com/bsm/redeo/info/refs
fatal: HTTP request failed
package github.com/alicebob/miniredis
imports github.com/bsm/redeo: exit status 1
cd /usr/local/codis/src/github.com/bsm/redeo; git pull --ff-only
error: The requested URL returned error: 403 while accessing https://github.com/bsm/redeo/info/refs
fatal: HTTP request failed
package github.com/bsm/redeo: exit status 1
go build -o bin/codis-proxy ./cmd/proxy
cmd/proxy/main.go:14:2: cannot find package "github.com/wandoulabs/codis/pkg/proxy/router" in any of:
/usr/local/go/src/github.com/wandoulabs/codis/pkg/proxy/router (from $GOROOT)
/usr/local/codis/src/github.com/wandoulabs/codis/pkg/proxy/router (from $GOPATH)
cmd/proxy/main.go:15:2: cannot find package "github.com/wandoulabs/codis/pkg/utils" in any of:
/usr/local/go/src/github.com/wandoulabs/codis/pkg/utils (from $GOROOT)
/usr/local/codis/src/github.com/wandoulabs/codis/pkg/utils (from $GOPATH)
make: *** [build-proxy] Error 1
root@james:/usr/local/codis# echo $GOROOT
/usr/local/go
root@james:/usr/local/codis# echo $GOPATH
/usr/local/codis

How do you download the codes of codis? Pls use go get -d github.com/wandoulabs/codis. The code will be downloaded to $GOPATH/src/github.com/wandoulabs/codis
And $GOPATH is a global path that all the golang project will be in this, not only codis.


Can we use aggregation framework?

Hi,
Is aggregation framework available at doctrine2? I need to execute a query like this:
db.products.aggregate({$match:{created_at:{$exists:true}}}, {$project: {name: 1, month: {$month: '$created_at'}, day: {$dayOfMonth: '$created_at'}}},{$match: {month: 6, day: 3}})
I cannot find any example talking about that.

Yes you can, underlying doctrine/mongodb is exposing a builder since https://github.com/doctrine/mongodb/pull/213 and while you can not hydrate results into objects yet (#1263) you're free to use the feature and will just get back plain arrays with data.

hum, that works with ODM? I will try. Do you have any complete example?

I got it :), here is the example:
    $pipeline = array(
        array('$match' =>
            array(
                'created_at' => array('$exists' => true)
            )
        ),
        array('$project' =>
            array(
                'name' => true,
                'month' => array(
                    '$month' => '$created_at'
                ),
                'day' => array(
                    '$dayOfMonth' => '$created_at'
                ),
            )
        ),
        array('$match' =>
            array(
                'month' => $month,
                'day' => $day,
            )
        ),
    );

    return $this->getDocumentManager()->getDocumentCollection('Product')->aggregate($pipeline);

and it returns an Doctrine\MongoDB\ArrayIterator

Glad it works :)


Better error capabilities for Fortran

PGP token was timing out so I've rerun jobs

Codecov Report

Merging #284 (296e456) into develop (07f1fa1) will not change coverage.
The diff coverage is n/a.

Additional details and impacted files

@@           Coverage Diff            @@
##           develop     #284   +/-   ##
========================================
  Coverage    77.93%   77.93%
========================================
  Files           57       57
  Lines         3666     3666
========================================
  Hits          2857     2857
  Misses         809      809


Width subtyping fails when type is an array

Reproduction
// width subtyping doesn't work with with an Array<union>
// 	https://flow.org/en/docs/lang/width-subtyping/

// Sample context: An AutoComplete component
type Suggestion = { text: string, value: string } | string

// works
function getSuggestionValue(suggestion: Suggestion) {
  return typeof suggestion === 'string' ? suggestion : suggestion.text
}

//                                                     fails here.
function filterByText (inputText: string, suggestions: Array<Suggestion>) {
  return 'bar'
}

// Sample context: A Region chooser that uses the ^^^AutoComplete component
type Region = { text: string, value: string }
const regions: Array<Region> = [{text: 'Washington', value: 'WA'}]

// width subtyping works on single item union
getSuggestionValue(regions[0])

// *********
// width subtyping does NOT work on item inside array
filterByText('foo', regions)

Perhaps I'm missing something, but I see no reason why width subtyping works on an individual parameter, but fails in a typed Array.
Bug or misunderstanding?

Here, you need to use a $ReadOnlyArray! Indeed, you pass regions to filterByText, but filterByText could add some strings to the array, since it works with suggestions.

@AugustinLF is correct-- this is not a bug.
https://github.com/facebook/flow/issues/4483 for more info :)


feat: add homepage link to AI blogs

Description

Adds a link to the Oak blogs for AI
Adds spacing to "Back to Oak" icon

How to test

Go to https://oak-ai-lesson-assistant-8b2s70h2x.vercel-preview.thenational.academy
Click on the menu
You should see the link
Click on the link
You should in on the oak blog, with the AI category (currently there aren't any published posts)

Screenshots

:tada: This PR is included in version 1.5.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Support for Archival of Workflow Execution History

Ideally it should be transparent to the clients. So any archived workflow should still be accessible through all the visibility APIs, but instead of Cassandra they should hit the backend blob store.


Troy

Changes to language server required for supporting troy

Constraints in the build system mean this won't go green until we release an updated tremor-script - we are considering collapsing the language server back into tremor-runtime and folding it into the tremor client - this would resolve the build issues and allow us to simplify some of the language server code and remaining build dependencies. Avoiding the release locking that is a side-effect the build.rs also means we can update the language server and fully test the IDE integration with vim and vscode without making a release - a developer productivity win which supports folding the language server into the main project/build

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 92 (0.0%)  changed or added relevant lines in 8 files are covered.
3 unchanged lines in 3 files lost coverage.
Overall coverage remained the same at 0.0%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/language.rs
0
3
0.0%

src/main.rs
0
3
0.0%

src/language/prelude.rs
0
4
0.0%

src/language/query.rs
0
10
0.0%

src/language/script.rs
0
10
0.0%

src/language/deploy.rs
0
13
0.0%

src/backend.rs
0
23
0.0%

src/lsp_utils.rs
0
26
0.0%

Files with Coverage Reduction
New Missed Lines
%

src/backend.rs
1
0%

src/language.rs
1
0%

src/lsp_utils.rs
1
0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
0

Relevant Lines:
262

💛  - Coveralls


Add suffix method and attribute for Route definition

Hello, I recently faced the following issue for a personnal project, while refactoring my routes/web.php and routes/api.php files. I wanted to make those files as clear as possible (by grouping the routes by prefix, middleware, controller,...) and I ran into this disadvantage.

Route::post('invoice/payment/intent', [PaymentIntentController::class, 'invoice']));
Route::get('invoice/payment/confirm', [PaymentConfirmController::class, 'invoice']);
Route::post('quote/payment/intent', [PaymentIntentController::class, 'quote']);
Route::get('quote/payment/confirm', [PaymentConfirmController::class, 'quote']);
Route::post('account/payment/intent', [PaymentIntentController::class, 'account']);
Route::get('account/payment/confirm', [PaymentConfirmController::class, 'account']));

I was searching a "suffix" method who would suffix the attributes ('payment/intent' and 'payment/confirm') to the current uri (pretty much like the prefix method) but I couldn't find it.
I developped the feature on a branch and did this PR to see if there is any need regarding this feature.
It behaves, quite obviously, in the following way :

Route::group(['prefix' => 'foo', 'suffix' => 'bar'], function () {
Route::get('1');
Route::get('2');
});

That would result with two routes

The method would also be available like the prefix as a static and non static method for the Route class

Route::prefix('foo')->get('bar', fn() => "");
Route::suffix('foo')->get('bar', fn() => "");
Route::prefix('foo')->suffix('bar')->get('test', fn() => "");

That would result with the 3 following routes:

Any thoughts about this feature ?

Seems like this PR's tests are failing. Please make sure any tests pass when submitting PR's. Thanks


[WEEX-17] Abstract a common weex var for each JS framework

Accepted weex variable from JS runtime before legacyFramework.createInstance.

Just FYI, usually one commit per pull request[1] is recommended. Once it synced with apache git, you can't change or clean up commit histories.

http://ndlib.github.io/practices/one-commit-per-pull-request/


Typescript and pnp enabled app fails to compile

Describe the bug
yarn start results in the following error
Failed to compile.

undefined
TypeScript error in undefined(undefined,undefined):
Cannot find type definition file for 'lodash'.  TS2688

Did you try recovering your dependencies?

Ran yarn, using yarn@latest ( atm<EMAIL_ADDRESS>)
Which terms did you search for in User Guide?

typescript
yarn pnp
typescript and yarn pnp
lodash

Environment

Environment Info:
  System:
    OS: Linux 5.0 Ubuntu 19.04 (Disco Dingo)
    CPU: (8) x64 AMD Ryzen 7 2700U with Radeon Vega Mobile Gfx
  Binaries:
    Node: 11.14.0 - ~/.nvm/versions/node/v11.14.0/bin/node
    Yarn: 1.17.3 - ~/.nvm/versions/node/v11.14.0/bin/yarn
    npm: 6.10.2 - ~/.nvm/versions/node/v11.14.0/bin/npm
  Browsers:
    Chrome: 75.0.3770.142
    Firefox: 68.0.1
  npmPackages:
    react: Not Found
    react-dom: Not Found
    react-scripts: Not Found
  npmGlobalPackages:
    create-react-app: 3.0.1

Steps to reproduce

(Write your steps here:)

create-react-app test --typescript --use-pnp
cd test
yarn
yarn start

Expected behavior

No error occurs and app start sucessfully
Actual behavior

Reproducible demo

https://github.com/mateja176/yarn-pnp-demo

Related #5647

I hit this recently too, and couldn't find a fix. @iansu perhaps we should ask the Yarn team for thoughts on this?

We can try but I think they're pretty focused on shipping Yarn 2 at the moment. cc @arcanis

Fwiw I'd recommend using the v2 for any project using PnP.
Afaik a single blocker remains, which is the following error. We've recently fixed it in typescript-eslint master (https://github.com/typescript-eslint/typescript-eslint/pull/990), which should hopefully be released in the next few days.
Failed to load plugin '@typescript-eslint' declared in 'BaseConfig » /home/arcanis/cra-test/.yarn/virtual/eslint-config-react-app-virtual-fa3c1f4ab2/0/cache/eslint-config-react-app-npm-5.0.2-b7b16e6537-1.zip/node_modules/eslint-config-react-app/index.js#overrides[0]': A package is trying to access a peer dependency that should be provided by its direct ancestor but isn't

Required package: typescript (via "typescript")
Required by: tsutils@virtual:db6b93c726ffe978bd77e193f18a927c151ea6cc1f8f135748afebc0f8711d2c012ede19de70d3728ab968fc466d9b07698ccfcc97c03d093191cd80d35ad23f#npm:3.17.1 (via /home/arcanis/cra-test/.yarn/virtual/tsutils-virtual-4a3a585d45/0/cache/tsutils-npm-3.17.1-ed6df1e57e-1.zip/node_modules/tsutils/typeguard/2.8/node.js)

For now you can experiment by cloning typescript-eslint and yarn linking it to your project. Doing this seems to work fine for me 🙂

@arcanis thanks for the suggestion. It sounds like Yarn v2 is getting close to release? I personally can't wait for it! 🙌

This issue is reproducible with Yarn v2, fwiw:
Failed to compile.

undefined
TypeScript error in undefined(undefined,undefined):
Cannot find type definition file for 'babel__traverse'.  TS2688

@SEAPUNK Can you send a PR to Yarn's E2E test? Afaik CRA and Yarn 2 work fine:
https://github.com/yarnpkg/berry/blob/master/.github/workflows/e2e-cra-workflow.yml

@arcanis Sure, I'll try to get a minimal repro first.

Never mind, it's not happening because of our CRA TS setup, it's happening because it's in our monorepo, I assume there are files in the directory above that are messing with how it starts up.

Im also having this issue trying to use CRA as a monorepo package (yarn workspaces) with yarn PNP

any clues not to solve? seems nohoist is not an option for PNP

Also running into this in a monorepo with CRA.

Same problem here with CRA 3.4.1 & typescript 3.8.3 & yarn 2 monorepo:
Failed to compile.

undefined
TypeScript error in undefined(undefined,undefined):
Cannot find type definition file for 'accepts'.  TS2688

This has been fixed already, it was an issue in ts-pnp fixed in https://github.com/arcanis/ts-pnp/commit/66433e65cad9b1fbddac8349c352690c9c5f449a which was updated in https://github.com/facebook/create-react-app/pull/8947

Closing as this seems to be fixed.


Compilation failed for Raspberry Pi with undefined reference to `__atomic_store_8'

When I try to compile MAVSDK C++ on my Raspberry Pi (RPi 3B+ armv7l with Raspbian GNU/Linux 10 buster), it fails with the v0.35.1 but not with the v0.33.1
With the v0.35.1, it finishes with the error below:
$ cmake -DCMAKE_BUILD_TYPE=Debug -DBUILD_SHARED_LIBS=ON -DENABLE_MAVLINK_PASSTHROUGH=ON -DBUILD_TEST=ON -Bbuild -H.

...

-- Build files have been written to: /home/pi/libs/MAVSDK/build
$ sudo cmake --build build

...

[ 39%] Building CXX object src/CMakeFiles/unit_tests_runner.dir/plugins/telemetry/math_conversions_test.cpp.o
[ 40%] Linking CXX executable unit_tests_runner
/usr/bin/ld: core/libmavsdk.so.0.35.1: undefined reference to `__atomic_store_8'
collect2: error: ld returned 1 exit status
make[2]: *** [src/CMakeFiles/unit_tests_runner.dir/build.make:349: src/unit_tests_runner] Error 1
make[1]: *** [CMakeFiles/Makefile2:127: src/CMakeFiles/unit_tests_runner.dir/all] Error 2
make: *** [Makefile:130: all] Error 2

You need something like https://github.com/mavlink/MAVSDK/blob/e869ba85909d39d69ad4b326e41c9378badd1417/src/core/CMakeLists.txt#L74 but it should only be added for armv6 and armv7 targets, so it would need some if to check for that. Would you mind trying that out and making a pull request once you find a solution that works?

Done ! I don't have the issue anymore with the check thank you

Thank you for the contribution!


Remove Sampler.Close() function

This might be a breaking change.

Coverage decreased (-0.2%) to 83.412% when pulling c2c5596ce754b2f28cbb17dc7685fb9aa23ad7cf on remove_sampler_close into 54d9b0c959bde1d73944da43bc5131c6a887bc53 on master.

Coverage decreased (-0.2%) to 83.412% when pulling 3b58720d4b65317aeaee0837a156834668b248c7 on remove_sampler_close into 54d9b0c959bde1d73944da43bc5131c6a887bc53 on master.

Coverage decreased (-0.1%) to 83.49% when pulling aef3ab0aed31cf6f9102f3e36e984669a09b63c9 on remove_sampler_close into 54d9b0c959bde1d73944da43bc5131c6a887bc53 on master.

closing old PR


YAML support needed for multiple commissioners scenario

In Multi admin testcases, we need to run multiple commissioners (Chip tool, python controllers) at the same time connecting to DUT. Need YAML support to have multiple commissioners to automate these scenarios.

P2 , manually possible to test . Only needed for scripting.

It exists already. You can use: "identity": alpha on any command and it will send the command from commissioner "alpha".
You can choose between:

alpha (default)
beta
gamma


perf: improve f16 performance for norm L2 on aarch64

Explicilty convert to f32 on Mac M1/M2, achieve about 40% speed up without explicit SIMD, due to the lacking of f16p instructions in std::arch::aarch64.

NormL2(half::binary16::f16, auto-vectorization)
                        time:   [247.36 ms 247.79 ms 248.32 ms]
                        change: [-40.906% -40.724% -40.542%] (p = 0.00 < 0.10)
                        Performance has improved.


Merged main dev 1.32

This PR syncs the dev-1.32 branch with the main branch to keep the dev-1.32 branch up to date.
cc: @chanieljdan @AnshumanTripathi @rdalbuquerque @spurin @michellengnx

@hacktivist123 Wouldn't it be more appropriate to set the target branch for the pull request to dev-1.32 instead of main? In other words, we should consider changing the PR's target branch to ensure it aligns with the intended sync objective?

/lgtm

/approve


Google Datastore as InstallationStore returns DateTimeWithNanoSeconds which is not compatible with Bot model

Hey,
I'm currently running Slack Bolt in Google Cloud Run with Google Datastore as InstallationStore.
When trying to fetch values from datastore using:
datastore_client: Client = datastore.Client()
installation_store = GoogleDatastoreInstallationStore(
    datastore_client=datastore_client,
    logger=logger,
)
installation_store.find_bot(enterprise_id=eid, team_id=tid).to_dict()

I'm getting
Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/slack_bolt/listener/thread_runner.py", line 103,
 in run_ack_function_asynchronously listener.run_ack_function(request=request, response=response) File "/usr/local/lib/python3.6/site-packages/slack_bolt/listener/custom_listener.py", line 55,
 in run_ack_function this_func=self.ack_function, File "/app/main.py", line 178,
 in open_modal headers = {'Authorization':'Bearer {}'.format(installation_store.find_bot(enterprise_id=eid, team_id=tid).to_dict()['bot_token'])} File "/usr/local/lib/python3.6/site-packages/slack_sdk/oauth/installation_store/models/bot.py", line 75,
 in to_dict "installed_at": datetime.utcfromtimestamp(self.installed_at),
 TypeError: an integer is required (got type DatetimeWithNanoseconds)

Hi @gifflarn, thanks for sharing this. Probably, you are talking about this code example I've shared at https://github.com/slackapi/bolt-python/issues/251#issuecomment-791891023
I just fixed the issue in the code: https://gist.github.com/seratch/d81a445ef4467b16f047156bf859cda8/revisions#diff-1f60d250b00876744b8f3068f0f6863288d92817bef70c8cd8edef6086fe7d8b
Thanks again for the feedback!


Update parsing for descriptions files to work with other languages

Description blocks appear to work? now just need to convert the files into blocks and then code from using old file parser to new

Seems to be working great, but running out of ram soooo will have to figure that out

done


[BEAM-12925] Correct behavior retrieving NULL values from JDBC.

When JdbcIo was retrieving values from a db, it would not check whether the value was null.  This led to default values being used (normally 0 for numbers and false for booleans).  This change adds null checks.
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

R: @jbonofre
R: @timrobertson100

LGTM. We need to mention this in release notes for Beam 2.34.0. I'll update CHANGES.md


Experimental

This PR sets the default leakiness of ReLU layers to 0.01. This was the default in the old careless, and it seems to work a bit better for laue.

Codecov Report

Merging #28 (722962b) into main (fcafd8e) will decrease coverage by 0.03%.
The diff coverage is 80.00%.

:exclamation: Current head 722962b differs from pull request most recent head d55c898. Consider uploading reports for the commit d55c898 to get more accurate results

@@            Coverage Diff             @@
##             main      #28      +/-   ##
==========================================
- Coverage   86.46%   86.42%   -0.04%
==========================================
  Files          38       38
  Lines        1226     1230       +4
==========================================
+ Hits         1060     1063       +3
- Misses        166      167       +1

Flag
Coverage Δ

unittests
86.42% <80.00%> (-0.04%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

careless/models/scaling/nn.py
95.23% <80.00%> (-4.77%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fcafd8e...d55c898. Read the comment docs.


fix: do not flag nested or .d.ts namespaces.

Note: I've been unable to test this properly as on my machine grunt for some reason picks up tsc 2.0.3, which fails to compile tslint.

Friendly ping?

Thanks @adidahiya!


OPENWHISK_HOME env variable is not recongnized in mac system

OPENWHISK_HOME  is retrieved in the way like
def whiskhome = "$System.env.OPENWHISK_HOME"
in the gradle.build.
It works for linux, but not mac.
windows needs to be verified as well.

This is now fixed


[HOLD for payment 2023-10-20] [$500] [$1,000] Android - App freezes after killed app is called via concierge deep-link

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Login to NewDot
Kill app
Tap concierge deep-link from another mobile application (https://staging.new.expensify.com/concierge)

Expected Result:
App is opened, concierge chat is displayed
Actual Result:
App freezes
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[x] Android / native
[ ] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[ ] MacOS / Chrome / Safari
[ ] MacOS / Desktop

Version Number: v1.3.64-0
Reproducible in staging?: Y
Reproducible in production?: Y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://github.com/Expensify/App/assets/115492554/fe336b81-6be7-455a-be1c-d495d4b82ec9
Expensify/Expensify Issue URL:
Issue reported by: Applause - Internal Team
Slack conversation: @
View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~01fb7f1ef5548a9cf1
Upwork Job ID:<PHONE_NUMBER>868582400
Last Price Increase: 2023-10-04

Problem Statement:
When tapping on a deep link to the Concierge chat from another mobile application, the Expensify app freezes instead of displaying the Concierge chat.
Root Cause:
The root cause could be a combination of factors:

Delay or failure in resolving the Welcome.serverDataIsReadyPromise() which waits for server data.
The variable isNetworkOffline might not be representing the true network state, causing unintended logic paths to be executed.
Complex navigation logic in the navigateToConciergeChat and openReportFromDeepLink functions may not always result in a navigation action, causing the app to hang.
Potential conflicts in report routing, especially in cases of optimistically created reports that are later found to be preexisting.
Issues with the initialization of deep links in the DeepLinkWrapper.

Proposed Solution:

Ensure that the Welcome.serverDataIsReadyPromise() resolves correctly. Implement a timeout to handle scenarios where it might take too long or not resolve at all.
Improve the accuracy of the isNetworkOffline check, possibly by using a more robust method or library to check network connectivity.
Refactor navigation logic to ensure there's always a definitive navigation action for every possible state. Add proper error handling and fallbacks.
Handle report rerouting more gracefully, ensuring that the user experience isn't disrupted.
Thoroughly test and debug the deep link initialization process to ensure it works seamlessly.

Alternative Solution:

Bypass the server data check if it's causing significant delays, especially for operations that don't strictly require the most recent server data.
Use service workers or background tasks to pre-fetch or update necessary data, reducing the reliance on real-time network checks.
Simplify navigation logic by having a default route or chat that the app can fall back to in ambiguous situations.
Implement a more user-friendly error or loading screen that informs users when there's a delay, providing options to retry or navigate elsewhere.
Reevaluate the deep link structure or implementation method to ensure it's optimized for the app's architecture.

Thank you, @imranaalam, for your thoughtful analysis. Your insights are indeed valuable. However, it seems that we might need to dig a bit deeper to pinpoint the exact root cause of the issue. While you've highlighted several potential scenarios and solutions, finding the specific factor would be immensely helpful.

@robertKozik any update here?

@NicMendonca, I'm still waiting for proposals. The one that's already here doesn't pinpoint exactly what's causing it or how to tackle it. It just lists potential root causes and general solutions.

Doubled the price

^ doubled the price this week

Still waiting for proposals

Still no proposals for this issue 🙅🏼

@izarutskaya can you please confirm this issue is still reproducible?

@izarutskaya is this still reproducible?

Thanks for proposal @HezekielT. The root cause you found seems to be more general and the issue should be encountered on every platform. How can you explain that app freezes only on android (per issue reporter)?

Thank you for the thorough explanation, @HezekielT. I've reviewed your solution, and it does indeed resolve the issue. The root cause is clear, and your solution addresses it directly. Without any hesitation, I'll recommend bringing you on board for this task.
🎀 👀 🎀 C+ reviewed

@HezekielT can you please apply to the job here? Thanks -- https://www.upwork.com/jobs/~01fb7f1ef5548a9cf1

@robertKozik bump on BZ checklist: https://github.com/Expensify/App/issues/26867#issuecomment-1765325902

[X]  [@robertKozik] The PR that introduced the bug has been identified. Link to the PR: In my opinion this issue was an adge-case which was not handled before, so no any particular PR introduced it
[X]  [@robertKozik] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment: N/A
[X]  [@robertKozik] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion: N/A
[X]  [@robertKozik] Determine if we should create a regression test for this bug. I don't think the regression test is needed here
[X]  [@robertKozik] If we decide to create a regression test for the bug, please propose the regression test steps to ensure the same bug will not reach production again. N/A

@HezekielT - sent offer!

@HezekielT yep - we apply that in the form of bonuses.


Add option to disable clock skew correction

While trying to merge this, we found some test issues that need to be investigated - will update tomorrow re: merging and getting a release out.


The regression has been introduced by commit 5880724f for fixing #137

Fixed with commit a8e1bf23


Ability to test using local instance of firebase

Is your feature request related to a problem? Please describe.
For integration test (where we don't want the identifier to conflict, we want the ability to simulate network issue  etc.) and local testing (where we want to be able to run our server without access to the internet, in the train/plane etc.)  we want to use a local http server implementing the same endpoint as firebase  (we already have that for S3 ,  Zendesk , Stripe etc. )
However Kreait/firebase-php does not let us configure some of the URL that are hardcoded like
Firebase/Auth/Token/HttpKeyStore.php:    const KEYS_URL =<EMAIL_ADDRESS>
Describe the solution you'd like
It should be possible to override these either (both are not needed, only one would be already great):

from env variable   "KRAIT_FIREBASE_KEYS_URL"   (pro it's easy to modify and latter revert if you prefer 2)
from the factory constructor ?  (more versatile, but certainly requires more modification in the code ?)

Additional context
I can propose a PR for 1)  if you're ok with this feature request

I absolutely understand your need, but I'm not sure I can provide this with the current implementation of the SDK.
Environment variables would become too messy over time (the keys URL would certainly not remain the only one that needs to be configurable), and a "clean" implementation of configurable URLs via the Factory will require - as you said - quite some modifications in the code (I just spent an hour trying to come up with something 😅 )
I do believe that you could get closer to what you're looking for by extending/overriding the Factory class and its methods, specifically the createApiClient() method (e.g. to inject a MockHandler or middleware that change the request URLs), or specifically the createIdTokenVerifier() method.
Another approach could be to wrap the SDK calls in an application service and let the application service handle things differently depending on the environment, but I'm just thinking out loud here.
To be clear: I usually strive for making things easier and more enjoyable for users of the SDK, but in this case, I feel it would be too big of an endeavor to do so without changes so big that I would rather invest the time in a full rewrite of the SDK 🙈

thanks for your quick answer, I understand perfectly your position
currently I'm doing the wrapper solution, but I prefer to keep this for unittest and to have the integrationtest / local env exercising as much as possible the whole code, vendor included, as I've been bitten in the past by bugs in the 3rd party SDK that was not discovered during the CI ( because bypassed by the mocks at the code level)  and not discovered during manual QA because it was a edge case (the 3rd party library going in a crazy retry loop when the service was timeout-ing)
anyway thanks for taking the time to investigate deeply into my problem , very much appreciated :)


Sample googlebot transaction:
{
  "transaction": {
    "client_ip": "<IP_ADDRESS>",
    "time_stamp": "Tue Jul 12 15:54:19 2022",
    "server_id": "9e7bc6878f15155f664887f5952d257c0d032745",
    "client_port": 61702,
    "host_ip": "<IP_ADDRESS>",
    "host_port": 443,
    "unique_id": "1657641259",
    "request": {
      "method": "GET",
      "http_version": 1.1,
      "uri": "/",
      "headers": {
        "Host": "redacted.com",
        "AMP-Cache-Transform": "google;v=\"1..8\"",
        "Connection": "keep-alive",
        "Accept": "text/html,application/xhtml+xml,application/signed-exchange;v=b3,application/xml;q=0.9,*/*;q=0.8",
        "User-Agent": "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.114 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)",
        "From": "googlebot(at)googlebot.com",
        "Accept-Encoding": "gzip, deflate, br",
        "If-Modified-Since": "Tue, 24 May 2022 23:58:16 GMT"
      }
    },
    "response": {
      "body": "",
      "http_code": 403,
      "headers": {}
    },
    "producer": {
      "modsecurity": "ModSecurity v3.0.6 (FreeBSD)",
      "connector": "ModSecurity-nginx v1.0.3",
      "secrules_engine": "Enabled",
      "components": [
        "OWASP_CRS/4.0.0-rc1\""
      ]
    },
    "messages": [
      {
        "message": "Fake bot detected: ",
        "details": {
          "match": "Matched \"Operator `Pm' with parameter `applebot bingbot facebookbot facebookcatalog facebookexternalhit googlebot twitterbot' against variable `REQUEST_HEADERS:User-Agent' (Value: `Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chr (100 characters omitted)' )",
          "reference": "o153,9v265,200",
          "ruleId": "9504110",
          "file": "/usr/local/share/modsecurity-crs/plugins/fake-bot-after.conf",
          "lineNumber": "19",
          "data": "Matched Data: if-modified-since found within REQUEST_HEADERS:User-Agent: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.114 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)",
          "severity": "2",
          "ver": "fake-bot-plugin/1.0.0",
          "rev": "",
          "tags": [
            "application-multi",
            "language-multi",
            "platform-multi",
            "attack-bot",
            "capec/1000/225/22/77/13",
            "PCI/6.5.10",
            "paranoia-level/1"
          ],
          "maturity": "0",
          "accuracy": "0"
        }
      }
    ]
  }
}

Sample fake transaction curl "https://redacted.com/" -H "User-Agent: asd googlebot asd":
{
  "transaction": {
    "client_ip": "<IP_ADDRESS>",
    "time_stamp": "Tue Jul 12 15:59:08 2022",
    "server_id": "9e7bc6878f15155f664887f5952d257c0d032745",
    "client_port": 64532,
    "host_ip": "<IP_ADDRESS>",
    "host_port": 443,
    "unique_id": "1657641548",
    "request": {
      "method": "GET",
      "http_version": 2,
      "uri": "/",
      "headers": {
        "host": "redacted.com",
        "accept": "*/*",
        "user-agent": "asd googlebot asd"
      }
    },
    "response": {
      "body": "redacted",
      "http_code": 200,
      "headers": {}
    },
    "producer": {
      "modsecurity": "ModSecurity v3.0.6 (FreeBSD)",
      "connector": "ModSecurity-nginx v1.0.3",
      "secrules_engine": "Enabled",
      "components": [
        "OWASP_CRS/4.0.0-rc1\""
      ]
    },
    "messages": [
      {
        "message": "Fake bot detected: Googlebot",
        "details": {
          "match": "Matched \"Operator `InspectFile' with parameter `fake-bot.lua' against variable `TX:0' (Value: `googlebot' )",
          "reference": "o0,9v69,9",
          "ruleId": "9504110",
          "file": "/usr/local/share/modsecurity-crs/plugins/fake-bot-after.conf",
          "lineNumber": "19",
          "data": "Matched Data: googlebot found within REQUEST_HEADERS:User-Agent: Googlebot",
          "severity": "2",
          "ver": "fake-bot-plugin/1.0.0",
          "rev": "",
          "tags": [
            "application-multi",
            "language-multi",
            "platform-multi",
            "attack-bot",
            "capec/1000/225/22/77/13",
            "PCI/6.5.10",
            "paranoia-level/1"
          ],
          "maturity": "0",
          "accuracy": "0"
        }
      }
    ]
  }
}

Using fb1381ffb37e5f2118896e4d4ca717b9efc1687f, modsecurity 3.0.6, modsecurity-nginx 1.0.2, nginx 1.22.0.

Thank you for your submission. This looks like a valid Google bot:
# host <IP_ADDRESS>
<IP_ADDRESS>.in-addr.arpa domain name pointer crawl-66-249-64-219.googlebot.com.

# host crawl-66-249-64-219.googlebot.com
crawl-66-249-64-219.googlebot.com has address <IP_ADDRESS>

@azurit, are you getting notifications for this issue?

@ics Is DNS resolving working ok on that server? Have you tried with modsec 2.9?

Yes, resolving works. Tested in lua context too:
# ldd /usr/local/lib/libmodsecurity.so | grep liblua
        liblua-5.4.so => /usr/local/lib/liblua-5.4.so (0x801584000)
# lua54 -l socket
Lua 5.4.2  Copyright (C) 1994-2020 Lua.org, PUC-Rio
> hosts = socket.dns.getnameinfo("<IP_ADDRESS>")
> function dump(o)
>>    if type(o) == 'table' then
>>       local s = '{ '
>>       for k,v in pairs(o) do
>>          if type(k) ~= 'number' then k = '"'..k..'"' end
>>          s = s .. '['..k..'] = ' .. dump(v) .. ','
>>       end
>>       return s .. '} '
>>    else
>>       return tostring(o)
>>    end
>> end
> dump(hosts)
{ [1] = crawl-66-249-64-219.googlebot.com,}
>

I didn't try 2.9.

Willing to do some debug?

Willing to do some debug?

Sure.
I've already tried runnning modsecurity with SecDebugLogLevel 9 and made 2 requests. Both returned 200. Access log:
<IP_ADDRESS> - - [13/Jul/2022:22:18:16 +0000] "GET /aircraft/ALO3 HTTP/1.1" 200 2346 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" "-""-"
<IP_ADDRESS> - - [13/Jul/2022:22:19:39 +0000] "GET /aircraft/ALO3 HTTP/2.0" 200 12294 "-" "asd googlebot asd" "-""-"

1st request is made using Google search console URL Inspection.
2nd curl -H
Debug log: out.log
Please let me know if something stands out and what else can I try.

I've must have mixed them up. An example of blocked googlebot:<PHONE_NUMBER>.txt. It looks like it's blocked by something else:
"match": "Matched \"Operator `ValidateByteRange' with parameter `32,34,38,42-59,61,65-90,95,97-122' against variable         `REQUEST_HEADERS:From' (Value: `googlebot(at)googlebot.com' )",

The invalid bot requests made with curl are still going through. Here's a new curl request:
Request:
curl "https://doc8643.com/aircraft/BALL" -H "User-Agent: fake googlebot"

Access log:
<IP_ADDRESS> - - [14/Jul/2022:21:21:44 +0000] "GET /aircraft/BALL HTTP/2.0" 200 10089 "-" "fake googlebot" "-""-"

Debug log:<PHONE_NUMBER>.log

@ics: Thanks for this new attempt, but the log provided confirms that the plugin works just fine:
[1657833704] [/aircraft/BALL] [9] Target value: "fake googlebot" (Variable: REQUEST_HEADERS:user-agent)
[1657833704] [/aircraft/BALL] [7] Added pm match TX.0: googlebot
[1657833704] [/aircraft/BALL] [9] Matched vars updated.
[1657833704] [/aircraft/BALL] [4] Rule returned 1.
[1657833704] [/aircraft/BALL] [4] Executing chained rule.
[1657833704] [/aircraft/BALL] [4] (Rule: 0) Executing operator "InspectFile" with param "fake-bot.lua" against TX:0.
[1657833704] [/aircraft/BALL] [9] Target value: "googlebot" (Variable: TX:0)
[1657833704] [/aircraft/BALL] [1] googlebot
[1657833704] [/aircraft/BALL] [9] Returning from lua script: Fake Bot Plugin: Detected fake Googlebot.

So it's likely your blocking settings are at miss. Anomaly threshold too high?
Also: Can we close this?

Using default anomaly thresholds. Thanks for looking into it. I'll investigate further.


[WIP] PR: Split Editor module

This is not ready, I just move editor files and create a new structure without the sourcecode folder (avoiding it to be really nested)
editor/
├── api
│   ├── decoration.py      (was sourcecode/api/decoration.py)
│   ├── folding.py            (was sourcecode/api/folding.py)
│   └── __init__.py
├── __init__.py
├── plugin.py                   (was plugins/editor.py)
├── tests
│   └── __init__.py
├── utils
│   ├── codeeditor.py      (was utils/sourcecode.py)
│   ├── decoration.py      (was sourcecode/utils/decoration.py)
│   ├── editor.py              (was utils/editor.py)
│   ├── folding.py            (was sourcecode/folding.py)
│   ├── __init__.py
│   ├── kill_ring.py           (was sourcecode/kill_ring.py)
│   ├── terminal.py          (was sourcecode/terminal.py)
│   └── tests
│       ├── __init__.py
│       └── test_codeeditor.py   (was utils/tests/test_sourcecode.py)
└── widgets
    ├── base.py                 (was sourcecode/base.py)
    ├── codeeditor.py        (was sourcecode/codeeditor.py)
    ├── editor.py                (was widgets/editor.py)
    ├── __init__.py
    └── tests
        ├── __init__.py
        ├── test_autocolon.py        (was sourcecode/tests/test_autocolon.py)
        ├── test_autoindent.py       (was sourcecode/tests/test_autoident.py)
        ├── test_editor.py               (was widgets/tests/test_editor.py)
        ├── test_folding.py             (was sourcecode/tests/test_folding.py)
        └── test_panels.py             (was sourcecode/tests/test_panels.py)

@ccordoba12 @goanpeca What do you think of this new structure?
[skip pep8]

Wow! You went after our white whale!! Good luck with this ;-)

@rlaverde I really like the structure :-)

@rlaverde why these tests
│   └── tests
│       ├── init.py
│       └── test_codeeditor.py   (was utils/tests/test_sourcecode.py)
inside utils?

@goanpeca these test are for utils/codeeditor.py should I rename them? maybe test_utils_codeeditor.py

@rlaverde yes, so it is clearer :-)

these test are for utils/codeeditor.py should I rename them? maybe test_utils_codeeditor.py

I didn't move these files, they are general utilities for working with files and filenames (used also in the console), I just move a part that is related to the languages to editor/utils/languages.py

Also I think that the white whale es ready 😄, waiting the tests to pass 🙈

Hello @rlaverde! Thanks for updating the PR.

In the file spyder/api/mode.py, following are the PEP8 issues :

Line 27:80: E501 line too long (83 > 79 characters)
Line 73:80: E501 line too long (90 > 79 characters)

In the file spyder/plugins/editor/utils/languages.py, following are the PEP8 issues :

Line 27:24: E231 missing whitespace after ':'
Line 27:32: E231 missing whitespace after ','

In the file spyder/plugins/editor/widgets/editor.py, following are the PEP8 issues :

Line 47:80: E501 line too long (84 > 79 characters)
Line 48:80: E501 line too long (84 > 79 characters)
Line 2751:80: E501 line too long (80 > 79 characters)
Line 2755:80: E501 line too long (84 > 79 characters)

In the file spyder/widgets/panels/codefolding.py, following are the PEP8 issues :

Line 22:80: E501 line too long (92 > 79 characters)

The errors mention my pep8speaks are caused for # analysis:ignore
from spyder.plugins.editor.widgets.base import TextEditBaseWidget  # analysis:ignore
from spyder.plugins.editor.widgets.codeeditor import Printer # analysis:ignore

how should I fix them?

how should I fix them?

Don't worry about them. The Editor still needs further splitting, as we talked about before.
Please focus on make our tests to pass.

I didn't think I'd see this day ;-)
Great work @rlaverde! Merging away.


Remove proxy-manager-bridge

Not needed in 6.2 after https://github.com/symfony/symfony/pull/46752

I'm afraid this works, with Symfony 6, but renders it incompatible with Symfony LTS (i.e. 5.4), as per #37 :-/


Code folding bug

Problem
When a line of code in a function shows a line break, at this point, the entire function is folded and a bug occurs.
code like this:

The last tow lines are not folded:

But when the line break is deleted, everything is ok.Code like this:

Everything is ok:

Versions

Spyder version: 4.0.0b4
Python version: 3.7
Qt version:
PyQt version:
Operating System name/version: win7

I noticed this too. We'll take a look at this for our next beta.

I noticed this too. We'll take a look at this for our next beta.

Thank you and your team, the spyder is getting better and better.
Especially the code hints and completion functions are much better than before.

Thanks for your kind words!! We put a lot of effort on improving that in this version.

This bug also affects indentation guides, I modified the indentation guides in a merged PR so that they follow the same level and styling as code folding. If this big is resolved indentation guides should be updated accordingly.

Another problem I noticed with code folding is that all the commented lines after a function definition (outside of function - with no indentation) belong to the function and will fold with it; until a non-comment line.
This:

Will fold like this:

this seems to be persistent in V4.0rc2
... simple example that does not fold correctly after installing V4.0rc2:
def some_function(x):
    adsf = [[1,2,3],
            [4,6,8]

    print('adsf')
    return x

V4.0.0.dev0 has no code folding.


All secondary alignments lost

In the following data: data.tar.gz
there is exactly one single-end short read in input.fasta and two sets of haplotypes, full.fa.gz and subset.fa.gz.
Running Strobealign v0.13 with the subset of haplotypes:
strobealign --no-progress -M 1000 -N 1000 -S 0.8 -f 0 -k 15 --eqx -r 150 -t 1 subset.fa.gz input.fasta > subset.sam

produces 90 alignments (one alignment per each haplotype).
However, when running with a superset of haplotypes
strobealign --no-progress -M 1000 -N 1000 -S 0.8 -f 0 -k 15 --eqx -r 150 -t 1 full.fa.gz input.fasta > full.sam

only produces two alignments.
Changing k-mer size to 13 solves the issue in this particular case but I was wondering if there is a bug in the code that leads to prematurely discarding some alignments.
If you need, I can send more reads from the same locus, that produce the same bug (only one/two alignments are found instead of 200+).
Thank you in advance,
Timofey

Hi @tprodanov ,
It seems -k 13 does not solve this issue for me. Instead, changing -S to 0.55 or lower fixes the problem. For example, default parameter value (0.5) works, but 0.6 higher has the bug.
So we need to look into the -S parameter in more detail.
Thanks for reporting!
Best,
Kristoffer

I only have access to machine with an M1 chip until end of next week so I can't do any further testing on the exact issue, i.e., why not all the locations would have the same number of hits. I think this is a crucial starting place to investigate: https://github.com/ksahlin/strobealign/blob/3a97f6b817824235a36ca8f9c5710ee533d3ad56/src/aln.cpp#L120C1-L121C65
I will have a look later.

Thank you! I looked a bit into this part of the code, and I think the problem is in the number of generated hits, but I did not go there yet. With the full reference panel best location has 9 hits and most have 5 hits, while with the subset panel best has 19 and most have 18 hits.
But also I guess I misunderstood the meaning of -S, maybe help message can be changed from "Try candidate sites with mapping score at least S of maximum mapping score" to something like "Try candidate sites with at least S minimizer hits compared to the best alignment".

Thanks for looking into this.

With the full reference panel best location has 9 hits and most have 5 hits, while with the subset panel best has 19 and most have 18 hits

It's strange that the mapping sites wouldn't all have the same number of hits if they are all identical and perfect. I will figure it out once I get to do some debugging.

But also I guess I misunderstood the meaning of -S, maybe help message can be changed from "Try candidate sites with mapping score at least S of maximum mapping score" to something like "Try candidate sites with at least S minimizer hits compared to the best alignment".

Yes. We have been trying different measures (score/hits) for cutoff, so the description to this parameter should be updated as you suggest.

Thank you for looking into that! I will use the -R 3 fix for now then!

Great, and with more references just scale up R, where R*100 should be a value higher than the number of references to be sure.
Strobealign has another hard limit of 1000 (if ((rh.count > rescue_cutoff && cnt >= 5) || rh.count > 1000) {)which we should also consider making a user parameter for highly redundant references as a quick fix until better indexing solutions for highly redundant databases are explored.


[Fix imjp94/gd-YAFSM#14] Add transitions priority

Closes #14
The new commit cleans up the code and is partly based on your comments.
The rest of the new codes adds priorities to the existing interface using a spinbox.

Thanks for your addon!

Great works! The inspector looks really clean with the spinbox.
Just 2 more modifications before merging:

There's a bug at StateMachine.gd line 52, where sort_custom() is called by from_transitions which is a dictionary. A possible fix might look like this:

var sorted_from_transitions = from_transitions.values()
sorted_from_transitions.sort_custom(Transition, "sort")

for transition in sorted_from_transitions:
    ...

Clean the commits. There's 2 ways I can think of, I will leave it up to you:

Just rebase and squash 2 commits into 1
Or
One commit for "Add transition priority"(changes made to StateMachine.gd & Transition.gd) & another one for "Add inspector for transition priority"(changes made to all the editor script)

Sorry for the issue I let through, I already fixed it when I pushed my update, but I forgot to add the file from my test project.
And I squashed the commits together! 😄

Thanks for the contribution @adamscott!


It is hard to see the black part of one .glb file content when previewing it under 'Dark/High Contrast - Black' theme

Storage Explorer Version: 1.34.0-dev (98)
Build Number: 20240316.1
Branch: main
Platform/OS: Windows 10/Linux Ubuntu 22.04/MacOS Sonoma 14.4(Apple M1 Pro)
Architecture: x64/x64/x64
How Found: From running test cases
Regression From: Not a regression
Steps to Reproduce

Launch storage explorer -> Make sure it is under 'Light' theme.
Expand one storage account -> File Shares.
Open one file share -> Upload one .glb file that contains black part when it's being previewed.
Select the file -> Click 'Preview' -> Change the theme to dark.
Check whether it is clear to see the content.

Expected Experience
It is clear to see the content.
The screenshot under 'Light' theme.

Actual Experience
It is hard to see the content.

After review, our conclusion is that if when previewing a 3D model it is hard to see in a certain theme the user can decide to switch to a theme where the background makes it easier to view.

We are going to close this issue because it is marked won't fix.


Please DON'T MERGE/CLOSE

This is just a test Pull Request to configure GitHub Actions. I'll close it later.

LGTM! Let's merge this one 🙌


Check for / reject zero amount tokens

Add a check for considering zero amounts as missing.
For native coins, filter out zeros, and then ensure then remaining balance has length > 0. Only save the filtered list.
For cw20 coins, reject zero amounts directly. Add a check to also reject such sends in cw20-base.

As part of this, let's pull out the Balance helper from cw1-subkeys into cw0 (as it only relies on cosmwasm-std and used by many different protocols).
This already has a normalize function which you could use (or add a more efficient is_empty one).
This would be a great help to build the higher-level checks for native coins in all contracts.
I would also suggest 2 PRs here: one for the cw20 additions, one for the native coin changes (that I mentioned here)

Yes, I was taking a look at the subkeys code, and was going to suggest exactly that.

This is done with #86 and #87, correct?

Correct.


datadog-agent: 7.33.1 -> 7.34.0

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/DataDog/datadog-agent/releases.
meta.description for datadog-agent is: "Event collector for the DataDog analysis service\n-- v6 new golang implementation.\n"
meta.homepage for datadog-agent is: "https://www.datadoghq.com"
meta.changelog for datadog-agent is: ""
Updates performed

Golang update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
Warning: no invocation of /nix/store/cm8h71r5af6wrvi0w00pv3i69q63phl0-datadog-agent-7.34.0/bin/.agent-wrapped had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/cm8h71r5af6wrvi0w00pv3i69q63phl0-datadog-agent-7.34.0/bin/agent had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/cm8h71r5af6wrvi0w00pv3i69q63phl0-datadog-agent-7.34.0/bin/dogstatsd had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/cm8h71r5af6wrvi0w00pv3i69q63phl0-datadog-agent-7.34.0/bin/trace-agent had a zero exit code or showed the expected version
0 of 4 passed binary check by having a zero exit code.
0 of 4 passed binary check by having the new version present in output.
found 7.34.0 with grep in /nix/store/cm8h71r5af6wrvi0w00pv3i69q63phl0-datadog-agent-7.34.0
found 7.34.0 in filename of file in /nix/store/cm8h71r5af6wrvi0w00pv3i69q63phl0-datadog-agent-7.34.0

Rebuild report (if merged into master) (click to expand)

4 total rebuild path(s)

2 package rebuild(s)

2 x86_64-linux rebuild(s)
2 x86_64-darwin rebuild(s)

First fifty rebuilds by attrpath
datadog-agent
datadog-process-agent

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/cm8h71r5af6wrvi0w00pv3i69q63phl0-datadog-agent-7.34.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A datadog-agent https://github.com/r-ryantm/nixpkgs/archive/12c1e476814ad4497c24998b1b64914b76aa712e.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/cm8h71r5af6wrvi0w00pv3i69q63phl0-datadog-agent-7.34.0
ls -la /nix/store/cm8h71r5af6wrvi0w00pv3i69q63phl0-datadog-agent-7.34.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

2 packages built:

datadog-agent
datadog-process-agent

Maintainer pings
cc @thoughtpolice @domenkozar @rvl @kalbasit for testing.

Result of nixpkgs-review pr 162569 run on x86_64-linux 1

2 packages built:

datadog-agent
datadog-process-agent


Check for duplicate layer names across providers

Layer names should never clash across providers.

dupe of #81


Process ENV_DIR.

Environment variables won't be set unless we source them from ENV_DIR.
See https://devcenter.heroku.com/articles/buildpack-api#bin-compile.

Ping.

Merge?


REPL maker

A nice idea of @j-l-s : ReplMaker to parse an ocp from the REPL

That's very nice!

@ocots REPL command wish list
/name
/show
/debug
/solve
/plot
/help

lol you just changed the commands

ct> DEBUG

parsing string: DEBUG

expression parsed: DEBUG

command: DEBUG

 debug mode off

ct> SHOW

 display model: on
 display code:  on

Yes it is not possible with / or !.

must use a unary operator. test with @dump to check whether it parses in Julia or not. also important to distinguish from typical math writing as it is what guided the syntax of the DSL.

https://github.com/control-toolbox/CTBase.jl/assets/66357348/4a43ad8f-1cfd-4ac0-8781-dcecfedb4a9f

OK with that. Then the rule is no tag iff full and simultaneous declaration of time, state, control (in this order), plus variable if needed.
Note that we still need to parse and allow variable time bounds.

Just for save:
        @match e begin
            :( $e_, time                 ) => __add!(ct_repl, :time, e, history)
            :( $t ∈ [$a, $b]             ) => __add!(ct_repl, :time, :($t ∈ [$a, $b], time), history)
            :( $e_, state                ) => __add!(ct_repl, :state, e, history)
            :( $e_, control              ) => __add!(ct_repl, :control, e, history)
            :( $x, $u ∈ $X, $U ) || :( ($x, $u) ∈ $X × $U ) || :( ($x, $u) ∈ $X * $U )=> begin
                __add!(ct_repl, :state, :($x ∈ $X, state), history)
                __add!(ct_repl, :control, :($u ∈ $U, control), history)
            end
            :( $e_, variable             ) => __add!(ct_repl, :variable, e, history)
            :( $x, $u, $v ∈ $X, $U, $V ) || :( ($x, $u, $v) ∈ $X × $U × $V ) || :( ($x, $u, $v) ∈ $X * $U * $V )=> begin
                __add!(ct_repl, :state, :($x ∈ $X, state), history)
                __add!(ct_repl, :control, :($u ∈ $U, control), history)
                __add!(ct_repl, :variable, :($v ∈ $V, variable), history)
            end
            :( $a = $e1                  ) => __add!(ct_repl, :alias, e, history)
            :( ∂($x)($t) == $e1          ) => __add!(ct_repl, :dynamics, e, history)
            :( ∂($x)($t) == $e1, $label  ) => __add!(ct_repl, :dynamics, e, history)
            :( $e1 == $e2                ) => __add!(ct_repl, :constraints, e, history)
            :( $e1 == $e2, $label        ) => __add!(ct_repl, :constraints, e, history)
            :( $e1 ≤  $e2 ≤  $e3         ) => __add!(ct_repl, :constraints, e, history)
            :( $e1 ≤  $e2 ≤  $e3, $label ) => __add!(ct_repl, :constraints, e, history)
            :( $e_ → min                 ) => __add!(ct_repl, :objective, e, history)
            :( $e_ → max                 ) => __add!(ct_repl, :objective, e, history)
            _ => (println("\nct parsing error\n\nType HELP to see the list of commands or enter a valid expression to update the model."); return nothing)
        end

@ocots need to add somewhere the way to start the REPL (__init_repl()...) manually. I assume that using CTBase supposedly does the job but not working on my side.

To launch __init_repl, Base.active_repl must be defined but it is only when the REPL is running. I don't manage to launch the method when making the using.
I tried, see:
https://github.com/control-toolbox/CTBase.jl/blob/88d0550760172991cd68e687f028730a41f19472/src/CTBase.jl#LL153C1-L153C47
Maybe, with a wait condition we could run the init later.

I note to print the ocp only if it is incomplete, that is when we have the table.

@ocots some minor things to fix:

declaring tf, variable then t ∈ [ 0, tf ] doesn't seem to work (the variable declaration is misplaced - before the time one)
when there is an error, the line nevertheless seems to be stored (requires to UNDO)
since aliases are not expanded when you pre-parse the code, expressions like ẋ(t) are not recognised for dynamics (only xxxxx syntax is), leading to side effects as below (multiple dynamics)

All in all, I advocate a very simple pre-parsing, not duplicating the grammar:

either no pre-parsing at all, relying on one-only declaration (but possibility to UNDO)
or (seems more user friendly), possibility of redeclaration of time, state, control, variable, dynamics, objective (constraints all remain, being cumulative) according to sth like

        @match e begin
            :( $e_, time        ) => __add!(ct_repl, :time, e, history)
            :( $e_, state       ) => __add!(ct_repl, :state, e, history)
            :( $e_, control     ) => __add!(ct_repl, :control, e, history)
            :( $e_, variable    ) => __add!(ct_repl, :variable, e, history)
            :( ∂($x)($t) == $e_ ) => __add!(ct_repl, :dynamics, e, history)
            :( $e_ → min        ) => __add!(ct_repl, :objective, e, history)
            :( $e_ → max        ) => __add!(ct_repl, :objective, e, history)
            _                     => __add!(ct_repl, :constraints, e, history)
        end

Ok thanks.
But you do not use the last version.

@ocots thanks for the update on ct-REPL. probably need to try ... catch everything (= the main loop) to avoid hard backtraces (@j-l-s can't stand them 🥲), e.g. when trying sth like NAME, SHOW=0, etc.

Yes indeed. This is version 0. Feel free to update as you wish.


simple standalone mode not starting zk 3.4.6?

Hi,
I'm attempting to get standalone zookeeper and exhibitor running, and then move to a 3-zk ensemble afterward, but the first part of this attempt isn't working as I hoped it would. I'm on zookeeper 3.4.6 and exhibitor 1.5.3.
My zoo config:
$ cat conf/zoo.cfg
#Auto-generated by Exhibitor - Mon Jan 12 17:11:31 UTC 2015
#Mon Jan 12 17:11:31 UTC 2015
clientPort=2181
dataDir=/home/ubuntu/zookeeper/data
syncLimit=5
tickTime=2000
initLimit=10
dataLogDir=/home/ubuntu/zookeeper/transactions

Starting zk manually, it runs and responds to ruok:
$ bin/zkServer.sh start
JMX enabled by default
Using config: /home/ubuntu/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
$ echo ruok | nc localhost 2181
imok$

If I stop zk, then start it in the Exhibitor control panel (basic exhibitor config, nothing fancy) using the "start" button, the log files show:
INFO	ZooKeeper down/not-serving waiting 30000 of 40000 ms before restarting
INFO	Restart of ZooKeeper skipped due to control panel setting
INFO	State: down
INFO	ZooKeeper Server: Starting zookeeper ... STARTED
ERROR	ZooKeeper Server: Using config: /home/ubuntu/zookeeper/bin/../conf/zoo.cfg
INFO	Process started via: /home/ubuntu/zookeeper/bin/zkServer.sh
ERROR	ZooKeeper Server: JMX enabled by default
INFO	Starting in standalone mode
INFO	Attempting to start instance

Am I missing something really trivial? The ERROR messages don't actually seem to be true error messages to me, but indeed zookeeper in standalone isn't running after that attempt to start it via Exhibitor. In fact, I see the same messages as STDERR output when starting zk in foreground mode:
$ bin/zkServer.sh start-foreground 2> zkstderr.txt
$ cat zkstderr.txt
JMX enabled by default
Using config: /home/ubuntu/zookeeper/bin/../conf/zoo.cfg

So... what's going on exactly?

Eh. So I ran tail -f /var/log/zookeeper/zookeeper.out and saw:
nohup: failed to run command ‘/usr/lib/jvm/jdk/bin/java’: No such file or directory
So, I changed bin\zkEnv.sh to directly point to my java binary location:
JAVA="/usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/java"
And I also edited bin\zkServer.sh so that the JMX and config outputs didn't redirect to STDERR.
Looks like it's starting up fine in standalone mode now.


Pass test document to test and step events

Not plausible, test document is not available when creating step events.
Superceded by #311, #312


surface: linux 6.9.12 -> 6.10.3

Description of changes
Surface:

Update linux from 6.9.12 to 6.10.3
Update linux-surface from arch-6.9.9-1 to arch-6.10.3-1

Cc: @tracteurblinde
Things done

[X] Tested the changes in your own NixOS Configuration
[ ] Tested the changes end-to-end by using your fork of nixos-hardware and
importing it via <nixos-hardware> or Flake input

Passes ad-hoc testing on my SLS1.
Thank you for turning the crank on this.

@mergify queue


Add missing documentation to Hash.merge!(other : Hash, &block)

Completely missing description in api docs for Hash.merge! method, version with block. Without description, one cannot known which arguments are passed to given block (Except Ruby programmers, of course, because it behave same).
Related to #8377.

And now i found that i forgot non bang version with block, def merge(other : Hash(L, W), &block : K, V, W -> V | W) forall L, W. But when i try it, i found maybe a bug, at least it's very confusing. It's documented to accept other hash with any types of keys and values, but block is defined to receive type of key K as in self, but if you look at implementation, it iterates second one so it can't compile at all.
Try for example this:
h1 = {"a" => 1, "b" => 2, "c" => 3}
h2 = {:a => 1, :b => 2, :c => 3}
h3 = h1.merge(h2){|k,v1,v2| v1||v2}

which leads to:
Error: argument #1 of yield expected to be String, not Symbol
In my opinion it must be reimplemented or changed to: def merge(other : Hash(K, W), &block : K, V, W -> V | W) forall W
Than we should add documentation to it.

@jan-zajic how about
def merge(other : Hash(K2, V2), &block : K | K2, V, V2 -> V | V2) forall K2, V2

?

@RX14

def merge(other : Hash(K2, V2), & : (K | K2, V, V2) -> V | V2) forall K2, V2

That would work, but I think it's to complicated. The type restriction is not necessary for technical reasons, but mainly for documentation. For improving the documentation, this looks too complex IMO. Only benefit would be more precise error messages when using this method with non-matching types.
Btw. the key in the block argument is not K | K2 but only K because only keys from self are yielded.

They must be covariant with K, V.

Ah, I didn't look at the implementation....
class Hash(K, V)
  def merge2(other : Hash(K2, V2), & : (K2, V, V2) -> V | V2) : Hash(K | K2, V | V2) forall K2, V2
    hash = Hash(K | K2, V | V2).new
    hash.merge! self

    other.each do |k2, v2|
      if self.has_key?(k2)
        hash[k2] = yield k2, self[k2], v2
      else
        hash[k2] = v2
      end
    end

    hash
  end
end

This allows for the full type signature.
Another PR though...

@RX14 Your code looks good. But that's actually #merge not #merge!. And it matches its current implementation 😄

@straight-shoota We were talking about the non-bang version the whole time! I was responding to @jan-zajic's comment about the non-bang version. Also, my implementation is different to the current implementation in that it uses self[k2] instead of hash[k2] which restricts the key type sent to the block from K | K2 to K2.

@straight-shoota @RX14 yes that https://github.com/crystal-lang/crystal/pull/8380#issuecomment-546398715 comment ;). Well that this is already merged. We can continue that discussion in some issue or in the forum ...

Yeah, I was planning to make this another PR.

Ooops, I think I got lost somewehere.


advertisementData from iOS and Android are different

Hi,
I am actually trying to use the advertismentData retrieved by the scan method but in iOS the information is very different than in iOS.
What could I do to retrieve the specific information I am looking for?
Thank you.

Maybe this is due to "iOS does not allow you to access the RAW advertisement data."?

Well. Sorry for my ignorance and lack of searching before creating this Issue.
CoreLocation / CBCentralManager doesn't retrieve this kind of data.

Closed in favor of #102


WARNING: This video is DRM protected and only images are available for download. Use --list-formats to see them despite --list-formats being passed

Checklist

[X] I'm reporting a bug unrelated to a specific site
[X] I've verified that I'm running yt-dlp version 2022.07.18 (update instructions) or later (specify commit)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue

Provide a description that is worded well enough to be understood
WARNING: This video is DRM protected and only images are available for download. Use --list-formats to see them is being output, despite me passing --list-formats. I suspect that this is reproducible on ever website that has drm protected content
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
yt-dlp https://www.youtube.com/watch?v=lNWJpl3U6UY --list-formats --verbose
[debug] Command-line config: ['https://www.youtube.com/watch?v=lNWJpl3U6UY', '--list-formats', '--verbose']
[debug] User config "C:\Users\redacted\AppData\Roaming\yt-dlp\config.txt": ['--ffmpeg-location', 'C:\\Users\\redacted\\ffmpeg\\bin', '-P', 'C:\\Users\\redacted\\youtube.dl', '--update', '--audio-quality', '0', '--write-subs', '--write-auto-subs', '--embed-subs', '--compat-options', 'no-keep-subs,no-live-chat']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.07.18 [135f05e] (win32_exe)
[debug] Compatibility options: no-keep-subs, no-live-chat
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.19043-SP0
[debug] Checking exe version: "C:\Users\redacted\ffmpeg\bin\ffmpeg" -bsfs
[debug] Checking exe version: "C:\Users\redacted\ffmpeg\bin\ffprobe" -bsfs
[debug] exe versions: ffmpeg N-106498-g854615adf2-20220405 (setts), ffprobe N-106498-g854615adf2-20220405
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {}
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.07.18, Current version: 2022.07.18
yt-dlp is up to date (2022.07.18)
[debug] [youtube] Extracting URL: https://www.youtube.com/watch?v=lNWJpl3U6UY
[youtube] lNWJpl3U6UY: Downloading webpage
[youtube] lNWJpl3U6UY: Downloading android player API JSON
[youtube] lNWJpl3U6UY: Downloading tv embedded player API JSON
[youtube] lNWJpl3U6UY: Downloading web embedded client config
[youtube] lNWJpl3U6UY: Downloading player 7a7465f5
[youtube] lNWJpl3U6UY: Downloading web embedded player API JSON
[debug] Loading youtube-nsig.7a7465f5 from cache
[debug] [youtube] Decrypted nsig fDRR89DXC67YpnXgT => RScpsqxSQ5M0lA
[youtube] lNWJpl3U6UY: Downloading MPD manifest
WARNING: [youtube] Failed to download MPD manifest: HTTP Error 404: Not Found
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, codec:vp9.2, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), acodec, lang, proto, filesize, fs_approx, tbr, vbr, abr, asr, vext, aext, hasaud, id
WARNING: This video is DRM protected and only images are available for download. Use --list-formats to see them
ERROR: [youtube] lNWJpl3U6UY: This video is DRM protected
Traceback (most recent call last):
  File "yt_dlp\YoutubeDL.py", line 1431, in wrapper
  File "yt_dlp\YoutubeDL.py", line 1521, in __extract_info
  File "yt_dlp\YoutubeDL.py", line 1578, in process_ie_result
  File "yt_dlp\YoutubeDL.py", line 2481, in process_video_result
  File "yt_dlp\YoutubeDL.py", line 1002, in raise_no_formats
yt_dlp.utils.ExtractorError: [youtube] lNWJpl3U6UY: This video is DRM protected

yt-dlp https://7plus.com.au/the-good-doctor?episode-id=GOOD05-018 --list-formats --verbose
[debug] Command-line config: ['https://7plus.com.au/the-good-doctor?episode-id=GOOD05-018', '--list-formats', '--verbose']
[debug] User config "C:\Users\redacted\AppData\Roaming\yt-dlp\config.txt": ['--ffmpeg-location', 'C:\\Users\\redacted\\ffmpeg\\bin', '-P', 'C:\\Users\\redacted\\youtube.dl', '--update', '--audio-quality', '0', '--write-subs', '--write-auto-subs', '--embed-subs', '--compat-options', 'no-keep-subs,no-live-chat']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.07.18 [135f05e] (win32_exe)
[debug] Compatibility options: no-live-chat, no-keep-subs
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.19043-SP0
[debug] Checking exe version: "C:\Users\redacted\ffmpeg\bin\ffmpeg" -bsfs
[debug] Checking exe version: "C:\Users\redacted\ffmpeg\bin\ffprobe" -bsfs
[debug] exe versions: ffmpeg N-106498-g854615adf2-20220405 (setts), ffprobe N-106498-g854615adf2-20220405
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {}
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.07.18, Current version: 2022.07.18
yt-dlp is up to date (2022.07.18)
[debug] [7plus] Extracting URL: https://7plus.com.au/the-good-doctor?episode-id=GOOD05-018
[7plus] GOOD05-018: Downloading JSON metadata
[7plus] GOOD05-018: Downloading m3u8 information
[7plus] GOOD05-018: Downloading MPD manifest
[7plus] GOOD05-018: Downloading MPD manifest
[7plus] GOOD05-018: Downloading MPD manifest
[7plus] GOOD05-018: Downloading MPD manifest
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[7plus] GOOD05-018: Downloading JSON metadata
[debug] Downloading subtitles: en
WARNING: This video is DRM protected and only images are available for download. Use --list-formats to see them
ERROR: [7plus] GOOD05-018: This video is DRM protected
Traceback (most recent call last):
  File "yt_dlp\YoutubeDL.py", line 1431, in wrapper
  File "yt_dlp\YoutubeDL.py", line 1521, in __extract_info
  File "yt_dlp\YoutubeDL.py", line 1578, in process_ie_result
  File "yt_dlp\YoutubeDL.py", line 2481, in process_video_result
  File "yt_dlp\YoutubeDL.py", line 1002, in raise_no_formats
yt_dlp.utils.ExtractorError: [7plus] GOOD05-018: This video is DRM protected

The video is geo-restricted. Could you show me the output of yt-dlp -vUF --allow-unplayable-formats "URL"

nvm; I didn't see the second log

There is actually no images available for it. The warning is wrong

The (youtube) video is geo-restricted

nvm; I didn't see the second log

For clarity, the second video (in the second log by OP) is also geo-fenced here, but (unlike the YT one) the block can be circumvented  via  --geo-bypass-country AU 😜 ...


Unit test fails with Python 3.8.1

Running the unit tests with Python 3.8.1 fails with the following error:
=================================== FAILURES ===================================
_____________________________ test_bmp_characters ______________________________

    def test_bmp_characters():
        for index in range(0xa0, 0xfffd):
            char = chr(index)
            # Exclude code points that are not assigned
            if unicodedata.category(char) not in ('Co', 'Cn', 'Cs', 'Mc', 'Mn', 'Sk'):
                garble = char.encode('utf-8').decode('latin-1')
                # Exclude characters whose re-encoding is protected by the
                # 'sequence_weirdness' metric
                if sequence_weirdness(garble) >= 0:
                    garble2 = char.encode('utf-8').decode('latin-1').encode('utf-8').decode('latin-1')
                    for garb in (garble, garble2):
                        fixed, plan = fix_encoding_and_explain(garb)
>                       assert fixed == char
E                       AssertionError: assert 'à±·' == '౷'
E                         - à±·
E                         + ౷

tests/test_characters.py:23: AssertionError
=================== 1 failed, 186 passed, 8 xfailed in 1.84s ===================

Python 3.7.6 does not produce this error.

Yep, thanks. This is a routine update we have to make to ftfy (to save the character classes from the latest Unicode that Python supports in char_classes.dat).

Thanks a lot of fixing this! Any chance a new release could be cut? NixOS is currently in beta freeze for the 20.03 releaseand a minor bump of ftfy with this change would make spaCy work with Python 3.8 on NixOS

Right, thanks for the reminder.

Alright, version 5.7 is uploaded.

Hey guys, i'm currently working to add your package in conda-forge. Just to let you know this problem happen in Python v3.9.1.

Hey guys, i'm currently working to add your package in conda-forge. Just to let you know this problem happen in Python v3.9.1.


Add CAPI provider Workers

Description of changes:
This allows to represent a provider specific CAPI workers for both. It implements logic to generate new names for immutable objects when these change. It does this by comparing the new spec to the current state of the objects in the cluster.
This is part of a bigger effort, allowing a transition from yaml go templates to go structs. For context: https://github.com/aws/eks-anywhere/pull/3432
Testing (if applicable):
Unit tests
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/approve


[Splashscreen][Android] Default system splashscreen on Android devices at API 30 or below is stretched and zoomed in

Create a Blank app from the Uno template (not Default)
Launch Android on a device/emulator at API level 30 or below
Observe:
Android API 28:

Android API 33:

Fixed in uno.templates


Fixed transition-ios animation to remove unwanted delay to click after navigation

When navigating with the transition-ios the UI is not responding for a little time that makes the app fell slow. This is a proposed new setting (that can probably be improved) that removes the slow experience.

This needs to be merged asap since it is crucial for production apps

Is there a reason why this has been removed from the 3.9.0? The transition are still slowing the navigation on iOS.

Still reproducing in 3.9.0, this will fix the delay but it's not a long-term option.

Hello and thank you for contributing to Ionic! We have been working on porting all of the Ionic components to web components and have recently updated master to reflect this. This significant change has caused this pull request to break. While we really appreciate the time and effort you put into creating this, we are not able to merge it because of the newly introduced conflicts. We are extremely sorry about this. We will not be merging any more features in to v3. If this is a feature and you have the time, please resubmit this PR against the master branch. If this is a critical security issue in v3, we would greatly appreciate it if you would resubmit the PR against the new v3 branch. Thanks so much for your time!


Email target should only get notified on transition to error state

Email target should only get notified on transition into error state, and not on transition into Ok state, as the pagerduty email trigger can't tell them apart and makes false alarms.
@tomhaigh says in https://github.com/justeat/AwsWatchman/issues/3:

We could create an SNS topic for each type of notification target - i.e. an email one (containing all email subscriptions) and a url one (with all url subscriptions) for each alerting group. That would mean we could restrict what types of notification go to each type of target without additional configuration.

The alternative is to stop supporting email alerts. But they are useful for trialing so rather keep them.

Should we close this now as the plan is to use CloudFormation for everything?


Long processing times when handling very large files

Hello,
I've been here before and I'm back 😊
This gem has become a cornerstone in one of the projects I've developed. In most cases, it handles very well, minus some configuration options we need to customize to each scenario, but so it goes.
Now, we are working with larger files, 1.5million and such rows. In some cases, it seems to take hours. I've tested this before in tests with files between 500,000 and 1,000,000 rows, and have experienced around 15 minutes or more to fully process diffs of these files using the gem. We can deal with that even though it's not lovely, but any time taking longer than that is detrimental.
Now, I am not sure if this is an issue with how we provide key_fields or such, but I am mainly writing this issue out as a question on what experiences people have had with comparing large files? Is this a gem constraint, our own CSVDiff configuration, or something else?
What have you recorded for working with files of one-million plus rows, with up to 100 columns?

@agardiner You're gonna hear from me a lot but that's because this tool is incredibly valuable to our team.
We have been experiencing very long times to process files over 100,000 rows. Mainly I am talking 500k+ rows in files. These run for hours without producing results. Have you done any performance testing with large files?

I've not used this with files larger than 100k records, but I'd expect that performance drops off exponentially as your inputs grow. The implementation is pretty simple and works well for small inputs, but it was not designed for speed or to scale to large volume inputs.
Sorry I don't have any better news for you.

Came across this issue. If I continue using I can look to see if I can add a debug option so that at least you have insight into stage of process.