[Parse/Sema] Typealiases inside protocols

Split parsing for typealias & associatedtype, allow typealias in protocol, but not as a constraint in generics.
First step in getting real type aliases in protocols is to finish smoking out the ones that ought to be associated types. Cleaned up parsing of the two, including dropping a now unneeded ParseDeclOptions flag, made typealias in a protocol an error for the moment, and then converted all the places in the tests that didn’t care about it being a deprecation warning before but do care now.
Then made typealias in a protocol actually valid, which works great except that you can't use them in where clauses in generics. So added new diagnosis if you attempt to use a protocol’s typealias in a where clause, suggesting an associatedtype instead.

I've backed off from trying to make type aliases inside protocols work in generic where clauses right away  - it was taking longer than I thought it would, and this much of the work is separable and seems better as an individual pull anyway.
As an interim step, type aliases are now allowed inside protocols, we produce a specific diagnosis if you try to use them as if they were an associatedtype in a where clause, but otherwise they act like you would hope for protocol conformance purposes (i.e. as an alias, not as another hidden generic param).
@lattner I ended up in a merge conflict with your generic type aliases work. I think the resolution was fairly straightforward, but it'd be great if you could review the ParseDecl.cpp and test/decl/typealias/typealias.swift changes here when you have the time.

Oh, sorry about that, I'll take a look when I get a chance.
Unrelatedly, @DougGregor and I were discussing this briefly, in terms of how to land this in Swift 3.  IIRC, he had some concerns about how this lands - we want to make sure that people who ignore the "deprecated" warnings in Swift 2 get an acceptable upgrade experience in Swift 3.  Doug, do you have any specific guidance here?

@lattner No problem! Conflicts Happen.
The current behavior if you've ignored the deprecation warnings is that:

typealias A will complain that there is a missing = (assigning a type to the alias is required now, not optional). Maybe this ought to still have a better 'you probably meant associatedtype' diagnosis.
typealias A = Int will silently work for any type whose conformance actually uses Int (is this a problem? Not sure), and will fail with reasonable diagnoses if not (e.g. "protocol requires property 'foo' with type 'A' (aka 'Int')")

Could you split test updates (typealias => associatedtype) into a separate PR?  We could merge that immediately.

@gribozavr Good idea. Just did that here, all tests pass locally. You want to kick off a ci test, or shall I go ahead and just merge?
I'll re-add the actual feature changes in a separate pull request.

@swift-ci Please test

@gregomni Thank you!  Could you also retitle the PR?

@gregomni Feel free to merge when tests pass, thank you!

@gribozavr Done, and will do!


Feature request/idea: shrink?

,<a-e> could do a reverse operation?

I considered "reverse expand" but it's not as easy as expanding, and I'm not sure what would be the correct semantics.
If it's to simply reverse previous expand commands I think solving this issue would be a better direction.
If it's to shrink arbitrary selections then the problem is that there is not necessarily just one object that is a subset current, and then we somehow have to chose which. Let's say your selection is this, what should it shrink to?
[
   [a,b,c],
   [d,e,f],
]

Mh having written this I realize that we could just select both. Let me try to think about this

Ok I started working on this feature, I found an algorithm I'm satisfied with. If you check the shrink branch you should be able to try the new shrink command. It's configured with an option, similarly to expand_commands.  If you have some feedback that'd be cool

I'll be doing some refactoring (possibly to merge the expand and shrink implementations) before I merge but I don't expect any drastic changes

I like how selection expands keeping single cusor. It matches how word-movement.kak acts when I move with capital W and B. But when I shrink (maybe reduce is a more appropriate name?) the selection, there are new cursors added, which involves additional steps to remove unneeded cursor with Alt+Space, which is somewhat different from what I've expected.
I'm using the very same package for Emacs: expand-region.el, and I like how it works for different languages with different semantics. Is something like this possible with your plugin?
Also, it would be cool if it could be possible to select current word first, then current WORD, then from first non-whitespace character to last non-whitespace, then full line, and after that expand over regions. This is how Emacs package does it.

I’ve been watching the video but it does not show how reduce would work with double <div> or when it has multiple children.
https://youtu.be/_RvHz3vJ3kA?t=76

In my opinion the shrink feature is a nop.

It hangs Kakoune if I expand selection to whole file of C code which is 3500 lines long and will try to shrink the selection. Perhaps too many multiple cursors are produced by shrinking

You're right, in that case s\w+ produces a lot of selections which explodes the computation time. I also think that the fact that the shrinking result is independent for each selection makes the result really hard to predict when you chain multiple calls to shrink, it's not really usable currently

I’m currently working on a split object feature.  Maybe it could help.

I also think that the fact that the shrinking result is independent for each selection makes the result really hard to predict when you chain multiple calls to shrink, it's not really usable currently

That's why I feel that going to previous selection would be more appropriate way to shrink, but as you've said it is impossible to do without https://github.com/mawww/kakoune/issues/898 being fixed. Unless you could store selections by script. I've tried to do it, but couldn't find a proper way to extract ranges from stack

POC implementation of expand-reduce-selection command.
# a variable to store selection stack
declare-option str selection_stack ''

# this command adds current selection to the beginning of the stack
define-command expand-store-selection %{ evaluate-commands %sh{
    eval "set -- ${kak_opt_selection_stack}"
    printf "%s " "set-option global selection_stack %{$kak_selection_desc $@}"
}}

# this command takes first selection out of stack and selects it
define-command expand-select-previous %{ evaluate-commands %sh{
    eval "set -- ${kak_opt_selection_stack}"
    [ -n "$1" ] && printf "%s\n" "select $1"
    shift
    printf "%s " "set-option global selection_stack %{$@}"
}}

Modifications to expand command:
def expand -docstring '
Expand the current selections up to their next semantic block
' %{
    expand-store-selection # store current selection before expanding.
    expand-shrink-impl expand %opt{expand_commands}
}

Now we can reduce selection to previous one without generating new selections in case we've expanded too far. I think we need to update current selection if it was modified before expanding.


Temp file used multiple times within group is not deleted

Snakemake version
5.8.2
Describe the bug
If an output file is specified as temp() and is an input to multiple rules within the same group, it is not deleted after all rules have finished running.
Logs
[jholt@login01 snakemake_tests]$ snakemake --snakefile test_groups_temp.smk --profile lsf results/test_final_test.txt results/test_final2_test.txt
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cluster nodes: 5000
Job counts:
	count	jobs
	1	test_final
	1	test_final2
	1	test_temp
	1	test_temp2
	4
[Sun Dec 22 11:14:39 2019]

group job group (jobs in lexicogr. order):

    [Sun Dec 22 11:14:39 2019]
    rule test_final:
        input: results/test_temp_test.txt, results/test_temp2_test.txt
        output: results/test_final_test.txt
        log: logs/test_final_test.log
        jobid: 0
        wildcards: val=test

        cat results/test_temp_test.txt results/test_temp2_test.txt > results/test_final_test.txt

    [Sun Dec 22 11:14:39 2019]
    rule test_temp:
        output: results/test_temp_test.txt
        log: logs/test_temp_test.log
        jobid: 2
        wildcards: val=test

        echo "bob2" > results/test_temp_test.txt && sleep 10

    [Sun Dec 22 11:14:39 2019]
    rule test_temp2:
        input: results/test_temp_test.txt
        output: results/test_temp2_test.txt
        log: logs/test_temp2_test.log
        jobid: 3
        wildcards: val=test

        echo "bob" > results/test_temp2_test.txt && sleep 10

Submitted group job cddc3fe5-86bf-5d3c-942e-285090551e4a with external jobid '2541074'.
[Sun Dec 22 11:15:10 2019]
Finished job 3.
[Sun Dec 22 11:15:10 2019]
Finished job 0.
[Sun Dec 22 11:15:10 2019]
Finished job 2.
3 of 4 steps (75%) done

[Sun Dec 22 11:15:10 2019]
rule test_final2:
    input: results/test_temp2_test.txt
    output: results/test_final2_test.txt
    log: logs/test_final_test.log
    jobid: 1
    wildcards: val=test

        cat results/test_temp2_test.txt > results/test_final2_test.txt

Submitted job 1 with external jobid '2541075'.
Removing temporary output file results/test_temp2_test.txt.
[Sun Dec 22 11:15:20 2019]
Finished job 1.
4 of 4 steps (100%) done
Complete log: /gpfs/gpfs1/home/jholt/interrupt/snakemake_tests/.snakemake/log/2019-12-22T111439.332979.snakemake.log

Minimal example

rule test_temp:
    output:
        txt=temp("results/test_temp_{val}.txt")
    group: "group"
    log: "logs/test_temp_{val}.log"
    shell:
        '''
        echo "bob2" > {output} && sleep 10
        '''

rule test_temp2:
    input:
        txt="results/test_temp_{val}.txt"
    output:
        txt=temp("results/test_temp2_{val}.txt")
    group: "group"
    log: "logs/test_temp2_{val}.log"
    shell:
        '''
        echo "bob" > {output} && sleep 10
        '''

rule test_final:
    input:
        f1="results/test_temp_{val}.txt",
        f2="results/test_temp2_{val}.txt"
    output:
        f3="results/test_final_{val}.txt"
    group: "group"
    log: "logs/test_final_{val}.log"
    shell:
        '''
        cat {input.f1} {input.f2} > {output.f3}
        '''

rule test_final2:
    input:
        f2="results/test_temp2_{val}.txt"
    output:
        f3="results/test_final2_{val}.txt"
    log: "logs/test_final_{val}.log"
    shell:
        '''
        cat {input.f2} > {output.f3}
        '''

Additional context
In the example, I have two "final" files that I'm trying to create, both of which are made successfully with the expected output.  However, upon completion, one of the temporary files still exists:
[jholt@login01 snakemake_tests]$ ls -l results/
total 0
-rw-r--r-- 1 jholt csl 4 Dec 22 11:15 test_final2_test.txt
-rw-r--r-- 1 jholt csl 9 Dec 22 11:15 test_final_test.txt
-rw-r--r-- 1 jholt csl 5 Dec 22 11:15 test_temp_test.txt

Specifically, the one that is a pre-req for two other jobs within the same group.  Note that the other temporary file is a pre-req within the group and outside the group and is successfully deleted.  If you remove the group tags, all temp files are correctly removed.  Note, I've only tested with the LSF profile.

This bug persists as of 5.32.0.

It seems I am having the same issue on 6.8.1.
I am not grouping rules, yet I have a temporary file that is being used by two rules as input. Some get deleted and others don't.

First of all, sorry for the late response. After many new features, Snakemake is approaching feature completeness (at least for my personal agenda and at the moment). Hence, now is the time to fix or close all the open bugs. In the future, I will try to be faster with the fixes again.
Should be fixed with above PR.


Set correct name and description for Change Password setting

No new failed tests

@gfedorenko Could you please take a screenshot?

@mod ping


fix: fix patch_catboost() and patch_ngboost()

Description of change

copilot:all

Pull-Request Checklist

[ ] Code is up-to-date with the main branch
[ ] This pull request follows Contributing.md
[ ] This pull request links relevant issues as Fixes #0000
[ ] pre-commit run -a passes with this change
[ ] poetry run pytest passes with this change
[ ] There are new or updated unit tests validating the change
[ ] Documentation has been updated to reflect this change
[ ] The new commits follow conventions outlined in the conventional commit spec

Codecov Report

Merging #23 (725955b) into main (da14814) will increase coverage by 14.65%.
The diff coverage is 81.48%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
@@             Coverage Diff             @@
##             main      #23       +/-   ##
===========================================
+ Coverage   41.56%   56.21%   +14.65%
===========================================
  Files           9       10        +1
  Lines         587      603       +16
  Branches       90       69       -21
===========================================
+ Hits          244      339       +95
+ Misses        333      250       -83
- Partials       10       14        +4

Impacted Files
Coverage Δ

src/boost_loss/sklearn.py
59.03% <64.28%> (+19.84%)
:arrow_up:

src/boost_loss/__init__.py
100.00% <100.00%> (ø)

src/boost_loss/regression/__init__.py
100.00% <100.00%> (ø)

... and 6 files with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


feat: implementation for sorted set length and sorted set length by score

Addresses #581 by exposing the SortedSetLength and SortedSetLengthByScore endpoints for the Node.js and Web JavaScript SDKs

Setting remote repo to momentohq on my local fork and trying again to see if it'll resolve the secrets access issue


[DRAFT] - Add workflow to update manifests in odh-manifests repository

Fixes [#60 ]
Description
This github action update the manifests, tests and compatiblity matrix in odh-manifests and raise the PR automatically in odh-manifests
How Has This Been Tested?

Merge criteria:

[x] The commits are squashed in a cohesive manner and have meaningful messages.
[ ] Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
[ ] The developer has manually tested the changes and verified that the changes work

Need to add a Personal Access Token as a secret to the repositories

In case there are no changes to commit the workflow fails.
Can you make sure that commit and PR are done only if there is any change?

Looks like I don't have permissions to manage secrets in this repo. I'll take a look next week and see if I can get the permissions/someone else to do it for us.

In case there are no changes to commit the workflow fails. Can you make sure that commit and PR are done only if there is any change?

Tested locally - https://github.com/ChughShilpa/distributed-workloads/actions/runs/5940277483

@ChughShilpa is this ready for review or are you still working on this PR?

@sutaakar PR is ready for review.
Here is the successful GH action workflow runs on my fork: https://github.com/ChughShilpa/distributed-workloads/actions/runs/5950811645
Here is the PR to my fork of odh-manifests repository created via the workflow: https://github.com/ChughShilpa/odh-manifests/pull/9

@ChughShilpa There is a problem - the odh-manifest PR contains changes which should be committed.
The odh-manifest repo contains differences in paths in tests/basictests/ray.sh which need to stay the same: https://github.com/sutaakar/odh-manifests/pull/6/files#diff-6b623c6347bb950ad8d9319faea0a67342ca5dc4b7776644a5add7b3044695afL7-R9

@ChughShilpa There is a problem - the odh-manifest PR contains changes which should be committed. The odh-manifest repo contains differences in paths in tests/basictests/ray.sh which need to stay the same: https://github.com/sutaakar/odh-manifests/pull/6/files#diff-6b623c6347bb950ad8d9319faea0a67342ca5dc4b7776644a5add7b3044695afL7-R9

Added overwrite for paths in ray.sh
Tested changes in workflow - https://github.com/ChughShilpa/distributed-workloads/actions/runs/6010905000
and this opened a PR - https://github.com/ChughShilpa/odh-manifests/pull/12/files

The following users are mentioned in OWNERS file(s) but are untrusted for the following reasons. One way to make the user trusted is to add them as members of the opendatahub-io org. You can then trigger verification by writing /verify-owners in a comment.

test_user

User is not a member of the org. User is not a collaborator. Satisfy at least one of these conditions to make the user trusted.

Oops, i pushed my local changes for testing also :(

Reverted the changes

/retest

/retest

The e2e test is failing because of the InstaScale issue brought by latest operator release. New release is in progress, once the release is available the test can be retriggered.

/retest

rebasr

@sutaakar e2e tests passed now

/lgtm
@anishasthana is it good to merge?


QuestDB Support

QuestDB is compatible with the postgres wire protocol, however lacks many of the features used by flyway such as locks, as with the SQLite integration, unsupported operations are no-ops. This also includes schema support, when using the postgres wire protocol, everything is on the qdb schema, this is implemented by ignoring schema changes and always returning the same schema.

@Barry-RG As someone else looking for QuestDB support in Flyway, is this going to get merged?
It seems new versions of flyway-community-db-support get published to maven, last one 11 days ago, yet seemingly all the PR's tagged with database support seem to be left open and never merged?
Are people not supposed to use flyway-community-db-support and instead pull from the various branches they need and build it themselves?

Thank you for your submission. We cannot merge this at this time as it would need re-implementing in the https://github.com/flyway/flyway-community-db-support repository.


[6.x] add type to method of middleware stub

There is type hinting, but there was no type declaration.
I add type declaration to middleware stub.
When you execute command below, the stub will be used.
php artisan make:middleware new_middleware_name

This type hint can't be there since you do not know what request object you are getting at given times. :)

@gocanto
Thank you to reply!
May I ask you to instruct me what another type request object supposed to?
I checked type notation that is already written.
     * @param  \Illuminate\Http\Request  $request

@yamotuki he means that it's not allways the \Illuminate\Http\Request type of the request
with php artisan make:request StorePost and injecting that into the controller it will be different type:
// App\Http\Controllers\PostController
// @param \App\Http\Requests\StorePost $request

public function store(StorePost $request)
{
...
}

@yamotuki Sure thing; The issue here is what @rcerljenko exposed above.
Also, the main problem here is Laravel will try its best to resolve this request object. Meaning, it will inject the global one is needed. Which is an issue for another talk!

@rcerljenko
Thank you giving example.
StorePost class extents FormRequest and then FormRequest extends Request.  So all classes that handle() receive will be child class of Request. I expect no error.
Should type-declaration be exactly same to expect class ?
（On the other hand, type-hinting by phpdoc could not exact same class ? ）


feat: make ui more tidy and unified

same appearance for cmp and wilder
clear appearance for bufferline working with nvim-tree and outline (lspsaga)
colored indent-blankline
similar highlights for indent lines in nvim-tree, indent-blankline and outline (lspsaga)

closed by mistake :|

It's better to use a branch with name like "feat/unified-ui" to open pr (next time).

OK


Create --debug option

There should be a one-to-one mapping between environment variables and CLI options, like TRACE_LEVEL and --trace-level. But there is no --debug option corresponding to DEBUG. There may be others...

There is nothing in the code referencing DEBUG, as far as I can tell. IOW, this variable is not used.
We only have TRACE_LEVEL (internal logging, logr based) and VERBOSITY (helper/termui based for user interaction).
Both have associated options.


Are the values probability of resting and probability of greedy used?

Just started level 5 today of the discrete event simulator. Are the values P_r and P_g used anywhere? They are not in the constructor of RandomGenerator

It is not. You trigger the RNG to generate the next number based on P_r and P_g instead.

@RussellDash332 Thanks. But how do I do that?

Use the RNG's genRandomRest() and genCustomerType() method and see if it's less than or equal to P_r and P_g or not.
Actually, according to #127, the seed of the RNG for both methods depends on the RNG's seed constructor.


confirm modal onOk and onFail methods called at the same tim

this.$awn.confirm(
'my message',
this.myMethod1,
this.myMethod2
);
as soon as modal appears both myMethod1 and myMethod2 are being called even if I don't click any of the buttons

so is there a way to fix this bug?

Hi,
You have to wrap your methods into anonymous functions as it described in docs here.
Ex.:
this.$awn.confirm( 'my message', () => {this.myMethod1}, () => {this.myMethod2} );


Higher order components

Would be nice if reflux also provided methods for wrapping components in higher order components instead of using mixins, since mixins are on their way out

I'd gladly help implement something like that if @spoike agrees. Shouldn't take long

+1
Sounds like the React team have no plans to implement mixins for ES6 style React components so this would be a handy addition to reflux.

Closing since this is already ongoing in #293 it seems


Pci framework 0926

Summary

pci.c：fix ops not implement after calling panic err
pci.c: fix judge err in pci_connect_irq
pci.c: add error handle for pci_alloc_bus/device

Impact
Testing

[Experimental Bot, please feedback here]
This PR summary is missing crucial information:
Missing in Summary:

Why is the change necessary?  Simply stating "fix" isn't helpful.  Explain the bug, the desired improvement, or the need for the new feature.
How does the change work? Provide a brief technical explanation of your solution.

Missing in Impact:

All sections are empty.  You must address each impact area. Even if the answer is "NO", state it explicitly. For example:

"Impact on user: NO"
"Impact on build: NO"

Missing in Testing:

Provide actual testing logs. "your testing logs here" is not helpful.  Include relevant snippets of logs demonstrating the issue before and the successful outcome after your changes.
Specify your testing environment. Detail the build host OS, CPU, compiler, target architecture, board, and configuration used for testing.

In addition to filling in the missing information, consider these suggestions:

Conciseness:  While the headings are good, the descriptions can be more concise.
Clarity: Use descriptive language to explain the changes and their effects.

By providing this missing information, you'll make it much easier for reviewers to understand and assess your PR, significantly increasing its chances of being accepted.

@lipengfei28 please resolve the conflict.

done


api/getTrips/ URL not found

When using transit-map (after setup with GTFS-viz), I got errors in the javascript console, like these ones, that prevent any line to display:
http://tag.acolytesanonymes.org/api/getTrips/2228 Failed to load resource: the server responded with a status of 404 (Not Found)
http://tag.acolytesanonymes.org/api/getTrips/2229 Failed to load resource: the server responded with a status of 404 (Not Found)
XHR finished loading: GET "http://tag.acolytesanonymes.org/api/getTrips/2229".
http://tag.acolytesanonymes.org/api/getTrips/2230 Failed to load resource: the server responded with a status of 404 (Not Found)
How can I fix this?

Same problem here. I think I've followed every step on its guide (I've imported the gtfs using GTFS-viz)

@vasile Any updates on this?

Not yet Nicolas, the API is missing completely, I will see if I can quickly hack something

@vasile - any updates in this?

guys? someone?

Is this Repo still active?

@vasile do you have any time to spend on this for a fix? Thanks :)

@kalon33, I think the project is dead, @vasile doesn't have any obligation to patch his code, neither to solve our issues.


will anybody add Juneteenth for US?

6 months left

This package is not longer maintained. Read here:
https://github.com/quantopian/trading_calendars#important-note


Adding contribution guideline

@cdpierse
While contributing one PR to this awesome repo, I observed that it would be great to have contribution guideline.
Some of possible points can be added to this guide are -

Fork the repo [If not done]
Fetch upstream (git fetch upstream)
Checkout dev branch
Rebase dev branch with upstream (git rebase upstream/dev)
Create new branch and add changes to that branch
Add/update/remove relevant test cases
Perform black formatting on changes
Raise PR against upstream dev branch
Ask for the review

Few observations about CI process -

CI should run on every PR, so PR raiser will get feedback about test failure or build issue early
Currently PR merge is very open, it does not check for PR approval and CI pass status

@lalitpagaria this is definitely a top priority, will hopefully be able to dedicate the weekend to getting a decent and thorough guide out.


[5.1] Undo "Disable TSan in coroutine functions" (0ca3f79).

Cherry-pick of https://github.com/apple/swift/pull/23952 to 5.1.

@swift-ci please test and merge


Feat: Allow custom individual summary

Allows for customized individual summaries read automatically from the combined-summary.adoc file in a catchup session.

@KartikSoneji can you handle https://github.com/OurTechCommunity/catchup/blob/d6b9d63ad88ac6b887bccd734d3a7f5108b1ad75/summary/combined-summary.adoc#otc-catchup-96 this as well in this PR?😅
I completely forgot about this.

Do we need this image still?
public/img/69th-catchup.png

Do we need this image still?
public/img/69th-catchup.png

Where it is used?👀

Do we need this image still?
public/img/69th-catchup.png

Where it is used?👀

It was only added for the 69th catchup in 059102dedd7428225fd45dac0a61f9bfadae26e8

It was never used anywhere in repo right?
Maybe it shouldn't be in that folder but let's keep it somewhere or use it. Because it's something which we created to celebrate 69th catchup. It's one of the memories which we should preserve 😅

It was never used anywhere in repo right?

It was used for the redirect page in the week of the 69th CatchUp.

Maybe it shouldn't be in that folder but let's keep it somewhere or use it. Because it's something which we created to celebrate 69th catchup. It's one of the memories which we should preserve 😅

Sure, we can add it to the 69th CatchUp's summary as a note.

@KartikSoneji can you open new issue for it


[usecase] Add all-in-one config for usecase docroot

https://crosswalk-project.org/jira/browse/XWALK-5905

I have no idea the relationship between docroot and changes in this PR.
@Honry do you know this?

CSP test requires a http server, we use tinyweb server via deploying source code in docroot.

LGTM


Do not preserve temporary results when no need to do so

PR Summary
Fix #17836
Do not preserve temporary results when no need to do so.
PR Checklist

[x] PR has a meaningful title

Use the present tense and imperative mood when describing your changes

[x] Summarized changes
[ ] Make sure all .h, .cpp, .cs, .ps1 and .psm1 files have the correct copyright header
[x] This PR is ready to merge and is not Work in Progress.

If the PR is work in progress, please add the prefix WIP: or [ WIP ] to the beginning of the title (the WIP bot will keep its status check at Pending while the prefix is present) and remove the prefix when the PR is ready.

Breaking changes

[ ] None
OR
[ ] Experimental feature(s) needed

[ ] Experimental feature name(s):

User-facing changes

[x] Not Applicable
OR
[ ] Documentation needed

[ ] Issue filed:

Testing - New and feature

[x] N/A or can only be tested interactively
OR
[ ] Make sure you've added a new test if existing tests do not effectively test the code changed

/remind me tomorrow

Should we document this as a breaking change (formally at least)?


My incoming and outgoing data spikes massively when sending usage statistics

Problem description
If I set the send usage statistics check in my docker preferences to true my incoming and outgoing data goes through the roof. When I don't send the usage statistics my usage goes back to normal. I'm running docker on my MacBook Pro (2019) running MacOS Mojave (10.14.6). While having normal usage I've received 252GB from my VPN and have uploaded 9 over the last 30 hours.

Since not sending the usage statistics this morning my incoming and outgoing data normalized. The pictures below show the before and after of my received (ontvangen gegevens) and sent (verzonden gegevens) data per seconds. The before screenshot is taken without sending the usage statistics and after is with usage statistics.
This has nothing to do with my VPN, it's just how we found out. The issue remains even when I'm not on my VPN.
Before

After

Some logs:
MBs differentiated by port
Usage VPN -> myself and myself -> VPN
-------------------------------------
| srcPort | dstPort | MBTransferred |
|---------|---------|---------------|
| 17277   | 65312   | 217599.541    |
| 65312   | 17277   | 7222.7983     |
| 17277   | 49707   | 6510.8354     |
| 17277   | 51552   | 6006.0026     |
| 17277   | 58875   | 4567.2904     |
| 17277   | 62444   | 3375.0162     |
| 17277   | 57010   | 1236.5622     |
| 58875   | 17277   | 524.8057      |
| 62444   | 17277   | 425.2852      |
| 17277   | 64655   | 328.5233      |
-------------------------------------

Usage VPN -> myself
-------------------------------------
| srcPort | dstPort | MBTransferred |
|---------|---------|---------------|
| 17277   | 65312   | 217599.541    |
| 17277   | 58875   | 11723.2787    |
| 17277   | 51552   | 6006.0026     |
| 17277   | 62444   | 3375.3365     |
| 17277   | 49707   | 2058.4076     |
| 17277   | 57010   | 1236.5622     |
| 17277   | 64655   | 328.5233      |
| 17277   | 51205   | 87.0528       |
| 17277   | 55062   | 3.6723        |
| 443     | 54210   | 0.008666      |
-------------------------------------

Project version(s) affected
docker desktop community <IP_ADDRESS> (38240)

Found the root cause: some old containers were still running which were using a lot of data. Unsure why unchecking the send usage statistics box also (temporarily) helped normalizing my usage.


Utility function to help implement API key patterns

Split from:

#45

Here's how datasette-enrichments-gpt does that at the moment: https://github.com/datasette/datasette-enrichments-gpt/blob/1353a46f08f73e9b18a58115d74730fdab7427ad/datasette_enrichments_gpt/init.py#L32-L116
    async def get_config_form(self, datasette, db, table):
        class ConfigForm(Form):
            # Select box to pick model from gpt-3.5-turbo or gpt-4-turbo
            model = SelectField(
                "Model",
                choices=[
                    ("gpt-3.5-turbo", "gpt-3.5-turbo"),
                    ("gpt-4-turbo", "gpt-4-turbo"),
                    ("gpt-4-vision", "gpt-4-turbo vision"),
                ],
                default="gpt-3.5-turbo",
            )
            # ... more stuff like that

        def stash_api_key(form, field):
            if not (field.data or "").startswith("sk-"):
                raise ValidationError("API key must start with sk-")
            if not hasattr(datasette, "_enrichments_gpt_stashed_keys"):
                datasette._enrichments_gpt_stashed_keys = {}
            key = secrets.token_urlsafe(16)
            datasette._enrichments_gpt_stashed_keys[key] = field.data
            field.data = key

        class ConfigFormWithKey(ConfigForm):
            api_key = PasswordField(
                "API key",
                description="Your OpenAI API key",
                validators=[
                    DataRequired(message="API key is required."),
                    stash_api_key,
                ],
                render_kw={"autocomplete": "off"},
            )

        plugin_config = datasette.plugin_config("datasette-enrichments-gpt") or {}
        api_key = plugin_config.get("api_key")

        return ConfigForm if api_key else ConfigFormWithKey

Then it accesses that stashed key here:
https://github.com/datasette/datasette-enrichments-gpt/blob/1353a46f08f73e9b18a58115d74730fdab7427ad/datasette_enrichments_gpt/__init__.py#L229-L244
def resolve_api_key(datasette, config):
    plugin_config = datasette.plugin_config("datasette-enrichments-gpt") or {}
    api_key = plugin_config.get("api_key")
    if api_key:
        return api_key
    # Look for it in config
    api_key_name = config.get("api_key")
    if not api_key_name:
        raise ApiKeyError("No API key reference found in config")
    # Look it up in the stash
    if not hasattr(datasette, "_enrichments_gpt_stashed_keys"):
        raise ApiKeyError("No API key stash found")
    stashed_keys = datasette._enrichments_gpt_stashed_keys
    if api_key_name not in stashed_keys:
        raise ApiKeyError("No API key found in stash for {}".format(api_key_name))
    return stashed_keys[api_key_name]

Since I'm encouraging integration with datasette-secrets anyway, I'm going to allow Enrichment subclasses to indicate that they need an API key. If they do that, the API key form trick will be applied for them.
One potential design:
class GptEnrichment(Enrichment):
    name = "AI analysis with OpenAI GPT"
    slug = "gpt"
    description = "Analyze data using OpenAI's GPT models"
    runs_in_process = True
    batch_size = 1
    needs_api_key = "OPENAI_API_KEY"
    needs_api_key_label = "OpenAI API Key"
    needs_api_key_obtain_url = "https://..."

Or... since we'll be using register_secrets() from https://github.com/datasette/datasette-secrets for this anyway, maybe we reuse that datasette_secrets.Secret() object here? That would be pretty neat.

I'm going to experiment with secret = Secret(...) - and then a pattern where register_secrets() returns those secrets.

Maybe don't even tell plugins they have to call register_secrets() themselves - the datasette-enrichments plugin could automatically return ANY secrets property of a subclass as part of its own register_secrets() hook implementation.

This depends on datasette-secrets and that depends on datasette>=1.0a13 so this can only be in an alpha release for the moment.

... which means the entire enrichments ecosystem is about to go alpha-only, which isn't ideal.

Shipped datasette-test 0.3 with these changes:

https://github.com/datasette/datasette-test/issues/4
https://github.com/datasette/datasette-test/issues/5

OK, datasette-secrets is now compatible with <1.0 and >=1.0a Datasette.

Writing the documentation first - documentation-driven development.

Here's that initial documentation: https://github.com/datasette/datasette-enrichments/blob/5e06d06db0572880668e3dd8e6fbde8b04a9db73/docs/developing.md#enrichments-that-use-secrets-such-as-api-keys

The register_secrets() bit will be handled by the datasette-enrichments plugin itself, no need for actual enrichments plugin to register their secret, it will be picked up from secret =  on all subclasses of Enrichment.

I'll use this pattern for that:
class BaseClass:
    subclasses = []

    def __init_subclass__(cls, **kwargs):
        super().__init_subclass__(**kwargs)
        cls.subclasses.append(cls)

I tested this and it works! The secrets show up in the /-/secrets interface as they should.
Running test server like this at the moment:
export KEY=$(datasette secrets generate-encryption-key)
datasette *.db -p 8026 --root \
  -s plugins.datasette-secrets.encryption-key $KEY \
  -s permissions.manage-secrets.id root \
  --secret 1

Next step: make it so this works - it should try get_secret() and, if it fails, it should apply that pattern where it adds an extra _secret key to the form.

Need to think a bit more about how that field is displayed. Also the field doesn't do anything yet.

Better:

That's for:
class GptEnrichment(Enrichment):
    name = "AI analysis with OpenAI GPT"
    slug = "gpt"
    description = "Analyze data using OpenAI's GPT models"
    runs_in_process = True
    batch_size = 1
    secret = Secret(
        name="OPENAI_API_KEY",
        description="OpenAI API key",
        obtain_label="Get an OpenAI API key",
        obtain_url="https://platform.openai.com/api-keys",
    )

Next step is to figure out how to have an Enrichment subclass use the resolved secret. I think a self.get_secret() method might be the way to do that.

Everything is working now, needs tests before I land it.


theHarvester error: the following arguments are required:  -d/--domain name

Greetings,
When I start theHarvester I get this error: the following arguments are required:  -d/--domain name
I cannot enter domain names. Is there a fix for this ?
Thank you for your time.

wow crickets eh?


[EV-3803] Add helm installation reference doc

Product Version(s):

Issue:

Link to docs preview:

SME review:

[ ] An SME has approved this change.

DOCS review:

[x] A member of the docs team has approved this change.

Additional information:

Merge checklist:

[ ] Deploy preview inspected wherever changes were made
[ ] Build completed successfully
[ ] Test have passed

First sentence on page needs period at end:
You can customize the following resources and settings during Calico Enterprise Helm-based installation using the file, values.yaml

lgtm


PHPCBF only partially fixes spacing in this block

With phpcbf --standard=WordPress-VIP test.php, spaces are added to the inner function, but not to usort().
Before:
<?php
usort($authors, function ($a, $b) {
    $a = 1;
});
?>

After:
<?php
usort($authors, function ( $a, $b ) {
    $a = 1;
});
?>

Expected:
<?php
usort( $authors, function ( $a, $b ) { // space after usort(
    $a = 1;
}); // maybe here too
?>

We use PEAR.Functions.FunctionCallSignature from upstream in PHPCS to sniff for function call spacing. PEAR standards require special spacing for multi-line function calls, and because this doesn't match WordPress's standards, we disable those parts of the multi-line function call checks in our ruleset. I think we need to subclass that sniff and override the method that handles multi-line function calls.

PR https://github.com/WordPress-Coding-Standards/WordPress-Coding-Standards/pull/1020 made a change to how this is handled in the fixer.
The snippet would now be fixed as:
usort(
	$authors, function ( $a, $b ) {
		$a = 1;
	}
);

@paulschreiber Considering the above, can this issue be closed ?

That seems like too much space. Putting $authors on its own line makes it harder to track that it's a parameter to usort(). I prefer the solution in "Expected" above.

Currently, the snippet would be fixed to:
usort(
	$authors,
	function ( $a, $b ) {
		$a = 1;
	}
);

While we can have opinions on whether this is too much space or not, it is not incorrect as a fix, so I'm going to close this issue as superseded by a changed reality.


deps: upgrade ed25519-dalek

Description

Notes & open questions

Change checklist

[ ] Self-review.
[ ] Documentation updates if relevant.
[ ] Tests if relevant.

sth is fishy, all cli tests are failing

yep.. looking at how to fix it, not sure how to yet. Tried a couple of things without luck

actually if I try this on main I get a failure too :thinking:

Please update to remove the cargo-deny exception in #1361

closing in favor of #1373


Update 3.0.x binaries to 3.0.0

Darn, I just released beta4..

Link to artifacts: http://repo1.maven.org/maven2/com/google/protobuf/protoc/3.0.0/
From page: https://github.com/google/protobuf

Ack

Released 3.0.0