Missing certificates resources

I was able to complete the installation of Flux using task cluster:install
kubectl -n flux-system get pods -o wide
NAME                                          READY   STATUS    RESTARTS      AGE   IP            NODE       NOMINATED NODE   READINESS GATES
helm-controller-784f764bfb-7m8pm              0/1     Running   1 (15s ago)   45s   <IP_ADDRESS>    worker-1   <none>           <none>
image-automation-controller-b7c5b48f6-vszrr   0/1     Running   1 (5s ago)    35s   <IP_ADDRESS>   worker-1   <none>           <none>
image-reflector-controller-6658b59bc4-fs2lx   0/1     Running   0             27s   <IP_ADDRESS>    worker-3   <none>           <none>
kustomize-controller-fd48f87c4-kfbxz          0/1     Running   0             20s   <IP_ADDRESS>   worker-2   <none>           <none>
notification-controller-6b9b544755-6v8fx      0/1     Running   0             13s   <IP_ADDRESS>    worker-3   <none>           <none>
source-controller-8448d9bf67-52qzt            0/1     Running   0             5s    <IP_ADDRESS>    worker-2   <none>           <none>

Verified that the Flux components are running in the cluster
Then failed to verified all the resources on the cluster with task cluster:resources
task: [cluster:certificates] kubectl get certificates -A
error: the server doesn't have a resource type "certificates"
task: Failed to run task "cluster:resources": exit status 1

Anyone having a similar issue?

It doesn't look like flux is healthy, none od the pods are reporting 1/1

Looks like there is a communication issues
Events:
  Type     Reason     Age                     From     Message
  ----     ------     ----                    ----     -------
  Normal   Pulled     47m (x38 over 152m)     kubelet  Container image "ghcr.io/fluxcd/source-controller:v1.1.2" already present on machine
  Warning  Unhealthy  22m (x284 over 152m)    kubelet  Readiness probe failed: Get "http://<IP_ADDRESS>:9090/": dial tcp <IP_ADDRESS>:9090: connect: connection refused
  Warning  BackOff    2m53s (x603 over 150m)  kubelet  Back-off restarting failed container manager in pod source-controller-8448d9bf67-52qzt_flux-system(3f9b9a9b-1707-4010-a3b5-c9b0c9f83ba0)

I'm looking into this further.

Looks like more gremlins!

@onedr0p I nuked and recreated the cluster. Everything in kube-system looks healthy
kubectl get pods -n kube-system
NAME                               READY   STATUS    RESTARTS       AGE
cilium-8t5z5                       1/1     Running   0              11m
cilium-d7zmg                       1/1     Running   0              11m
cilium-m24pg                       1/1     Running   0              11m
cilium-operator-7779cbffcf-xpz2r   1/1     Running   1 (11m ago)    11m
cilium-plfks                       1/1     Running   0              11m
coredns-c996f6cfc-8rv4v            1/1     Running   0              11m
kube-vip-control-plane             1/1     Running   14 (11m ago)   12m

I found this after doing some Googling, I'm checking my network configuration

Does not look like a configuration issue on my router at the moment.
My route tables on a couple of my nodes in the cluster
worker-1
ubuntu@worker-1:~$ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         _gateway        <IP_ADDRESS>         UG    0      0        0 eth0
<IP_ADDRESS>       worker-1        <IP_ADDRESS>   UG    0      0        0 cilium_host
worker-1        <IP_ADDRESS>         <IP_ADDRESS> UH    0      0        0 cilium_host
<IP_ADDRESS>     <IP_ADDRESS>         <IP_ADDRESS> UH    0      0        0 lxce06f980a9564
<IP_ADDRESS>     <IP_ADDRESS>         <IP_ADDRESS> UH    0      0        0 lxc64bb53ac21b9
<IP_ADDRESS>     <IP_ADDRESS>         <IP_ADDRESS> UH    0      0        0 lxc_health

worker-2
ubuntu@worker-2:~$ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         _gateway        <IP_ADDRESS>         UG    0      0        0 eth0
<IP_ADDRESS>       worker-2        <IP_ADDRESS>   UG    0      0        0 cilium_host
worker-2        <IP_ADDRESS>         <IP_ADDRESS> UH    0      0        0 cilium_host
<IP_ADDRESS>      <IP_ADDRESS>         <IP_ADDRESS> UH    0      0        0 lxc_health
<IP_ADDRESS>      <IP_ADDRESS>         <IP_ADDRESS> UH    0      0        0 lxc0fcd46f68a97
<IP_ADDRESS>     <IP_ADDRESS>         <IP_ADDRESS> UH    0      0        0 lxc547e2c11c67e


[cpr] Update to 1.5.1

Fix #13337
Update cpr to the latest version 1.5.1
Note: No feature need to test

Cpr has been upgraded in #13558  ,This pull request will be closed


Fix code scanning alert - Potentially unsafe external link

Tracking issue for:

[x] https://github.com/Miccighel/Crowd_Frame/security/code-scanning/2

Added rel="noopener noreferrer" in search-engine-body-component


Retry publisher creation less frequently on Kafka authorization errors

Kafka Authorization errors aren't really intermittent and usually require some manual intervention outside of the app to fix - either by changing the resource your app is trying to access to something it has permissions for or correcting the ACLs on the broker side.  This change reduces the default retry time on authorization errors in the KafkaProducerActor from 3 seconds to 1 minute and exposes configuration settings to allow overriding the retry time if desired.

Codecov Report

Merging #17 (f2f38b3) into main (edaca7b) will increase coverage by 0.10%.
The diff coverage is 100.00%.

:exclamation: Current head f2f38b3 differs from pull request most recent head 10ff5f4. Consider uploading reports for the commit 10ff5f4 to get more accurate results

@@            Coverage Diff             @@
##             main      #17      +/-   ##
==========================================
+ Coverage   74.70%   74.80%   +0.10%
==========================================
  Files         129      129
  Lines        2182     2187       +5
  Branches       64       60       -4
==========================================
+ Hits         1630     1636       +6
+ Misses        552      551       -1

Impacted Files
Coverage Δ

.../surge/internal/kafka/KafkaProducerActorImpl.scala
88.79% <100.00%> (+0.23%)
:arrow_up:

...n/scala/surge/internal/streams/StreamManager.scala
77.21% <0.00%> (+1.26%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update edaca7b...10ff5f4. Read the comment docs.


[samples] sample todo app upgrade

Motivation
Show a more complete example of a DXOS app for demonstration purposes that shows off canonical DXOS SDK usage with React.
Finish Line

[x] #3916
[x] #3970
[ ] Update the Sample page on the Docs site
[ ] Update the sample app to take advantage of progressive multiplayer (blocked by #3845)
[ ] Add the tasks app as a template (blocked by #3914)

Deps/Blockers

[ ] Progressive multi-player
[x] Shell panels

Progressive multiplayer for the new todo app is less-than-ideal until we introduce something like #3968 without introducing an entire space-picker UI, which is not desired.

I should break this into more than one issue. Most of the work was done in week 34.

Just noting that as of today I'm unblocked on Progressive Multiplayer stuff. Have a go-forward strategy with the existing APIs.

#4273


Trie in Symspell

Hi @wolfgarbe ,
Greetings
Currently, we use Dictionary/Map for holding data?
can we move it to a more optimized trie?
similar to your repo  PruningRadixTrie (https://github.com/wolfgarbe/PruningRadixTrie)

Of course we could use a Trie instead of a Dictionary/Map for holding the deletes. But I don't think there will be a benefit, neither in lookup speed nor in memory consumption. But I will do a quick experiment to verify that.

Test results: 500.000 terms from frequency_dictionary_en_500_000.txt resulting in 12.577.734 deletes:
Radix trie<string,long>:        1800 MB
Dictionary<string,long>:       1366 MB
Dictionary<uint hash, long>:  600 MB
The dictionary is more space efficient than the trie. For spelling suggestion I use a trie for prefix  range search. But for SymSpell this is not required, I just need a simple key lookup.
Of course there are ways to optimize the space efficiency of tries, e.g. Succinct Tries . But they are slow.


Roll Flutter (stable) from e99c9c7cd9f6 to 52b3dc25f647 (4 revisions)

https://github.com/flutter/flutter/compare/e99c9c7cd9f6...52b3dc25f647
If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-stable-plugins
Please CC<EMAIL_ADDRESS>on the revert to ensure that a human
is aware of the problem.
To file a bug in Flutter (stable): https://github.com/flutter/flutter/issues/new/choose
To file a bug in Plugins: https://github.com/flutter/flutter/issues/new/choose
To report a problem with the AutoRoller itself, please file a bug:
https://bugs.chromium.org/p/skia/issues/entry?template=Autoroller+Bug
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

Commit queue failed; closing this roll.


Flip exit codes to the correct ones

https://github.com/purescript-spec/purescript-spec/commit/da88e3cd14df721c93b7f48b65ce896e89e1df5e#diff-6284d4c18f32617e31426063da6476aeR170 introduced a bug: it makes the test suite exit with a success exit code on failure, and with a failure exit code on success.

Thanks -- we should've caught that :+1:
I'll release 3.1.1

Thanks for the fast response @felixmulder !

we should've caught that

FYI: Seems that Travis isn't working, because it's set up for the old repo name (owickstrom/purescript-spec). https://travis-ci.org/purescript-spec/purescript-spec gives me 404, and https://travis-ci.org/owickstrom/purescript-spec/builds shows latest build from 4 months ago.

Thanks! I need the org to allow access from travis, the appropriate people should've gotten pinged now :)


Added more elements .to

https://github.com/ioam/holoviews/issues/2918
Test notebook
https://anaconda.org/ahuang11/test_hv_ds_to/notebook

Looks good to me, ready to merge?


Ori/mgmt 1220 bm-inventory should invoke the free-ip tool on the agent and validate the VIPs provided by the user

Where is the part of the validation for the VIPs provided by the user?

In inventory.go


found 1 high severity vulnerability

Description
High            Denial of Service
Package         css-what
Patched in      >=5.0.1
Dependency of   google-play-scraper
Path            google-play-scraper > cheerio > css-select > css-what
More info       https://npmjs.com/advisories/1754
found 1 high severity vulnerability in 1384 scanned packages
1 vulnerability requires manual review. See the full report for details.
pls help

don't worry about this. That package isn't maintained iirc.


UI customisation?

Currently, I am customising our sidekiq UI by placing custom .erb files in app/views/sidekiq.
Is there a similar method by which I can customise sidekiq-status UI?  I've found that it seems to disregard files in either app/views/sidekiq or app/views/sidekiq-status.

I would imagine this would be possible if you were to redefine the Sidekiq::Status::Web::VIEW_PATH constant or by overriding the #sidekiq_status_template helper.


finding sentence limits

While it seems quite reasonable to cut on naro + shad, there are so many edge cases where the proper cut is difficult to find that it would be helpful to have some code doing that in pybo. i'm thinking of the various types of punctuation that are at the beginning of sentences and not at the end, etc. I have some code that does it here plus some tests here (only part of this code is interesting, namely the getAllBreakingCharsIndexes) but it could certainly improved (and better documented, mea culpa!). This could then be combined with some euristics to find actual sentences, not just shunits

out of the box, here is what pybo's preprocessor does:
In the first line of output, ('TEXT', 0, 4), 0 stands for the starting index of the chunk, 4 for the length of the chunk.
I think it covers some part of the behaviour you expect, but not everything...
    from pybo import Chunks

    strs = ["གཅིག། གཉིས",
            "གཅིག། །གཉིས",
            "གཅིག༑ གཉིས",
            "གཅིག ༑ གཉིས",
            "༑ གཉིས",
            "གཅིག ༑གཉིས",
            "གཅིག\u0f14གཉིས",
            "གཅིག\u0f7f གཉིས",
            "གཅིག།། །། ༆ ། །གཉིས",
            "གཅིག༎ ༎༆ ༎གཉིས",
            "སྤྱི་ལོ་༢༠༡༧ ཟླ་༡ ཚེས་༡༤ ཉིན་ལ་བྲིས་པ་དགེ",
            "བཀྲིས་ ༼བཀྲ་ཤིས༽ ངའི་གྲོགས་པོ་རེད།",
            "ག གི གྲ ཀ ཤ པ མ",
            "༄༅། ཀ༌ཀོ་ཀཿཀ࿒ཀ་ཀ ཀ་རང་ཀ།་"]
    for s in strs:
        c = Chunks(s)
        chunks = c.make_chunks()
        r = c.get_markers(chunks)
        readable = c.get_readable(chunks)
        print(r)
        print(readable)
        print()

output:
[('TEXT', 0, 4), ('PUNCT', 4, 2), ('TEXT', 6, 4)]
[('TEXT', 'གཅིག'), ('PUNCT', '། '), ('TEXT', 'གཉིས')]

[('TEXT', 0, 4), ('PUNCT', 4, 3), ('TEXT', 7, 4)]
[('TEXT', 'གཅིག'), ('PUNCT', '། །'), ('TEXT', 'གཉིས')]

[('TEXT', 0, 4), ('PUNCT', 4, 2), ('TEXT', 6, 4)]
[('TEXT', 'གཅིག'), ('PUNCT', '༑ '), ('TEXT', 'གཉིས')]

[('TEXT', 0, 4), ('PUNCT', 4, 3), ('TEXT', 7, 4)]
[('TEXT', 'གཅིག'), ('PUNCT', ' ༑ '), ('TEXT', 'གཉིས')]

[('PUNCT', 0, 2), ('TEXT', 2, 4)]
[('PUNCT', '༑ '), ('TEXT', 'གཉིས')]

[('TEXT', 0, 4), ('PUNCT', 4, 2), ('TEXT', 6, 4)]
[('TEXT', 'གཅིག'), ('PUNCT', ' ༑'), ('TEXT', 'གཉིས')]

[('TEXT', 0, 4), ('PUNCT', 4, 1), ('TEXT', 5, 4)]
[('TEXT', 'གཅིག'), ('PUNCT', '༔'), ('TEXT', 'གཉིས')]

[('TEXT', 0, 5), ('TEXT', 5, 5)]
[('TEXT', 'གཅིགཿ'), ('TEXT', ' གཉིས')]

[('TEXT', 0, 4), ('PUNCT', 4, 11), ('TEXT', 15, 4)]
[('TEXT', 'གཅིག'), ('PUNCT', '།། །། ༆ ། །'), ('TEXT', 'གཉིས')]

[('TEXT', 0, 4), ('PUNCT', 4, 6), ('TEXT', 10, 4)]
[('TEXT', 'གཅིག'), ('PUNCT', '༎ ༎༆ ༎'), ('TEXT', 'གཉིས')]

[('TEXT', 0, 5), ('TEXT', 5, 3), ('NUM', 8, 5), ('TEXT', 13, 3), ('NUM', 16, 2), ('TEXT', 18, 4), ('NUM', 22, 3), ('TEXT', 25, 4), ('TEXT', 29, 2), ('TEXT', 31, 5), ('TEXT', 36, 2), ('TEXT', 38, 3)]
[('TEXT', 'སྤྱི་'), ('TEXT', 'ལོ་'), ('NUM', '༢༠༡༧ '), ('TEXT', 'ཟླ་'), ('NUM', '༡ '), ('TEXT', 'ཚེས་'), ('NUM', '༡༤ '), ('TEXT', 'ཉིན་'), ('TEXT', 'ལ་'), ('TEXT', 'བྲིས་'), ('TEXT', 'པ་'), ('TEXT', 'དགེ')]

[('TEXT', 0, 6), ('PUNCT', 6, 2), ('TEXT', 8, 4), ('TEXT', 12, 3), ('PUNCT', 15, 2), ('TEXT', 17, 4), ('TEXT', 21, 6), ('TEXT', 27, 3), ('TEXT', 30, 3), ('PUNCT', 33, 1)]
[('TEXT', 'བཀྲིས་'), ('PUNCT', ' ༼'), ('TEXT', 'བཀྲ་'), ('TEXT', 'ཤིས'), ('PUNCT', '༽ '), ('TEXT', 'ངའི་'), ('TEXT', 'གྲོགས་'), ('TEXT', 'པོ་'), ('TEXT', 'རེད'), ('PUNCT', '།')]

[('TEXT', 0, 15)]
[('TEXT', 'ག གི གྲ ཀ ཤ པ མ')]

[('PUNCT', 0, 4), ('TEXT', 4, 2), ('TEXT', 6, 3), ('TEXT', 9, 2), ('TEXT', 11, 1), ('SYM', 12, 1), ('TEXT', 13, 2), ('TEXT', 15, 4), ('TEXT', 19, 3), ('TEXT', 22, 1), ('PUNCT', 23, 2)]
[('PUNCT', '༄༅། '), ('TEXT', 'ཀ༌'), ('TEXT', 'ཀོ་'), ('TEXT', 'ཀཿ'), ('TEXT', 'ཀ'), ('SYM', '࿒'), ('TEXT', 'ཀ་'), ('TEXT', 'ཀ ཀ་'), ('TEXT', 'རང་'), ('TEXT', 'ཀ'), ('PUNCT', '།་')]

thanks! Indeed, if we take
[('TEXT', 'གཅིག'), ('PUNCT', '།། །། ༆ ། །'), ('TEXT', 'གཉིས')]

for instance, ideally it would do something like
[('TEXT', 'གཅིག'), ('PUNCT', '།། །། ), ('PUNCT', '༆ ། །'), ('TEXT', 'གཉིས')]

so that we could tag that a sentence break happens between tokens 2 and 3... I'm not sure what the API would be to tag the sentence limits... maybe some sort of tags parallel to the POS tags?

the cases of ག གི གྲ ཀ ཤ པ མ are not handled yet (the issue is still open).
I think the best will be to work on a list of Token objects. We will be able to use all the info gathered during tokenization. Every Token object has a Token._ attribute containing an empty dict that can receive any custom data. That sounds like a good usecase.
Ideally, we would have something similar to the SplitAffixed class (that splits affixed particles into a different token) calling the TokenSplit class at some point to split the punct tokens into two.
Then, in a second pass over the list of tokens, we could use matchers (like in this test), providing a match_query to identify where to insert the end-of-sentence info, and a replace to add the end-of-sentence info in the right token.
The best would be that the code to split in sentences is run first, so we only process the punctuations that are near the end-of-sentence already found.
That might be the best integration in pybo I can currently think of... Although it might sound overkill for what you are trying to acheive.

@ngawangtrinley says:

shad always belong to what is on their left
all yigo types belong to the text on their right
separators like drulshad belong to what is on their right

in case of a yigo+shad+text, the shad will belong to the yigo, which in turn will belong to the text
Does that sound like what you have implemented?

in short yes, I also consulted NT for the code and nagged him with many boring edge cases... I think what's implemented in Java is pretty good, unfortunately it's not very fresh in my mind...

ok, then it's perfect.
When I have more time, I'll port your implementation in python.

thanks a lot! Finding the utterances is the first step before segmentation in the workflow to create the ACTIB corpus (and probably others) so I think having a way to do that with Pybo will be helpful!

finding utterances is the same as tokenizing in sentences. This can't be done with the punctuation alone (correct me if I'm wrong).
That is why I tried to do with the sentencify code that is here
What it does is to first tokenize, then use heuristics to identify sentences. punctuation plays a role in it, but there are many other things involved.
As for the next function, paragraphify, I group sentences from the sentencify until I reach a certain size.
What your code seems to be doing is to identify where does the punctuation belong, either to the text preceding it or to the one following it. Am I right?

yes, properly finding utterances involves more than punctuation and this script is a very good start, it's just that ideally it would be able to give the index of the exact right place (right of the shad, left of the yigo, etc.). Thanks a lot for considering it!

also, just a small detail, for the sake of completeness in the sentencify script, some titles in the Tengyur end with བཞུགས (without the སོ), perhaps other sentences too. maybe it could be added to the ending_word list?

There is also another bug that @10zinten reported to me (still waiting for test data): the སྟེ་ ཏེ་ དེ་ particles often seems to be used in the middle of sentences. Gar Tsering and others complained that they had partial sentences preventing them from correctly tag sentences in LightTag. On the other hand, removing these markers ended up in huge sentences, which they anyhow seemed to have prefered.


Fixing repr of iset

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 1 (0.0%)  changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.004%) to 84.873%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pynapple/core/interval_set.py
0
1
0.0%

Totals

Change from base Build<PHONE_NUMBER>:
-0.004%

Covered Lines:
2345

Relevant Lines:
2693

💛  - Coveralls


Symfony Plugin Configuration

Hi,
great work - thx for sharing.
I am using PHPStorm and use the Symfony Plugin, but I don't know how to configure the plugin when Symfony is not in root dir of the project, but in a sub dir.

Greetings,
Dome

Hello @thedomeffm!
I've never use the Symfony Plugin, so I dunno how to do that 🤔


tried docs example for foreign key, doesn't work.

followed the docs for has manyhttp://gorm.io/docs/has_many.html for sqlilite
package main

import (
	"fmt"

	"github.com/jinzhu/gorm"
	_ "github.com/mattn/go-sqlite3"
)

type User struct {
	Id          string
	Name        string
	CreditCards []CreditCard `gorm:"FOREIGNKEY:user_id;ASSOCIATION_FOREIGNKEY:id"`
}

type CreditCard struct {
	Id     int
	Number string
	UserID string
}

func main() {
	db, _ := gorm.Open("sqlite3", "testdb.db")
	db.Exec("PRAGMA foreign_keys = ON")
	db.LogMode(true)
	db.AutoMigrate(&User{}, &CreditCard{})
	db.Model(&CreditCard{}).AddForeignKey("id", "users(id)", "RESTRICT", "RESTRICT")

and I look at it in db browser and I can not see the table has reference int !
however if I add // sql:"type:text REFERENCES users(id)" to make it look like
type CreditCard struct {
	Id     int
	Number string
	UserID string  `sql:"type:text REFERENCES users(id)"`
}

it will create the foreign key.
it is not clear, am I supposed to use sql struct tags?

hi @medyagh
there is an error in your code, you want to map the user_id as foreign_key to User table, so you need to map the UserID not the ID
db.Model(&CreditCard{}).AddForeignKey("user_id", "users(id)", "RESTRICT", "RESTRICT")
instead of
db.Model(&CreditCard{}).AddForeignKey("id", "users(id)", "RESTRICT", "RESTRICT")


add public src

Metric
Value

alert_status
:white_check_mark: OK

bugs
0

code_smells
3

reliability_rating
A (0 Bugs)Note: A being best, E being worst

security_rating
A (0 Vulnerabilities)Note: A being best, E being worst

vulnerabilities
0


No manifest requested for Youtube non-live

For live content I see calls made to manifest.googlevideo.com. However, for non-live content no such calls are made. Can you please tell me how does youtube-dl downloads manifest for non-live content? Plus, how is the manifest downloaded in a regular browser, given that there are no calls to manifest.googlevideo.com in the browser network trace.

Some formats are served via DASH, which are available as manifests. Browsers may pick non-DASH formats so no manifests are downloaded.

Can you please elaborate or point me where to look at on what other non-DASH formats does Youtube uses? I was under the impression that most of the adaptive streaming protocols such as MPEG-DASH and HLS require a manifest to be downloaded. Thanks!

In general, YouTube uses monolithic files for non-DASH formats in non-live streams. You can get their URLs with --get-url via youtube-dl. On the webpage it's usually stored in a variable url_encoded_fmt_stream_map.


Does not build with --no-bin-links

Does not build with npm install --no-bin-links

@fab1an can you post the error you're getting?


'Question: [scrollEnd detection]'

Is it possible to detect when a user lets go of the trackpad in a ScrollView? (I believe this is what Cocoa calls touchesEndedWithEvent)
Seems like an important event to detect for more gesture-based apps, but I haven't found a way to make it work (onScrollEnd and onMomentumScrollEnd don't seem to fire)?
Thanks!

Hi.   No, we never implemented those events in react-native-macos.   I think it would be relatively easy to implement by listening for NSScrollViewDidLiveScrollNotification in RCTScrollView.m.   We're accepting contributions if you would like to implement it.   Thanks!


sift: fix build for darwin

Motivation for this change
This package can build on darwin but suffers from the same problem described here: https://github.com/NixOS/nixpkgs/issues/18131#issuecomment-252252209
Before:
$ nix-build -I nixpkgs=. -E 'with import <nixpkgs> {}; callPackage pkgs/tools/text/sift {}'
these derivations will be built:
  /nix/store/7ggs2rii2jz7viwcfsgd56ibw9i7s2fj-sift-0.8.0.drv
building path(s) ‘/nix/store/p1g7jzj6rsdxb3xay7612ykqi7bpnp8j-sift-0.8.0’, ‘/nix/store/xzfa87j2gb1fhwypd3hblgm4b3gn3m4w-sift-0.8.0-bin’
unpacking sources
unpacking source archive /nix/store/5ngj6nj90hjjsxpz3ynnkyzw5kzzhxmj-sift-v0.8.0-src
source root is sift-v0.8.0-src
patching sources
configuring
grep: Invalid range end
unpacking source archive /nix/store/ggc90hhc25jpymh4hx45qq7nldnyh7hb-crypto-575fdbe
unpacking source archive /nix/store/gd7l7hfks2011r91xr01bzm475bsgbpi-go-flags-4bcbad3
unpacking source archive /nix/store/zg6y9v9gkxyjb72rq0zranxf1zbms722-go-nbreader-7cef48d
building
github.com/svent/sift/gitignore
github.com/svent/go-flags
golang.org/x/crypto/ssh/terminal
github.com/svent/go-nbreader
github.com/svent/sift
# github.com/svent/sift
ld: warning: /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation, ignoring unexpected dylib file
installing
/private/var/folders/jt/qgrx8lp10ylbdlnb93t54r6c0000gn/T/nix-build-sift-0.8.0.drv-0/go /private/var/folders/jt/qgrx8lp10ylbdlnb93t54r6c0000gn/T/nix-build-sift-0.8.0.drv-0
/private/var/folders/jt/qgrx8lp10ylbdlnb93t54r6c0000gn/T/nix-build-sift-0.8.0.drv-0
post-installation fixup
stripping (with flags -S) in /nix/store/xzfa87j2gb1fhwypd3hblgm4b3gn3m4w-sift-0.8.0-bin/bin
patching script interpreter paths in /nix/store/xzfa87j2gb1fhwypd3hblgm4b3gn3m4w-sift-0.8.0-bin
patching script interpreter paths in /nix/store/p1g7jzj6rsdxb3xay7612ykqi7bpnp8j-sift-0.8.0
cycle detected in the references of ‘/nix/store/p1g7jzj6rsdxb3xay7612ykqi7bpnp8j-sift-0.8.0’
error: build of ‘/nix/store/7ggs2rii2jz7viwcfsgd56ibw9i7s2fj-sift-0.8.0.drv’ failed
nix-build -I nixpkgs=. -E   3.15s user 2.03s system 36% cpu 14.235 total

After:
$ nix-build -I nixpkgs=. -E 'with import <nixpkgs> {}; callPackage pkgs/tools/text/sift {}'
...
/nix/store/f8i8sqqs3k23pvviz7bb20l9356jzqbz-sift-0.8.0-bin
$ ./result-bin/bin/sift
Error: No pattern given. Try 'sift --help' for more information.

The problem in https://github.com/NixOS/nixpkgs/pull/16820 also seems to have gone away.
Things done

[ ] Tested using sandboxing
(nix.useSandbox on NixOS,
or option build-use-sandbox in nix.conf
on non-NixOS)
Built on platform(s)

[ ] NixOS
[x] macOS
[ ] Linux

[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

Nice!  I just tested this diff on my Linux machine to sanity check that it doesn't change anything there.  LGTM


Update project's Readme files

Add information regarding project building, prerequisites and links on where to find more relevant documentation.

Once there will be a showcase website running - include it as a badge.


[FEATURE REQUEST] Keybindings for moving blocks

It would be quite nice to add keybinding to move blocks using < or > in VISUAL MODE.
I believe vim has this shortcut.

I think the default Tab and Shift+Tab is better option. Actually I only implemented few Vim-Like shortcuts. Most of VIM ability can not be implemented in Logseq.

I'd like this feature as well :)
Of course, Tab can do the same, but that messes with my vim memory.


Make touch-time and wait-time proportional

Touch time (box widths) should be proportional to the time indicated. Similarly wait time (lines length between boxes) could be proportional to the time indicated.
Or perhaps a toggle that allows the time-proportional render to be on/off?

Hmmm. That’s a good idea. I think a view that lets you toggle would be good so you can scroll easily. I’ll add it to the feature list. Trying to fix drag and drop with the new control and I should have parity with the current but with more customization options.
On Feb 11, 2021, 8:25 AM -0600, Paul Hammant<EMAIL_ADDRESS>wrote:

Touch time (box widths) should be proportional to the time indicated. Similarly wait time (lines length between boxes) could be proportional to the time indicated.
Or perhaps a toggle that allows the time-proportional render to be on/off?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Touch time (box widths) should be proportional to the time indicated. Similarly wait time (lines length between boxes) could be proportional to the time indicated.
Or perhaps a toggle that allows the time-proportional render to be on/off?

Wait time is working. I need to implement the toggle. I'm not sure I can make the nodes different widths with the library I'm using, but I'm still looking into it.


Integration Issue with Helia: Custom Service Configuration Disables Peer Discovery in Browser Environment

Description:
I am encountering a significant issue in a pure browser environment when initializing IPFS nodes with Helia, which affects the functionality of OrbitDB. Even a minimal custom services configuration in Helia (e.g., libp2p: { services: {} }) seems to disable the node's ability to automatically discover peer nodes. However, removing the services option entirely restores this functionality. Given OrbitDB's reliance on specific libp2p configurations involving the services option (especially for pubsub with gossipsub), it is crucial to enable services without hindering peer discovery?
This issue has already been documented in Helia's repository (see issue #532), but I am raising it here as well because of the dependency OrbitDB has on this functionality. This problem could potentially affect many other OrbitDB users.
Steps to Reproduce:

Initialize a Helia-based IPFS node in a clean browser environment with the following configuration:

// index.js
import { IDBBlockstore } from 'blockstore-idb';
import { IDBDatastore } from 'datastore-idb';
import { createHelia } from 'helia';

const instantiateHeliaNode = async () => {
  const datastore = new IDBDatastore('/datastore2');
  const blockstore = new IDBBlockstore('/blockstore2');
  await datastore.open();
  await blockstore.open();
  const heliaInstance = await createHelia({
    libp2p: {
      services: {}
    },
    peerStore: {
      persistence: false,
      threshold: 5
    },
    keychain: { pass: 'very-strong-password' },
    datastore: datastore,
    blockstore: blockstore
  });

  return heliaInstance;
};

document.addEventListener("DOMContentLoaded", async () => {
  const helia = window.helia = await instantiateHeliaNode();

  // monitor
  console.log('Helia node created with ID:', helia.libp2p.peerId.toString())
  setInterval(async () => {
      const peers = await helia.libp2p.peerStore.all();
      console.log(`Total number of store peers: ${peers.length}`);
      console.log('Connected peers count:', helia.libp2p.getPeers().length);
  }, 3000);
});

<!DOCTYPE html>

  <meta charset="UTF-8" /> <meta http-equiv="X-UA-Compatible" content="IE=edge" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" />

IPFS in the Browser via Helia

Observe the node's inability to discover peers (It may take several minutes or more).
Remove or delete the services configuration and reinitialize the node in a clean browser environment (e.g., a new incognito window):

// In the index.js file:

  const heliaInstance = await createHelia({
    // libp2p: {
    //   services: {}
    // },
    peerStore: {
      persistence: false,
      threshold: 5
    },
    keychain: {
      pass: 'very-strong-password'
    },
    datastore: datastore,
    blockstore: blockstore
  });

Observe that the node can now discover peers.

Expected Behavior:
The node should be able to discover peers automatically, even when the services option is used, provided it is configured correctly or even if it is empty.
Actual Behavior:
The node fails to discover peers when the services option is included in the configuration. Removing this option restores peer discovery functionality.
Environment:

Browser Version: Google Chrome 123.0.6312, Microsoft Edge 123.2420
Operating System: Windows 2022
Helia Version: 4.2.1
Libp2p Version: 1.3.3

Additional Information:
Despite reviewing the documentation on service-related configurations (libp2p interface documentation and source code), I am still unsure how to resolve this issue.
Use the Service option to start Helia:

Start Helia without using the Service option:

I would definitely recommend reaching out to Helia/Libp2p via their various channels. I think that the more information they have to work with the easier they can make out-of-the-box connectivity and "iron out" many of these issues. The browser has always been difficult to work with especially due to its networking limitations. Hopefully, this will be resolved as more decentralized apps are built within the browser environment.
OrbitDB is working closely with Libp2p to improve connectivity, but, with limited resources, it is not always possible to address every connection issue.
Please feel free to share any more information you discover either within this issue or via the other OrbitDB channels.

Thank you for the prompt response and your suggestions.
I appreciate the insight into the challenges of working with browser environments and the limitations they impose on networking.
Based on your advice, I will reach out to the Helia and Libp2p teams through their respective channels to provide them with as much information as possible.
I will keep this issue updated with any new information or progress from my discussions with Helia and Libp2p.

Closing for now but feel free to re-open if you have any more information you would like to share.


[第三組] Refactor/error handling

Purpose
This pull request introduces significant updates to the codebase, including enhancements to the CI workflow, the addition of a new crawler implementation, and improvements to the exception handling and LLM client modules.
Exception Handling:

Introduced a base class APIException in backend/src/exception_handler/base.py for standardized API exception handling.
Added specific exception classes in backend/src/exception_handler/exception.py for various error scenarios, such as NoResourceFoundException, AuthenticationExceptionBase, and its subclasses.

Configuration Updates:

Updated SENTRY_DSN in backend/src/config.py to use a new DSN for error tracking.

Please fix your conflict so I can merge!
Also your CI didn't pass.

這次更新顯示了卓越的技術實力，從 CI 工作流程的強化到新增爬蟲實作及改進例外處理與 LLM 客戶端模組，都展現了對系統穩定性與可擴展性的高度重視。標準化的 API 例外處理和精細的錯誤追蹤設定，為系統帶來了更高的健壯性與可靠性，值得讚賞。

從 CI 工作流程的強化到新增爬蟲實作及改進例外處理與 LLM 客戶端模組，都展現了對系統穩定性與可擴展性的高度重視。為系統帶來了更高的健壯性與可靠性。但...code跟coffee高度重疊欸，而且有conflict。然後github上的ci test也沒過啊，新增了很多函式但感覺沒用到太多 重複性很高。

此次更新強調用logger和錯誤處理來記錄問題以增加系統穩定性，但存在CI 測試未通過及高函式重複性等問題，需進一步優化與整合。

老兄的Exception做的很好，可以直觀看到錯誤內容，但你CI沒過ㄟQQ

Overall

Error Handling Addition: It's great to see that error handling has been introduced in this PR. Proper error handling is crucial for making the application more robust and user-friendly.

Logging Implementation: The introduction of log files (app.log and app_rotating.log) is a step forward for debugging and monitoring. However, ensure that sensitive information is not logged to prevent security risks.

Abstract Base Classes: The use of an abstract base class (NewsCrawlerBase) for crawlers is a good design practice. It sets a clear structure for future crawlers and enforces method implementation in subclasses.

Specific Feedback:

Domain Validation:

The validate_and_parse method ensures that only valid URLs are processed, which is good for security. It would be beneficial to add unit tests to cover various scenarios for this validation to ensure reliability.

Handling NoneType Exceptions:

The log message indicates an error related to NoneType. It would be prudent to include checks that prevent this from happening upstream in your application logic, especially before trying to access attributes like hashed_password.

Usage of AnyHttpUrl:

The usage of AnyHttpUrl in Pydantic models is a great way to enforce URL validity at the schema level. However, ensure that necessary validation logic is executed when you receive these URLs, particularly if they come from external inputs.

Sentry DSN Change:

Make sure that the change in the Sentry DSN is documented or discussed within your team. It would be beneficial to have a configuration management solution for such sensitive configurations instead of hardcoding them.

Typing and PEP 484 Compliance:

The typing hints used here are appropriate, though make sure that they adhere strictly to PEP 484. Consistent adherence to typing can greatly enhance code readability and maintainability.

Test Coverage:

Ensure that there are sufficient tests covering the new error handling code. It would be valuable to see both unit and integration tests that validate the correctness of expected behaviors in various error scenarios.

Suggestions for Improvement:

Consider implementing a custom logger setup for different log levels (INFO, WARNING, ERROR) to improve the granularity of log messages.
If the integration tests cover external API interactions, make sure to use mocks to avoid hitting the actual endpoints during testing.
It would be beneficial to peer review the database operations for potential race conditions or handling of concurrent writes.

Overall, this PR significantly enhances the handling of errors within the application. With the implementation of foundational elements such as logging and structured error handling, it lays a strong groundwork for future improvements and stability.

CI沒過但其他頗好

優點

新增的 Try-Except 功能：

異常捕捉範圍明確：針對特定的異常類型進行處理，如 SQLAlchemyError、ExpiredSignatureError 等，有助於提高異常處理的精確性。
詳細的日誌記錄：使用 logger.error 記錄詳細的異常訊息，有助於後續的錯誤追蹤和分析。
自訂異常類別：定義了多個自訂異常類別，如 NoResourceFoundException、ArticleNotFoundException 等，這些異常類別名稱直觀，能夠清楚地表達異常的具體情況。

改進建議

Try-Except 內是否處理太多事情：

細化 Try-Except 塊：在某些情況下，Try-Except 塊內處理的事情過多，建議將其細化。例如在 router.py 中的 search_news 函數內，可以將不同的步驟分別放在各自的 Try-Except 塊中，以提高可讀性和可維護性。

try:
    response = requests.get(news["titleLink"])
    soup = BeautifulSoup(response.text, "html.parser")
except Exception as e:
    logger.error(f"Error fetching news content: {str(e)}")
    raise HTTPException(status_code=500, detail="Error fetching news content")

try:
    title = soup.find("h1", class_="article-content__title").text
    time = soup.find("time", class_="article-content__time").text
    content_section = soup.find("section", class_="article-content__editor")
    paragraphs = [p.text for p in content_section.find_all("p") if p.text.strip() != "" and "▪" not in p.text]
except Exception as e:
    logger.error(f"Error parsing news content: {str(e)}")
    raise HTTPException(status_code=500, detail="Error parsing news content")

總結來說，這次改動在異常處理、日誌記錄和自訂異常的設計上做得很好，但可以通過進一步細化 Try-Except 塊、增加更多的日誌記錄和統一異常處理方式來進一步提升程式碼的可讀性和可維護性。

基本上改動增加了系統穩定性，可惜CI 測試未通過，
建議可以優化錯誤處理及日誌分級，避免高重複性函式；
細化異常類型並統一處理方式，提升穩定性與可維護性。


Broken links: ticket:1234

Links of the form ticket:1234 or [ticket:1234 link description] were not transformed to Markdown, nor mapped to the new GitHub URLs.
If anyone finds this repository and wants to run it on another project, this is a bug that was not fixed.

Yes. this is custom Trac wiki syntax.
I think that for now this is low priority.
Also, if someone is worried about it, an update can be executed over the existing comments and edit the comments to add the links.
But I think is a lot of work and trouble for little reward


@cronner-period does not actually accept all possible values for strtotime()

This https://github.com/stekycz/Cronner/blob/master/Cronner/Tasks/Parser.php#L49 causes errors for strtotime acceptable values that does not start with + sign like "first day of this month" and so on...

Thank you for report! What would be a solution from your point of view? I think I should update documentation only because period needs something with "+" sign.

Yes, I'd update documentation and maybe the error message as well. Says that the period is not valid if actually is, is confusing :)

Consider it as solved :-)


Home option and Sign up option in Header

What's the issue
The home and sign up option in header are not working
Solution
I will correct and fix the error in the header and the links will work properly.
Please assign me this task under GSSOC 23

Please assign me this task under GSSOC 23


training results are random with same seed set in caret

A number of methods in caret_6.0-58 give slightly different results. Setting the seed should result  in  the same RMSE and R2 (80 methods in caret do that). Exceptions are:
treebag
gbm
bdk
evtree
elm
xyf
parRF
Example code
# load caret and DT the cars data set
require(caret); require(DT);  require(mlbench);
library(AppliedPredictiveModeling)
data(solubility)

# load the data and coerce into single frame (legacy)
training_data = data.frame(solTrainX,solTrainY)[1:80,]
testing_data = data.frame(solTestX,solTestY)

# just rename columsn to stay conform with style below
colnames(training_data)[colnames(training_data) == 'solTrainY'] <- 'y'
colnames(testing_data)[colnames(testing_data) == 'solTestY'] <- 'y'

# all the training data (just named x and y)
y <- training_data$y
x <- training_data[, -ncol(training_data)]

# load all libraries
library(doParallel); cl <- makeCluster(8); registerDoParallel(cl)

# RMSE and R2 results should be the same, three times see knn
set.seed(123); result <- train(x,y,"knn"); getTrainPerf(result)
set.seed(123); result <- train(x,y,"knn"); getTrainPerf(result)
set.seed(123); result <- train(x,y,"knn"); getTrainPerf(result)

# Now the methods that give random results
set.seed(123); result <- train(x,y,"treebag"); getTrainPerf(result)
set.seed(123); result <- train(x,y,"treebag"); getTrainPerf(result)
set.seed(123); result <- train(x,y,"treebag"); getTrainPerf(result)

set.seed(123); result <- train(x,y,"gbm"); getTrainPerf(result)
set.seed(123); result <- train(x,y,"gbm"); getTrainPerf(result)
set.seed(123); result <- train(x,y,"gbm"); getTrainPerf(result)

set.seed(123); result <- train(x,y,"bdk"); getTrainPerf(result)
set.seed(123); result <- train(x,y,"bdk"); getTrainPerf(result)
set.seed(123); result <- train(x,y,"bdk"); getTrainPerf(result)

set.seed(123); result <- train(x,y,"evtree"); getTrainPerf(result)
set.seed(123); result <- train(x,y,"evtree"); getTrainPerf(result)
set.seed(123); result <- train(x,y,"evtree"); getTrainPerf(result)

# elm is the worst offender
set.seed(123); result <- train(x,y,"elm"); getTrainPerf(result)
set.seed(123); result <- train(x,y,"elm"); getTrainPerf(result)
set.seed(123); result <- train(x,y,"elm"); getTrainPerf(result)

set.seed(123); result <- train(x,y,"xyf"); getTrainPerf(result)
set.seed(123); result <- train(x,y,"xyf"); getTrainPerf(result)
set.seed(123); result <- train(x,y,"xyf"); getTrainPerf(result)

set.seed(123); result <- train(x,y,"parRF"); getTrainPerf(result)
set.seed(123); result <- train(x,y,"parRF"); getTrainPerf(result)
set.seed(123); result <- train(x,y,"parRF"); getTrainPerf(result)

# and some working results again
set.seed(123); result <- train(x,y,"rvmLinear"); getTrainPerf(result)
set.seed(123); result <- train(x,y,"rvmLinear"); getTrainPerf(result)
set.seed(123); result <- train(x,y,"rvmLinear"); getTrainPerf(result)

# stop the parallel processing and register sequential front-end
stopCluster(cl); registerDoSEQ();

Results (see correct and incorrect ones, for parRF increase train number to 160).
> # RMSE and R2 results should be the same, three times see knn
> set.seed(123); result <- train(x,y,"knn"); getTrainPerf(result)
  TrainRMSE TrainRsquared method
1 0.2901132    0.03771506    knn
> set.seed(123); result <- train(x,y,"knn"); getTrainPerf(result)
  TrainRMSE TrainRsquared method
1 0.2901132    0.03771506    knn
> set.seed(123); result <- train(x,y,"knn"); getTrainPerf(result)
  TrainRMSE TrainRsquared method
1 0.2901132    0.03771506    knn
>
> # Now the methods that give random results
> set.seed(123); result <- train(x,y,"treebag"); getTrainPerf(result)
  TrainRMSE TrainRsquared  method
1  0.295095    0.04499159 treebag
> set.seed(123); result <- train(x,y,"treebag"); getTrainPerf(result)
  TrainRMSE TrainRsquared  method
1 0.2935537    0.04086652 treebag
> set.seed(123); result <- train(x,y,"treebag"); getTrainPerf(result)
  TrainRMSE TrainRsquared  method
1 0.2925668    0.03898617 treebag
>
> set.seed(123); result <- train(x,y,"gbm"); getTrainPerf(result)
  TrainRMSE TrainRsquared method
1 0.2947946    0.02526446    gbm
> set.seed(123); result <- train(x,y,"gbm"); getTrainPerf(result)
  TrainRMSE TrainRsquared method
1 0.2953287    0.02494099    gbm
> set.seed(123); result <- train(x,y,"gbm"); getTrainPerf(result)
  TrainRMSE TrainRsquared method
1 0.2937742    0.02779757    gbm
> set.seed(123); result <- train(x,y,"bdk"); getTrainPerf(result)
  TrainRMSE TrainRsquared method
1 0.3475528     0.0539665    bdk
> set.seed(123); result <- train(x,y,"bdk"); getTrainPerf(result)
  TrainRMSE TrainRsquared method
1 0.3485172    0.05422798    bdk
> set.seed(123); result <- train(x,y,"bdk"); getTrainPerf(result)
  TrainRMSE TrainRsquared method
1 0.3497129    0.03251845    bdk
>
> set.seed(123); result <- train(x,y,"evtree"); getTrainPerf(result)
  TrainRMSE TrainRsquared method
1 0.2964285    0.02148905 evtree
> set.seed(123); result <- train(x,y,"evtree"); getTrainPerf(result)
  TrainRMSE TrainRsquared method
1 0.2970565    0.02178834 evtree
> set.seed(123); result <- train(x,y,"evtree"); getTrainPerf(result)
  TrainRMSE TrainRsquared method
1 0.2970565    0.02178834 evtree
>
> # elm is the worst offender
> set.seed(123); result <- train(x,y,"elm"); getTrainPerf(result)
Warning message:
In nominalTrainWorkflow(x = x, y = y, wts = weights, info = trainInfo,  :
  There were missing values in resampled performance measures.
  TrainRMSE TrainRsquared method
1 0.3061933    0.02887321    elm
> set.seed(123); result <- train(x,y,"elm"); getTrainPerf(result)
Warning message:
In nominalTrainWorkflow(x = x, y = y, wts = weights, info = trainInfo,  :
  There were missing values in resampled performance measures.
  TrainRMSE TrainRsquared method
1 0.3223984     0.0338859    elm
> set.seed(123); result <- train(x,y,"elm"); getTrainPerf(result)
Warning message:
In nominalTrainWorkflow(x = x, y = y, wts = weights, info = trainInfo,  :
  There were missing values in resampled performance measures.
  TrainRMSE TrainRsquared method
1 0.2954139    0.02630014    elm
>
>
> set.seed(123); result <- train(x,y,"xyf"); getTrainPerf(result)
  TrainRMSE TrainRsquared method
1 0.2798505     0.0172159    xyf
> set.seed(123); result <- train(x,y,"xyf"); getTrainPerf(result)
Warning message:
In nominalTrainWorkflow(x = x, y = y, wts = weights, info = trainInfo,  :
  There were missing values in resampled performance measures.
  TrainRMSE TrainRsquared method
1 0.2790053    0.01891035    xyf
> set.seed(123); result <- train(x,y,"xyf"); getTrainPerf(result)
  TrainRMSE TrainRsquared method
1  0.279014    0.01531723    xyf
> set.seed(123); result <- train(x,y,"parRF"); getTrainPerf(result)
Something is wrong; all the RMSE metric values are missing:
> set.seed(123); result <- train(x,y,"parRF"); getTrainPerf(result)
Something is wrong; all the RMSE metric values are missing:
> set.seed(123); result <- train(x,y,"parRF"); getTrainPerf(result)
Something is wrong; all the RMSE metric values are missing:

> # and some working results again
> set.seed(123); result <- train(x,y,"rvmLinear"); getTrainPerf(result)
  TrainRMSE TrainRsquared    method
1  1.066702    0.03438293 rvmLinear
> set.seed(123); result <- train(x,y,"rvmLinear"); getTrainPerf(result)
  TrainRMSE TrainRsquared    method
1  1.066702    0.03438293 rvmLinear
> set.seed(123); result <- train(x,y,"rvmLinear"); getTrainPerf(result)
  TrainRMSE TrainRsquared    method
1  1.066702    0.03438293 rvmLinear

see also
http://stackoverflow.com/questions/13403427/fully-reproducible-parallel-models-using-caret


Fail mapping with a bacteria genome

hi
I am working with a bacteria genome
How to modify parameters on ngmlr
Let me execute successfully it
time ./ngmlr -t 25 -r S1.fasta -q m160310_143125_42180_c100906132550000001823204104301691_s1_p0.b.subreads.fasta -o ngmlr_aTOb.sam
NextGenMap-LR 0.2.3 (build: Feb  8 2017 14:29:07, start: 2017-02-10.15:01:38)
Size of reference genome 4 Mbp (max. 68719 Mbp)
Allocating 2461172 (5001353) bytes for the reference.
BinRef length: 2461169ll (elapsed 0.051456)
0 reference sequences were skipped (length < 10).
Writing encoded reference to S1.fasta-enc.2.ngm
Writing to disk took 0.00s
Building reference table
Allocated 1 hashtable units (tableLocMax=2^32.000000, genomeSize=2^22.230912)
Building RefTable #0 (kmer length: 13, reference skip: 2)
        Number of k-mers: 67108865
        Counting kmers took 0.28s
        Average number of positions per prefix: 1.057819
        0 prefixes are ignored due to the frequency cutoff (1000)
        Index size: 335544325 byte (67108865 x 5)
        Generating index took 0.85s
        Allocating and initializing prefix Table took 0.00s
        Number of prefix positions is 1639771 (4)
        Size of RefTable is 6559084
        Number of repetitive k-mers ignored: 0
        Overall time for creating RefTable: 1.61s
Writing RefTable to S1.fasta-ht-13-2.2.ngm
Writing to disk took 0.46s
Processed: 273991 (0.81), R/S: 209.77, RL: 5431, Time: 1.00 4.00 87.00, Align: 0.99, 423, 0.96
Done **(223206 reads mapped (81.46%), 50785 reads not mapped, 312927 lines written)(elapsed: 21m, 170 r/s)**

thank you to help

Please look at the output:
"Done **(223206 reads mapped (81.46%),..." -> It run through.

@fritzsedlazeck
Thank you for your timely reply
It was my fault and didn't read thoroughly.
l have been used Sniffle to search  a structural variant from the result of both BWA-MEM and NGM-LR
We're sorry for the inconvenience.
I appreciate your help

Dont worry happens to everyone sooner or later.
Cheers


Is possible to generate same result of pytorch stft

Thanks for great work. I want to use this library to mimic pytorch STFT, so that given same signal( wav data ), generate same output. I used test.cpp with same wave file: data/male_1.wav. The output is so different( which I think expected, because parameters even different, like window length and shift length).
For pytorch STFT, code and parameters as following
stft = torch.stft(
x,
self.n_fft,
self.hop_length,
self.win_length,
self.window.to(x.device),
self.center,
self.pad_mode,
self.normalized_stft,
self.onesided,
return_complex=False,
)
n_fft = 400
win_length = 400
hop_length = 160
window function: torch.hamming_window
center: True
pad_mode : "constant"
normalized_stft: False
onesided: True
How hard if I want change this library to make it produce same result as pytorch STFT?

Not possible..
It only support 2^n size of n_fft and 25%,50% of n_fft for hop_length..
I heard https://github.com/mborgerding/kissfft support non 2^n n_fft size.


Fix the build for ARM

Fixes the ARM neon detection in the configure file, which currently makes the build fail and was not detected by the CI before the merge of https://github.com/project-everest/hacl-star/pull/408

Sorry: this fixes ARM

Is gcc-compatible the only place where you need to update libintvector ?
Also the commit message makes no sense, there is nothing AMD related.
x86_64?

The important file is dist/configure, dist/gcc-compatible was updated only for a CI test, and the other configure files will be refreshed during the build.


Add missing perl module dependencies (for release/95)

See https://github.com/Ensembl/ensembl-rest/pull/300

Definitely needed but let's hold off merging until Travis stops failing on missing release branches, just in case.


fix: Fixes interactions with alert API & style tool version constraints

This PR is meant to be merged before #91 to solve a few aspects:

pinning the black version used by the CI
doing the same for the mypy version
updating the service name to a more appropriate one
fixed the headers

Apart from codebase edits, I did:

boot a dev bucket & API instance
updated the secrets in this repo accordingly

Final note: it's "normal" that some CI jobs fail as the API url is not dynamic yet until #91 is merged

Thanks @frgfm ! Now the credentials are fixed, I re-ran the checks in #91 and they now pass

The issue with mypy was fixed #91
I didn't ask before but why do you think it is better to stick to the previous version of black instead of moving to the one used in #91 (and sticking to that version) ?

My apologies for the late reply @blenzi :upside_down_face:
Regarding black, we need to stick to one version. And the one I specified in this PR is the one that has been used on all our repos so I suggest staying on that one for harmonization sake :smiley:


An Installation problem

Hello, @Fabio Silva.  I followed the instruction and succeeded installing for the first time, but I couldn't download it again although I repeated the instruction and I got enough space. It showed the problem as follows when I ran bash easy_install_bbp_19.4.0.sh:
=> Installing Broadband Platform Velocity Model Packages
==> LA Basin
tar (child): labasin500-velocity-model-19.4.0.tar.gz：Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now
md5sum: labasin500-velocity-model-19.4.0.tar.gz: No such file or directory

=> ERROR! MD5SUM check failed for labasin500-velocity-model-19.4.0.tar.gz

I use Ubuntu 20.04 with VMware Workstation. My software dependencies:
Python 2.7.14 :: Anaconda, Inc.
scipy (1.2.3)
matplotlib (2.2.3)
numpy (1.16.6)
pyproj (1.9.6)
FFTW library 3.3.10
tar (GNU tar) 1.30
I checked the https://hypocenter.usc.edu/research/bbp/versions/19.4.0/*** (in the easy_install_bbp_19.4.0.sh) and failed to enter the website, showing 502 error. Should I change something in easy_install_bbp_19.4.0.sh file to solve this problem? Thank you in advance for reviewing.

have you solved this issue?


Create a separate taskbar entry of Windows Terminal of a specific profile

I have been using WT for cmd and pwsh and the other console, whatever it's called, for the Ubuntu bash.

I know WT can open bash as well, but I use tmux with bash, so I don't need the extra functionalities of WT. What I need is a way to switch between the Ubuntu bash and Windows cmd with a single stroke. Having separate taskbar entries let me do just that (Win+1 and Win+2).
Now my question is: can I have a separate taskbar entry for a single specific profile of WT? MSYS2 has separate exe files for launching separate terminals for each of their shells, and the exe files are essentially cmd-scripts inside. But making exe out of those shell scripts let the terminals pinnable in taskbar. I want that for Ubuntu bash of WT.

Sorry for asking what may be a question irrelevant to this project.

Having separate taskbar entries is a bit tricky...
We've been discussing something similar over in #12135. Basically you'd have different taskbar shortcuts that each summon the terminal window that has whatever profile you're looking for. The taskbar entries wouldn't get highlighted separately (only the actual Terminal window would get focused), but it would otherwise effectively work the same.
You think that would work for you?

I'm running into a similar issue now that I'm on Windows 11.
I'd like my profiles (WSL, CMD, etc.) to have different slots on the taskbar, as it's easier for me to organize that way. Having everything clumped into one icon on the task bar makes it harder to find what I need. Instead of 1 easy click, it's 2 clicks with some pausing to see what I'm actually after.


caught this in my console upon startup. not sure what it is

Description (Required)

see pastebin
https://pastebin.com/sKwS62z1
Steps to reproduce the Issue (Required)

Expected behavior (Required)

Server Log / Error Report

Environment (Required)

Minecraft Version:1.18.1
CS-CoreLib Version:n/a
Slimefun Version: 1006
Plugin Version: 19

AlchimiaVitae requires 1.18.2 now, due to a change in Paper.
I might make a legacy version later though (for Spigot 1.16.x-1.18.x or Paper < 1.18.2)


Correlation function BAO damping

There appears to be under-damping of the BAO signal in the correlation_3d function for halofit. The attached plot shows the comparison between linear, halofit and cosmic emu results from ccl using the code snippet below. (Note that I didn't explictly set m_nu to zero here, but doing this doesn't substantially change the result)

import pyccl
import numpy as np
import matplotlib.pyplot as plt
cosmo = pyccl.Cosmology(transfer_function='boltzmann_camb', matter_power_spectrum='linear', Omega_c=0.26067, Omega_b=0.02242/0.67662,h=0.6766, n_s=0.9665, sigma8=0.8102)
cosmo2 = pyccl.Cosmology(transfer_function='boltzmann_camb', matter_power_spectrum='halofit', Omega_c=0.26067, Omega_b=0.02242/0.67662,h=0.6766, n_s=0.9665, sigma8=0.8102)
cosmo3 = pyccl.Cosmology(transfer_function='boltzmann_camb', matter_power_spectrum='emu', Omega_c=0.26067, Omega_b=0.02242/0.6766**2,h=0.6766, n_s=0.9665, sigma8=0.8102, Neff = 3.04)
r = np.linspace(0.0,200.,1000)
corr_lin = pyccl.correlation_3d(cosmo,1.0,r/0.6766)
corr_halofit = pyccl.correlation_3d(cosmo2,1.0,r/0.6766)
corr_emu = pyccl.correlation_3d(cosmo3,1.0,r/0.6766)
plt.figure()
plt.plot(r,corr_linr**2,label="linear")
plt.plot(r,corr_halofitr2,'--',label="halofit")
plt.plot(r,corr_emu*r2,'-.',label="emu")
plt.legend()
plt.xlabel('r Mpc/h')
plt.ylabel('r^2 zeta_mm (r) Mpc/h ^2')
plt.legend()
plt.show()

Note that this may be a halofit model limitation rather than a CCL issue @beckermr and I are looking into this to confirm

@patricialarsen - It looks like this is an issue that occurs with halofit in general rather than CCL. I'm not sure we have any immediate fix available.

This is what we found too. I am going to close this issue for now.

I'm re-opening this and assigning it to myself. FAST-PT has a resummation method built in (from Ivanov) to do BAO damping at 1loop PT. Exposing this should be straightforward. Generalizing this to damp the halofit BAO is more complicated, but is a good stretch goal.

For a more ad-hoc approach, the new HMCode uses a perturbation-theory inspired fitting function (Eq. 14 in https://arxiv.org/pdf/2009.01858.pdf).

@jablazek should this still be open?

@damonge : Good question. I have not exposed the FAST-PT functionality to PTTracers. However, as @tilmantroester notes, there are now more modern approaches which might be better. Is the HMCode method already available in CCL? If so, maybe that is sufficient?

We can access hmcode via CAMB. It's not ideal, but if we'd rather aim for that than expose the fastpt functionality, we can close this issue and open an HMCode-specific one (or keep both open!)

Keep this open for now. I have a few fast-pt -> CCL housekeeping tasks to do.

Tagging this as a v3 milestone since it's time we updated the old halofit option with a more modern version like the one implemented in HMCode.

Removed it from v3 milestones because it will work much better after #1063 .


The sqlx dose not support with sync style  ?

The sqlx dose not support with sync style  ?
Any sync example code ?
Thanks!

It doesn't appear this was done in 0.6 or 0.7, it could probably be achieved using maybe_async


Add Bluejay

Bluejay
Project URL
https://github.com/steamclock/bluejay
Description
A simple Swift framework for building reliable Bluetooth LE apps.
Why it should be included to awesome-ios (optional)
Checklist

[x] Only one project/change is in this pull request
[x] Addition in chronological order (bottom of category)
[x] Supports iOS 9 / tvOS 10 or later
[x] Supports Swift 3 or later
[x] Has a commit from less than 2 years ago
[x] Has a clear README in English

          1 Error

:no_entry_sign:
Found 2 link issues

Link issues by awesome_bot

Line
Status
Link

767
301
https://github.com/zalando-incubator/MapleBacon redirects tohttps://github.com/JanGorman/MapleBacon

994
301
http://www.netmera.com redirects tohttps://www.netmera.com/

  Generated by :no_entry_sign: Danger

Thx


[BUG] only bucket list should scroll

Please confirm these before moving forward

[X] I have searched for my issue and not found a work-in-progress/duplicate/resolved issue.
[X] I have tested that this issue has not been fixed in the latest (beta or stable) release.
[X] I have checked the FAQ section for solutions
[X] This issue is about a bug (if it is not, please use the correct template)

Describe your issue
currently both the settings page and the bucket list are scrolled together. It would make more sense if only the bucket list were scrolled in case the mouse is hovering it
Steps to reproduce the issue
see video
WingetUI Log
not needed

Package Managers Logs
not needed

Relevant information
No response
Screenshots and videos
https://github.com/marticliment/WingetUI/assets/73800734/a233d250-db32-42c4-b89f-eab77ff7bed6

I am aware of this behaviour, but I haven't succeeded at fixing it


Many errors appearing at compilation

Using Xcode 8.2.1 and Swift 3.0 I got many errors like this :

Use of undeclared type 'ExprNode'
Use of unresolved identifier 'get'
Use of unresolved identifier 'Lexer'
Use of unresolved identifier 'Parser'

I have no idea where it could come from .
Thanks for your help !

Should be fixed now. Sorry.


Validating conformance with registry.rnc

https://github.com/KhronosGroup/OpenXR-SDK-Source/blob/f122f9f1fc729e2dc82e12c3ce73efa875182854/specification/Makefile#L552-L557
How should xr.xml be validated against registry.rnc? This seems to be the only mention of registry.rnc in the repository, but neither checkXml.sh or registry/registry.rnc actually exist.
Relatedly, which registry.rnc file is xr.xml valid for? It's unclear which revision of the upstream Vulkan repository is meant to be used.

Oh, I see, they're located in the OpenXR-Docs repo. https://github.com/KhronosGroup/OpenXR-Docs/tree/cc81ecbe6559ee3c6081d53e6670d00972deee72/specification/registry
Top, that's what I was looking for :) I probably missed some documentation about the relationships between these repos

Yeah, these are all derived from an internal monorepo, so sometimes the documentation is a bit awkward. I'm not sure if I ever put up the diagram showing how they fit together.


feat(ui): custom queries on the frontend

Description
This PR removes the custom queries and replaces it with TanStack queries. The only remaining query is the query in the Table component which will be removed when we rework the whole Table component next.
Type of Change

[ ] Bug Fix
[ ] New Feature
[ ] Breaking Change
[x] Refactor
[ ] Documentation
[ ] Other (please describe)

Checklist

[x] I have read the contributing guidelines
[ ] Existing issues have been referenced (where applicable)
[x] I have verified this change is not present in other open pull requests
[ ] Functionality is documented
[x] All code style checks pass
[ ] New code contribution is covered by automated tests
[x] All new and existing tests pass

will be removed when we rework the whole Table component next

my only suggestion is to capture this in a ticket and have it as a reference here


Comments removed from empty tuple structs

- struct Foo(/* TODO */);
+ struct Foo();

The code remains valid, but comments should not be removed.

Looks like this is a duplicate of #919.

Yup, I agree. Closing as duplicate.


GetPurchasesAsync Exception

If you are creating an issue for a BUG please fill out this information. If you are asking a question or requesting a feature you can delete the sections below.
Failure to fill out this information will result in this issue being closed. If you post a full stack trace in a bug it will be closed, please post it to http://gist.github.com and then post the link here.
Bug Information
Version Number of Plugin: 1.2.2 (nuget)
Device Tested On: iPhone 6S Plus
iOS Version: iOS 8
Hello.
I'm using this library to purchase Auto-Renewable Subscriptions on Xamarin Forms.
The purchase process works, but after 1 month (subscription expiration) I'm having an issue to check if already exists (restore).
When I try to get the purchased items, using GetPurchasesAsync, an exception occurs:
https://gist.github.com/MarcorOnline/319897c5256614272aed9c62acc47376
Code snippet
var storePurchased = await billing.GetPurchasesAsync(ItemType.Subscription);
Thanks for your support

I'm looking the source code of this plugin and, maybe, there is a bug in "ToIABPurchase" method:
`var p = transaction.OriginalTransaction;
if (p == null)
p = transaction;
if (p == null)
return null;`
Should be replaced with:
var p = transaction?.OriginalTransaction ?? transaction; if (p == null)   return null;
I don't know if the crash is related with this, but can be an improvement.
Thanks

@jamesmontemagno I discovered that the real problem is that is necessary to call "CrossInAppBilling.Dispose()" to reset the "CrossInAppBilling.Current" instance. It's not enoug to simply call dispose (for example with using) on the "CrossInAppBilling.Current" instance.
Working with Subscriptions I've found another problem in the "ToIABPurchase" method. Preferring the "OriginalTransaction" it's not possible to detect the last renewing DateTime.
Why you don't use directly the "transaction" object?
Best regards,
Marco

I am not sure what you are talking about on the Dispose...
It should be:
CrossInAppBIlling.Current.Dispose();
However you never really need to do this from my testing.

@jamesmontemagno CrossInAppBIlling.Current.Dispose(); is not sufficient.
If you look the code, the "Current" entity is not resetted and if you try to use it again you obtain an exception because the "observer" (in iOS implementation) is nulla after CrossInAppBIlling.Current.Dispose();
If you call "CrossInAppBilling.Dispose()" the CrossInAppBIlling.Current.Dispose();  is included, but also a new Lazy for "Current" is created.
To reproduce the problem:

Get CrossInAppBIlling.Current
Call "GetPurchasesAsync"
Call dispose
Get CrossInAppBIlling.Current again
Call "PurchaseAsync" -> Exception!

Best regards,
Marco

I have updated the documentation.

@MarcorOnline  Can you able to make it working restore purchases for subscription? If so could you please post the code on how to do it?. I am in the same situation.
Thanks


png image can't be cache

What system are you running Yazi on?
Linux X11
What terminal are you running Yazi in?
foot version: 1.17.2 +pgo +ime +graphemes -assertions
yazi --debug output

Yazi
    Version: 0.3.0 (7a380c2 2024-08-01)
    Debug  : false
    OS     : linux-x86_64 (unix)

Ya
    Version: 0.3.0

Emulator
    Emulator.via_env: ("foot", "")
    Emulator.via_csi: Ok(Foot)
    Emulator.detect : Foot

Adapter
    Adapter.matches: Sixel

Desktop
    XDG_SESSION_TYPE: Some("wayland")
    WAYLAND_DISPLAY : Some("wayland-1")
    DISPLAY         : Some(":0")

SSH
    shared.in_ssh_connection: false

WSL
    /proc/sys/fs/binfmt_misc/WSLInterop: false

Variables
    SHELL              : Some("/bin/fish")
    EDITOR             : Some("nvim ")
    YAZI_FILE_ONE      : None
    YAZI_CONFIG_HOME   : None
    ZELLIJ_SESSION_NAME: None

Text Opener
    default: Some(Opener { run: "$EDITOR \"$1\"", block: true, orphan: false, desc: "neovim", for_: None, spread: false })
    block  : Some(Opener { run: "$EDITOR \"$1\"", block: true, orphan: false, desc: "neovim", for_: None, spread: false })

tmux
    TMUX   : false
    Version: 3.4

Dependencies
    file             : 5.45
    ueberzugpp       : No such file or directory (os error 2)
    ffmpegthumbnailer: 2.2.2
    magick           : 7.1.1-36
    fzf              : 0.54.0
    fd               : 10.1.0
    rg               : 14.1.0
    chafa            : 1.14.1
    zoxide           : 0.9.4
    7z               : No such file or directory (os error 2)
    7zz              : 23.01
    jq               : 1.7.1

Did you try the latest nightly build to see if the problem got fixed?
Yes, and I updated the debug information above (yazi --debug) to the nightly that I tried
Describe the bug
png image file preload always reutrn 2 (make cache fail)
  2024-08-01T08:33:58.640057Z ERROR yazi_scheduler::prework::prework: Returned 2 when running preloader `image` with `/tmp/test/sl.png`
    at yazi-scheduler/src/prework/prework.rs:76

  2024-08-01T08:33:58.668939Z ERROR yazi_scheduler::prework::prework: Returned 2 when running preloader `image` with `/tmp/test/home.png`
    at yazi-scheduler/src/prework/prework.rs:76

Expected Behavior
all image file should able to cache.
To Reproduce
1.watch the log
2.enter a folder which has png image
Configuration
No response
Anything else?
No response

Please upload the preload failed images

I tested it again and it's not about the image format, it's about the image size.

like this one.
https://github.com/leihaojun/test/blob/main/home.png

It sounds like a similar issue as https://github.com/sxyazi/yazi/issues/373

It doesn't seem to be all about size either, but the following files are stable to fail
https://github.com/leihaojun/test/blob/main/home.png

the bug is also from this commit.


remove known for CASSANDRA-11395

forgot to pull these in my pr with the fix.

Looks good, if a smoke test of just running the module successfully runs tests, then +1.

looks gtg, thanks @mambocab


Deprecated versions of packages

Expected Behavior

Package version dependencies should be fixed to a specific version tag, preferrably the latest version of a package.
Current Behavior

When running legainit in my python environment I hit:
CryptographyDeprecationWarning: signer and verifier have been deprecated. Please use sign and verify instead.
Possible Solution

See an example of the install_requires directive in the rega CLI repository:
    install_requires=[         'click==7.0',         'pyhcl==0.3.11',         'click-plugins==1.0.4',         'docker==3.7.0',         'cryptography==2.5.0',         'flake8==3.7.7',         'PyYAML==5.1'     ]
Steps to Reproduce

Activate python environment
sudo python setup.py install

Context (Environment)

Python version: 3.7.1

@jonandernovella that Warning comes from pgpy it is not related to legainit
/3.7.2/lib/python3.7/site-packages/pgpy/packet/fields.py:1177: CryptographyDeprecationWarning: signer and verifier have been deprecated. Please use sign and verify instead.
  signer = self.__privkey__().signer(padding.PKCS1v15(), hash_alg)

PGPy did not have a release in ~2years: https://github.com/SecurityInnovation/PGPy/releases , there is nothing we can do about it.
I tested in 3.7.2 pip freeze shows:
╰─$ pip freeze
asn1crypto==0.24.0
cffi==1.12.2
click==6.7
cryptography==2.6.1
filelock==3.0.10
lega-init==0.1.0
PGPy==0.4.3
pluggy==0.9.0
py==1.8.0
pyasn1==0.4.5
pycparser==2.19
PyYAML==5.1
singledispatch==<IP_ADDRESS>
six==1.12.0
toml==0.10.0
tox==3.7.0
virtualenv==16.4.3

The only thing I can update is click maybe.

@jonandernovella on a different note flake8 should go in extra_require for test not in install_requires and you might want flake8-docstrings for pep257.

@blankdots Not necessarily. The user might want to run make, which necessitates of make

@blankdots Right. This means that it is not an actual bug

yeap, sure use it how you want, but usually lint check is done as part of tests and tox is a nice way to work with that.


Partially delivered lines statement is incorrect

Please can you review the statement where you the article says that partially delivered purchase order line order quantity cant be amended? This only applies when the partial delivery is also invoiced. While only delivered, you can still make changes to the quantity field. Once invoiced, only delivery remainder can be canceled. (tested in contoso USMF 10.0.35)

Thanks for your feedback. We have passed this feedback onto the product team for consideration.


Drop forge stubbing support

6c87f62050c617fcf64ff7de228e2e1488ca4ee1 deprecated forge stubbing in favor of using --module-repository as a proper Puppet option. This works properly with TLS validation.

@mwaggett - I think this might supersede the draft PR you have open

thanks for the heads up @genebean!
These methods are definitely still used in PE acceptance tests, so this has the potential to break things.

Given they've been deprecated for nearly 3 years now, I'd prefer to proceed with removal. It's inherently broken if you want to use TLS validation without custom CAs and issuing certificates for servers you don't own.
Can we agree on a timeframe for PE acceptance tests to migrate?

@ekohl I think we may be able to avoid a lot of PE-related pain if you all do a major version bump on the release that includes this change. That should keep liberally configured gem files from pulling it in... does that seem right to you @mwaggett ?

I don't see a backwards-incompatible label, but IMHO this is certainly a major version bump since it changes the API.

I have no doubt labels could stand to be updated .......

Yes, I'll see about doing that in bulk. While I haven't checked, I'm fairly certain it's true for all beaker repos that were migrated.

I share that opinion @ekohl. Thanks!

yes, this should definitely happen across a major version boundary. I hope the projects that use this wouldn't automatically pull in a new major version, but I'm not really sure.
To be clear, I am in favor of this change, I'm just trying to minimize the disruption to PE teams.

I'm not in any hurry to merge this, but https://github.com/voxpupuli/beaker-puppet/pull/159 triggered me to submit it. I prefer not to change deprecated functionality.
Can we agree to close that PR and merge this once you give approval? That should give the PE teams sufficient time to prepare (either by pinning or migrating to the module repository option) while still moving forward.

@ekohl yeah, that makes sense to me, feel free to close #159. I was hoping that PR would fix a test failure I'm seeing, but it didn't, and if the forge methods are potentially broken anyway, that might be why. hoping to try using the generic host methods soon.
since I don't have the time (nor is it my job) to audit all the PE tests, I think this is fine as-is, as long as it's released in a major version, and teams can update their pipelines if/when they break.

@ekohl Is the plan to continue supporting the generic host equivalents of these methods (e.g. stub_hosts_on)?

Yes, they look generic and I can see use cases for that. While IMHO DNS should generally be used, I know it can be hard to guarantee in labs/dev/test/CI. That's why I didn't deprecate those APIs in my original PR.

I think the implementation could be cleaned up, for example, we could call Resolv.getaddress(hostname) instead of curling and trying to match an IP address in stderr.

I'm not entirely sure what you mean by that, but stubbing the forge this way is just wrong. IMHO this code shouldn't have existed in the first place.
However, recently I dove into beaker-puppet and besides a few methods (apply_manifest_on, install_puppetlabs_release_repo_on, copy_module_to, install_puppet_module_via_pmt_on, fact_on and puppet_config) it's mostly just useless to Voxpupuli's use case. I'm thinking of puling those out and getting rid of this gem in our CI. Then we can also get rid of beaker-pe and beaker-puppet_install_helper (the latter is archived but we still use it for now).
I just need to figure out the best way to avoid a conflict.

we could call Resolv.getaddress(hostname)

Since ruby is present on all systems under test, it's possible to run execute a ruby process to do the lookup, e.g. ruby -e 'require "resolv"; puts Resolv.getaddress("#{hostname}")' Or shell out to something like nslookup. Although it may not be installed on supported platforms.

IMHO this code shouldn't have existed in the first place.

Yes I completely agree. Making a TLS connection to resolve an IP address and scraping stderr for the IP address is :banana:'s

it's mostly just useless to Voxpupuli's use case. I'm thinking of puling those out and getting rid of this gem in our CI

We still use beaker-puppet in our CI systems, yet we will still need to support new puppet collections and platforms in the future. So I'd like to avoid having to fork.

it's mostly just useless to Voxpupuli's use case

How can we solve this such that it meet's Vox's use case but also still Puppet's use case?

@joshcooper are you looking at the right issue? This is forge stubbing. The right solution to that is configuring Puppet to use a different module repository, right?
So this is getting a bit off topic, but @binford2k: there are a lot of methods that are really test cases. Like spinning up Puppetserver instance. Also there are a lot of hardcoded paths which makes sense if you can't rely on Puppet being present or you want to be explicit, but in Voxpupuli's case you generally want things to be dynamic and utilize Puppet's built in capabilities as much as possible. That means the only thing we need is to bootstrap Puppet itself. For that we need install_puppetlabs_release_repo_on and install_package (provided in Beaker itself).
Once we have Puppet present, we do utilize helpers. At least in Voxpupuli we use copy_module_to and install_module_via_pmt_on but in Foreman we've replaced those with fixture-based installations. apply_manifest_on is used a lot since that's how we run most tests. I think puppet_config (which is a simple wrapper for puppet config print) is used by beaker-hiera to determine the code directory.
We don't need to install from development builds nor install from git. All of that code is really complex and part of it really only works from the Puppet VPN. For our use case we don't even care about installing on Windows since we don't really test on that. Last week I started on a path where I started ripping things out and came to the conclusion that it may be easier to consider what needs to stay.
And just to be clear, I am really talking about this gem in particular (beaker-puppet).

are you looking at the right issue? This is forge stubbing. The right solution to that is configuring Puppet to use a different module repository, right?

Ah yeah, I'm somewhat conflating this with the other issue (about how to resolve the hostname into an IP). I agree the most sensible thing would be to point puppet at a different module repository, but we can't do that due that due to constraints in our infrastructure.
Also agree there is a lot of cruft in here, some of it is left over from puppet 3 and earlier days like running puppet master and passenger. But the forge stubbing is unfortunately still needed.
Maybe pull out the methods that you want/need into a separate gem and let this one go off into the sunset?

Maybe pull out the methods that you want/need into a separate gem and let this one go off into the sunset?

That is indeed what I'm currently thinking about. Now just to figure out the naming ;)

We have moved away from beaker-puppet so I no longer have interest in this gem.


[Bug] DBT Seed in dbt-athena-community is not working for .csv with encoding utf-8

Is this a new bug in dbt-core?

[X] I believe this is a new bug in dbt-core
[X] I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior
I would like to start by apologizing if there is already a bug report on this subject, but I couldn't find it.
Lets go, I have a .csv file (sds_teste.csv) that I am using as a seed, like this:

So, I execute the command dbt seed -s sds_teste and the dbt is successful executed

But, when i execute a select to see the table created by dbt seed command, I can see that the table cannot read the special characters (accented letters)

I already try some things that I found around de internet, like to pass the encoding: utf-8, but i not found nothing that working.
My profiles.yml

Expected Behavior
The expected behavir is that the dbt seed would can read a .csv file in encoding utf-8.
Should be: A text with special characters, like Ã, Á, Í, or Ç
Instead of: A text with special characters, like �, �, �, or �
Steps To Reproduce
1 - Install the python 3.11.9 in a windows computer
2 - Create a python environment with python venv
3 - Install dbt-core==1.8.7 and dbt-athena-community==1.8.4 by pip install
4 - Create a dbt project
5 - Create a .csv file in the folder seeds/ and write some example with special characters
6 - Configure the profile.yml to connect a AWS Athena(storage: AWS S3)
7 - Run the dbt seed command
Relevant log output
============================== 13:38:20.928977 | c0ac6503-3146-44d1-84fa-552929776d79 ==============================
13:38:20.928977 [info ] [MainThread]: Running with dbt=1.8.7
13:38:20.929979 [debug] [MainThread]: running dbt with arguments {'printer_width': '80', 'indirect_selection': 'eager', 'write_json': 'True', 'log_cache_events': 'False', 'partial_parse': 'True', 'cache_selected_only': 'False', 'warn_error': 'None', 'fail_fast': 'False', 'profiles_dir': 'C:\\Users\\julio.padoveze\\.dbt', 'log_path': 'C:\\Users\\julio.padoveze\\github\\prd-Copastur-dbt-source-qv\\logs', 'version_check': 'True', 'debug': 'False', 'use_colors': 'True', 'use_experimental_parser': 'False', 'no_print': 'None', 'quiet': 'False', 'empty': 'None', 'log_format': 'default', 'introspect': 'True', 'invocation_command': 'dbt seed -s sds_teste', 'static_parser': 'True', 'target_path': 'None', 'warn_error_options': 'WarnErrorOptions(include=[], exclude=[])', 'send_anonymous_usage_stats': 'True'}
13:38:21.527840 [debug] [MainThread]: Sending event: {'category': 'dbt', 'action': 'project_id', 'label': 'c0ac6503-3146-44d1-84fa-552929776d79', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x0000020521A27B10>]}
13:38:21.561425 [debug] [MainThread]: Sending event: {'category': 'dbt', 'action': 'adapter_info', 'label': 'c0ac6503-3146-44d1-84fa-552929776d79', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x000002051CDCDE10>]}
13:38:21.563666 [info ] [MainThread]: Registered adapter: athena=1.8.4
13:38:21.575796 [debug] [MainThread]: checksum: 4af21dafb485259c48497ac86b711ddb1982f3d0f1c0ca4e09356de488b753c0, vars: {}, profile: , target: , version: 1.8.7
13:38:21.689602 [info ] [MainThread]: Unable to do partial parsing because of a version mismatch
13:38:21.690599 [debug] [MainThread]: Sending event: {'category': 'dbt', 'action': 'partial_parser', 'label': 'c0ac6503-3146-44d1-84fa-552929776d79', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x0000020521C79010>]}
13:38:22.937805 [debug] [MainThread]: Sending event: {'category': 'dbt', 'action': 'load_project', 'label': 'c0ac6503-3146-44d1-84fa-552929776d79', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x0000020522118250>]}
13:38:23.152658 [debug] [MainThread]: Sending event: {'category': 'dbt', 'action': 'resource_counts', 'label': 'c0ac6503-3146-44d1-84fa-552929776d79', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x0000020522129ED0>]}
13:38:23.152658 [info ] [MainThread]: Found 17 models, 9 seeds, 96 sources, 568 macros
13:38:23.153659 [debug] [MainThread]: Sending event: {'category': 'dbt', 'action': 'runnable_timing', 'label': 'c0ac6503-3146-44d1-84fa-552929776d79', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x0000020521F006D0>]}
13:38:23.155683 [info ] [MainThread]:
13:38:23.157687 [debug] [MainThread]: Acquiring new athena connection 'master'
13:38:23.160683 [debug] [ThreadPool]: Acquiring new athena connection 'list_awsdatacatalog'
13:38:23.161689 [debug] [ThreadPool]: Opening a new connection, currently in state init
13:38:24.834482 [debug] [ThreadPool]: On list_awsdatacatalog: Close
13:38:24.840486 [debug] [ThreadPool]: Acquiring new athena connection 'list_awsdatacatalog_db_dbt_validation'
13:38:24.840486 [debug] [ThreadPool]: Opening a new connection, currently in state init
13:38:29.476313 [debug] [ThreadPool]: On list_awsdatacatalog_db_dbt_validation: Close
13:38:29.495682 [debug] [MainThread]: Sending event: {'category': 'dbt', 'action': 'runnable_timing', 'label': 'c0ac6503-3146-44d1-84fa-552929776d79', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x0000020521B74F10>]}
13:38:29.496682 [info ] [MainThread]: Concurrency: 10 threads (target='dev')
13:38:29.498679 [info ] [MainThread]:
13:38:29.512664 [debug] [Thread-1 (]: Began running node seed.qv.sds_teste
13:38:29.514172 [info ] [Thread-1 (]: 1 of 1 START seed file db_dbt_validation.db_qvextracao__sds_teste .............. [RUN]
13:38:29.516618 [debug] [Thread-1 (]: Acquiring new athena connection 'seed.qv.sds_teste'
13:38:29.518105 [debug] [Thread-1 (]: Began compiling node seed.qv.sds_teste
13:38:29.519110 [debug] [Thread-1 (]: Began executing node seed.qv.sds_teste
13:38:29.553617 [debug] [Thread-1 (]: seed by upload...
13:38:29.566419 [debug] [Thread-1 (]: Opening a new connection, currently in state init
13:38:32.830681 [debug] [Thread-1 (]: dbt.adapters.athena.constants adapter: Table "awsdatacatalog"."db_dbt_validation"."db_qvextracao__sds_teste__dbt_tmp" does not exists - Ignoring
13:38:32.832678 [debug] [Thread-1 (]: dbt.adapters.athena.constants adapter: Table "awsdatacatalog"."db_dbt_validation"."db_qvextracao__sds_teste__dbt_tmp" does not exist - Ignoring
13:38:32.834710 [debug] [Thread-1 (]: Dropping relation via Glue and S3 APIs
13:38:34.321827 [debug] [Thread-1 (]: dbt.adapters.athena.constants adapter: Table "awsdatacatalog"."db_dbt_validation"."db_qvextracao__sds_teste__dbt_tmp" does not exists - Ignoring
13:38:34.323331 [debug] [Thread-1 (]: dbt.adapters.athena.constants adapter: Table "awsdatacatalog"."db_dbt_validation"."db_qvextracao__sds_teste__dbt_tmp" does not exist - Ignoring
13:38:35.891369 [debug] [Thread-1 (]: dbt.adapters.athena.constants adapter: Table "awsdatacatalog"."db_dbt_validation"."db_qvextracao__sds_teste__dbt_tmp" does not exist and will not be deleted, ignoring
13:38:35.893875 [debug] [Thread-1 (]: Using athena connection "seed.qv.sds_teste"
13:38:35.893875 [debug] [Thread-1 (]: On seed.qv.sds_teste: -- /* {"app": "dbt", "dbt_version": "1.8.7", "profile_name": "qv", "target_name": "dev", "node_id": "seed.qv.sds_teste"} */
create external table `db_dbt_validation`.`db_qvextracao__sds_teste__dbt_tmp` (`id` string, `some_text` string)
    row format serde 'org.apache.hadoop.hive.serde2.OpenCSVSerde'
    location 's3://prd-lakehouse-copastur-analytics/analytics/env=prod/top=copastur/domain=analytics/dbt/db_dbt_validation/db_qvextracao__sds_teste__dbt_tmp'
    tblproperties (
      'skip.header.line.count'='1'
    )

13:38:36.637497 [debug] [Thread-1 (]: dbt.adapters.athena.constants adapter: Athena query ID 87dea510-7789-40ce-849c-4d2a4d6428bf
13:38:38.371586 [debug] [Thread-1 (]: SQL status: OK -1 in 2.477 seconds
13:38:38.386232 [debug] [Thread-1 (]: Skip partitioning: False
13:38:38.387230 [debug] [Thread-1 (]: dbt.adapters.athena.constants adapter: get_work_group for %s
13:38:39.758786 [debug] [Thread-1 (]: dbt.adapters.athena.constants adapter: S3 path does not exist
13:38:39.759788 [debug] [Thread-1 (]: Using athena connection "seed.qv.sds_teste"
13:38:39.759788 [debug] [Thread-1 (]: On seed.qv.sds_teste: -- /* {"app": "dbt", "dbt_version": "1.8.7", "profile_name": "qv", "target_name": "dev", "node_id": "seed.qv.sds_teste"} */
create table "awsdatacatalog"."db_dbt_validation"."db_qvextracao__sds_teste"
    with (
      table_type='hive',
      is_external=true,
      format='parquet'
    )
    as

    select

              cast(nullif("id", '') as integer) as "id"
            ,
              cast(nullif("some_text", '') as varchar) as "some_text"

    from
        "awsdatacatalog"."db_dbt_validation"."db_qvextracao__sds_teste__dbt_tmp"

13:38:39.966615 [debug] [Thread-1 (]: dbt.adapters.athena.constants adapter: Athena query ID 3ff995b0-dbb9-42b2-ab05-90844aeb9925
13:38:42.972477 [debug] [Thread-1 (]: SQL status: OK 2 in 3.212 seconds
13:38:44.460039 [debug] [Thread-1 (]: dbt.adapters.athena.constants adapter: table_name :<PHONE_NUMBER>44.db_dbt_validation.db_qvextracao__sds_teste__dbt_tmp
13:38:44.462029 [debug] [Thread-1 (]: dbt.adapters.athena.constants adapter: table type : TableType.TABLE
13:38:44.462029 [debug] [Thread-1 (]: Dropping relation via Glue and S3 APIs
13:38:45.992330 [debug] [Thread-1 (]: dbt.adapters.athena.constants adapter: table_name :<PHONE_NUMBER>44.db_dbt_validation.db_qvextracao__sds_teste__dbt_tmp
13:38:45.993334 [debug] [Thread-1 (]: dbt.adapters.athena.constants adapter: table type : TableType.TABLE
13:38:45.994409 [debug] [Thread-1 (]: dbt.adapters.athena.constants adapter: "awsdatacatalog"."db_dbt_validation"."db_qvextracao__sds_teste__dbt_tmp" is stored in s3://prd-lakehouse-copastur-analytics/analytics/env=prod/top=copastur/domain=analytics/dbt/db_dbt_validation/db_qvextracao__sds_teste__dbt_tmp
13:38:46.756532 [debug] [Thread-1 (]: dbt.adapters.athena.constants adapter: Deleting table data: path='s3://prd-lakehouse-copastur-analytics/analytics/env=prod/top=copastur/domain=analytics/dbt/db_dbt_validation/db_qvextracao__sds_teste__dbt_tmp', bucket='prd-lakehouse-copastur-analytics', prefix='analytics/env=prod/top=copastur/domain=analytics/dbt/db_dbt_validation/db_qvextracao__sds_teste__dbt_tmp/'
13:38:49.383198 [debug] [Thread-1 (]: dbt.adapters.athena.constants adapter: Deleted table from glue catalog: "awsdatacatalog"."db_dbt_validation"."db_qvextracao__sds_teste__dbt_tmp"

Environment
- OS: Windons 11
- Python: 3.11.9
- dbt-core: 1.8.7
- dbt-athena-community: 1.8.4

Which database adapter are you using with dbt?
other (mention it in "Additional Context")
Additional Context
No response

Hello,
It works on my side with dbt-athena on Windows.
Could you please try to add this parameter to .vscode/settings.json and open a new terminal in vscode ?
{
    "terminal.integrated.env.windows": {
        "PYTHONUTF8": "1"
    }
}

Hello @e-quili , thank you so much for collaboration! I tested this solution and it works!!
I will use this in my local environment for developments, but I still think that there is a bug, once that anothers adapters can identify the Encoding of the .csv file. What do you think?

Sooo I have the same issue. Right now I need to upload a big seed file with a lot of non latin letters e.g.:
Sedlišćo pódzajtšo, Dolna Łužyca, etc.
This will not work with the athena dbt adapter even if the letters are utf8.
I checked all possbilities of misconfigured encodings with:
os.environ["PYTHONIOENCODING"]
sys.getfilesystemencoding()
sys.getdefaultencoding()

And they are all set to utf8.
The issue is withing agate csv_py3.py:
The function writerow() calling self.writer.writerow(row) if I add a try / except I get all the rows that cannot be processed.
So here I am actually stuck - these are the rows that cannot be processed because somehow it always checks with cp1252 which does not contain any special characters.
Here is some test data:
"Sedlitz Ost (Sedlišćo pódzajtšo)",
"Senftenberg (Zły Komorow)",
"Cottbus Hbf Calau (NL) Chóśebuz gł.dw",
"Gollmitz (NL) Chańc (Dolna Łužyca)",
"Calau (NL) Kalawa (Dolna Łužyca",
"Kolkwitz Süd Gołkojce pódpołdnjo",
"Cottbus-Sandow Chóśebuz-Žandow",
"Cottbus-Merzdorf Chóśebuz-Žylowk",
"Cottbus-Willmersdorf Nord Chóśebuz-Rogoznow pódpołnoc",

Okay for everyone having issues with seeds and dbt athena on Windows: Set this: $Env:PYTHONUTF8=1


Standardize common utility naming conventions between languages?

We now support 3 languages: Scala, Java, and Kotlin. Each of these has linters, autoformatters, code coverage reporters, on top of the existing "standard" tasks like compile, run, test, publish, etc.
It would be nice if we could standardize .lint, .autoformat, .coverage, etc. as well. That will help keep the experience of using Mill consistent regardless of what language the codebase is written in, and would especially help in multi-language polyglot codebases

I'd like to extends this also for standard build steps. For example have a no-op trait defining compile, and all existing Languages extend that, so that we can easily stack languages on top of each other. Currently we can stack Kotlin on top of Java, but not Java on top of Kotlin.
As a more in-depth motivation: Although Kotlin as well as Scala support Java sources, they get in trouble when there comes a newer Java language feature or when the Java code is supposed to be processed further (thinking of annotation processors or AspectJ compilers). Having a interface-only trait as a base, we can make sure, a super-call always succeeds. Being free to stack in arbitrary order would be of great help.
And since we try to learn from this, we also should introduce interface-only traits for Linting and coverage collecting, so users can easily stack these in whatever order to achieve their setups.

Gradle calls these Lifecycle Tasks https://docs.gradle.org/current/userguide/organizing_tasks.html, which are tasks which don't do any of the work themselves but are used to group related tasks together under standardized labels for easy discoverability and usage


Update the charts and docs

update the chart to enable longhorn by default https://github.com/rancher/harvester/issues/348
add app-mode installation docs

@gitlawr  Updated


[CI] Previous CI runs aren't cancelled on PR change push

Describe your issue
When new changes are pushed to a PR, pipeline runs from the previous set of changes aren't automatically cancelled.

I don't know how to fix this, if anyone has any ideas please let me know


Save s3 message_id and event_time to table

Save S3 SQS notification message's id and time to table.

LGTM

https://github.com/aamine/bricolage-streaming-loader/pull/4 に含む


Make List.minBy and List.maxBy behave like the standard F# library methods

Make List.minBy and List.maxBy behave like the standard F# library methods

@Martin521 Thanks very much. Actually there a couple more related things to fix, I will sort these all out with optimized implementations https://github.com/intellifactory/websharper/issues/796

Thanks again for alerting us on these, I have added specialized, more performant implementations to all min/max methods, and also fixing this issue, see #796

You are really fast. Thanks!


Suggestion: maxLength for Phone Number Input

HI, I really like this library, but I cannot find the option to increase the number of digits as input available.
Since there are alot of countries that accept 11 numbers and more, rather than 10. this seems like an issue which users might face when trying to enter their phone number for authentication.
I would like to increase the number of digits accepted in the phone number input.

Hey @KushaanKapoor !
I see your PR about this discussion and I apreciate it.
So, I realease a new version with your contribute.
Please, update your react-native-international-phone-number lib to version 0.4.15 and enjoy!
Thanks,
🚀 AstrOOnauta 🚀

That's great! Thanks a lot @AstrOOnauta. I look forward to this library.


[Doc] add  custom properties for the reconsumeLater

This PR adds docs for   Add custom properties for the reconsumeLater interface  #13461
Preview looks good:

Documentation
Check the box below and label this PR (if you have committer privilege).
Need to update docs?

[x]  doc

@freeznet can you help review this PR from the technical perspective? Thanks

@freeznet PTAL


fix reference to renamed message length field

It seems the returned structure of "extractPacket" (now "extractPacketIntoFrame") used to have a field "packet_len".
This field was renamed first to "frame_len" in this commit:
https://github.com/tidepool-org/chrome-uploader/commit/8fafd23a4e7eb139bdaa6088f8695ab07e1fef8a#diff-faa8e43c937889d080484b85f6ab4115R68
and then to "messageLength" in this commit:
https://github.com/tidepool-org/chrome-uploader/commit/6c14f760abf56486801f4c91e96085cf00a04a6b#diff-faa8e43c937889d080484b85f6ab4115R64
But the check against it being 0 was never updated.
Thus, the referenced "packet_len" is now always "null", so the comparison will always return "true" (null !== 0).
This will lead to discarding of data, even when the "messageLength" is 0.
This fix is only needed in master, as on the electron branch this part of code has disappeared altogether.

The Bayer driver was originally refactored on gniezen/medtronic branch (which are the same changes seen on the Electron branch), and fixes this bug. The gniezen/medtronic branch will be merged (probably within the next week) into master before the next push to production, so closing this one.


Allowing the passing of the 'queue' option to manualy route tasks

If calling a task with { queue: 'whatever' } as the options object, the manual queue routing was never being passed through to RabbitMQ. This seems to fix that!

Merged, thanks! Please add your name to https://github.com/mher/node-celery/blob/master/AUTHORS