Add scala/cla-checker github action

@Sciss @benhutchison any objections? I am not insisting this be merged
probably we should have had it from the start, but we didn't 🤷
if you'd like to discuss, let me know. the Scala org doesn't, to my knowledge, have any further real interest in this repo, but also, we'd rather see it stay alive than die. whether this gets merged or not is a secondary concern. the text of the Apache 2 license is already very CLA-like


Renaming Api to Client

The OperationsApi -> OperationsClient rename was performed by an updated toolkit; toolkit's changes will appear in a separate PR.

LGTM once tests pass

Current coverage is 70.44% (diff: 84.61%)

Merging #150 into master will increase coverage by 0.04%

@@             master       #150   diff @@
==========================================
  Files            59         59
  Lines          2365       2365
  Methods           0          0
  Messages          0          0
  Branches        301        301
==========================================
+ Hits           1665       1666     +1
  Misses          616        616
+ Partials         84         83     -1

Powered by Codecov. Last update d4279b6...42e7965


Local (Country) Leaderboard?

Leaderboard having a filter-by-country option.
Or maybe add a local Leaderboard tab which shows the leaderboard of warriors in your country!

that would require a settign in the profile of any user to know if they are ok to be displayed in that board (it's actually kind of personnal information).


demo ui in jx

Note that merging this would break the automated dockerhub build for demo UI.
Also note that the Jenkinsfile config is particular to my repo and anyone else using it would have to find-replace

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 2 committers have signed the CLA.:white_check_mark: ryandawsonuk:x: almericoYou have signed the CLA already but the status is still pending? Let us recheck it.

obsolete

obsolete


Topic/i10n

This was mostly just an attempt to move all strings presented to the user to one place (and get rid of toString on the error case classes), but localisation came easily once that was done.

I would like to merge this, I think it's worthwhile just for the fact that it pulls all strings to one area.


Interactive tests do not count towards coverage

See for instance #177

I think we need to add a step there that pushes the code overage data


Candidates don't have tracknumbers

Hi,
thanks for maintaining a solution while the official beatport plugin is gone! I just came across some older Techno tracks that I'd like to import and they are nowhere to be found on Discogs or MusicBrainz. Just Beatport ;-) So: Thanks a ton!
I realised that tracknumbers are always missing in the candidates. I didn't investigate what to the problem could be, just reporting for now:
(Similarity: 42.5%) (missing tracks, media, source, tracks) (Beatport, Digital, 2009, Plus 8 Records, PLUS8100B)
 * Theychx (Tobias. Fuse Remix) (#12) (10:53) -> Intro (From Our Minds To Yours) (Tiefschwarz Remix) (#None) (10:40) (title, source, length)
Missing tracks (9/10 - 90.0%):
 ! 5 (Alex Under Remix)             (#None) (6:57)
 ! Ethnik (Paco Osuna Remix)        (#None) (9:45)
 ! Goo (Paco Osuna Remix)           (#None) (8:28)
 ! Hardphunk (Mike Shannon Reshake) (#None) (8:27)
 ! Kriket (Danilo Vigorito Remix)   (#None) (5:58)
 ! Statikphunk (Ambivalent Remix)   (#None) (6:04)
 ! Substance Abuse (Barem Remix)    (#None) (8:17)
 ! Vortex (Function Remix)          (#None) (7:18)
 ! Vortex (Hobo's Vile Remix)       (#None) (5:49)
[A]pply, More candidates, Skip, Use as-is, as Tracks, Group albums,
Enter search, enter Id, aBort, plaY, eDit, edit Candidates?

Hope that helps and thanks again!

Also I just realise right now that not all tracks containend on this release are listed as the missing tracks. That's a second issue I guess: https://beatport.com/release/the-remixes/2292897

Thank you, @JOJ0, for the kind words and reporting those issues with the examples! I'll have a look this week :)

Hi, thanks for taking a look and sorry for the late delay!
Ok, so now what I've tried so far:

Checking out from current beets master (had quite some unmerged features in use)
Disabling ALL plugins except beatport4

It's happening still.
Also I found out that it happens with single files as well. This is an example import command.
$ beet import -t /remote/data/music-archive/0-andi/Februar\ 2011/Alexander\ Ross\ -\ 01\ -\ Room.mp3

/remote/data/music-archive/0-andi/Februar 2011/Alexander Ross - 01 - Room.mp3 (1 items)
Correcting tags from:
    Alexander Ross - Universal Consciousness 1.1
To:
    Argy, Alexander Ross - Universal Consciousness 1.1
URL:
    https://beatport.com/release/universal-consciousness-11/217494
(Similarity: 78.1%) (media, missing tracks, artist, source, tracks) (Beatport, Digital, 2010, These Days, TD05)
 * Room (#1) -> Room (#None) (source)
Missing tracks (1/2 - 50.0%):
 ! Sometimes I'm Blind (#None) (6:31)
[A]pply, More candidates, Skip, Use as-is, as Tracks, Group albums,
Enter search, enter Id, aBort?

I include the beatport plugin config from an extra yaml file, if that matters:
beatport4:
    source_weight: 0.2
    username: <secret>
    password: <secret>
    tokenfile: beatport_token.json
    client_id:

Some more config parts that might be of interest:
match:
    ignored_media:
    - Data CD
    - DVD
    - DVD-Video
    - Blu-ray
    - HD-DVD
    - VCD
    - SVCD
    - UMD
    - VHS
    strong_rec_thresh: 0.15
    medium_rec_thresh: 0.65
    max_rec:
        missing_tracks: strong
        unmatched_tracks: medium
    distance_weights:
        track_title: 1.5
        album: 4.0
        catalognum: 2.0
        track_id: 2.5
    preferred:
        media:
        - Vinyl
        - 12"
        - 2x12"
        - LP
        - CD
        - Digital Media|File
        - Cassette
        countries: [GB, DE, FR]
        original_year: no

import:
    log: remotebeets-import.log
    copy: yes
    move: no
    write: yes
    resume: ask
    from_scratch: yes
    default_action: apply
    detail: yes
    non_rec_action: ask
    duplicate_action: merge
    incremental_skip_later: no
    incremental: yes
    group_albums: no
    bell: yes
    autotag: yes
    autotag_exclude_fields: []
    duplicate_keys:
        album: albumartist album catalognum
        item: artist title catalognum
    set_fields:
        genre_orig_dir: Andi2011
    ```

I also tested if it is only a "displaying" problem or if it really affects the imported track numbers being applied to the media files:
The source file:
$ mutagen-inspect /remote/data/music-archive/0-andi/Februar\ 2011/Alexander\ Ross\ -\ 01\ -\ Room.mp3 | grep TRCK
TRCK=1/2

The imported file:
$ beet info room alexander ross |grep -i track
       mb_trackid: 1079663
            track: 0
       tracktotal: 2

$ mutagen-inspect "/remote/data/music-library/Andi2011/Argy, Alexander Ross - 2010 - \$incomplete Universal Consciousness 1.1/\$disc_and_track - Alexander Ross - Universal Consciousness 1.1 - Room (2010).mp3" | grep TRCK
TRCK=0/2

We also see that some of the path formatting and inline plugin things that rely on the track number are not working, but that's not surprising I'd say :-)

Hi, did some debug printing today. Where in this API response should the track number be found? I can't find it or it is too early in the morning ;-)
 prod  (beets)    ~/git/beets-beatport4   main ●  beet import -I -t /remote/data/music-archive/0-andi/Februar\ 2011/Alexander\ Ross\ -\ 01\ -\ Room.mp3
In Beatport4Client.get_release_tracks, this is response:
[   {   'artists': [   {   'id': 8642,
                           'image': {   'dynamic_uri': 'https://geo-media.beatport.com/image_size/{w}x{h}/ba586072-e78a-4eac-a393-567101c46f7f.jpg',
                                        'id': 31657857,
                                        'uri': 'https://geo-media.beatport.com/image_size/590x404/ba586072-e78a-4eac-a393-567101c46f7f.jpg'},
                           'name': 'Argy',
                           'slug': 'argy',
                           'url': 'https://api.beatport.com/v4/catalog/artists/8642/'}],
        'available_worldwide': True,
        'bpm': 120,
        'catalog_number': 'TD05',
        'current_status': {   'id': 10,
                              'name': 'General Content',
                              'url': 'https://api.beatport.com/v4/auxiliary/current-status/10/'},
        'encoded_date': '2010-01-21T05:35:39-07:00',
        'exclusive': False,
        'free_download_end_date': None,
        'free_download_start_date': None,
        'free_downloads': [],
        'genre': {   'id': 6,
                     'name': 'Techno (Peak Time / Driving)',
                     'slug': 'techno-peak-time-driving',
                     'url': 'https://api.beatport.com/v4/catalog/genres/6/'},
        'id': 1079662,
        'image': {   'dynamic_uri': 'https://geo-media.beatport.com/image_size/{w}x{h}/83529a43-1875-41df-b05d-3f8455b19d30.png',
                     'id': 1445824,
                     'uri': 'https://geo-media.beatport.com/image_size/1500x250/83529a43-1875-41df-b05d-3f8455b19d30.png'},
        'is_available_for_streaming': True,
        'is_hype': False,
        'isrc': 'DECY50923051',
        'key': {   'camelot_letter': 'A',
                   'camelot_number': 5,
                   'chord_type': {   'id': 1,
                                     'name': 'Minor',
                                     'url': 'https://api.beatport.com/v4/catalog/chord-types/1/'},
                   'id': 5,
                   'is_flat': False,
                   'is_sharp': False,
                   'letter': 'C',
                   'name': 'C Minor',
                   'url': 'https://api.beatport.com/v4/catalog/keys/5/'},
        'label_track_identifier': '',
        'length': '6:31',
        'length_ms': 391500,
        'mix_name': 'Original Mix',
        'name': "Sometimes I'm Blind",
        'new_release_date': '2010-02-03',
        'pre_order': False,
        'pre_order_date': None,
        'price': {   'code': 'EUR',
                     'display': '€1.29',
                     'symbol': '€',
                     'value': 1.29},
        'publish_date': '2010-02-03',
        'publish_status': 'published',
        'release': {   'id': 217494,
                       'image': {   'dynamic_uri': 'https://geo-media.beatport.com/image_size/{w}x{h}/2230dabd-8bf5-48dc-a206-37fe57a67808.jpg',
                                    'id': 578334,
                                    'uri': 'https://geo-media.beatport.com/image_size/500x500/2230dabd-8bf5-48dc-a206-37fe57a67808.jpg'},
                       'label': {   'id': 12526,
                                    'image': {   'dynamic_uri': 'https://geo-media.beatport.com/image_size/{w}x{h}/371f84a5-f950-4065-ae7f-de6baf197d82.jpg',
                                                 'id': 353166,
                                                 'uri': 'https://geo-media.beatport.com/image_size/60x60/371f84a5-f950-4065-ae7f-de6baf197d82.jpg'},
                                    'name': 'These Days',
                                    'slug': 'these-days'},
                       'name': 'Universal Consciousness 1.1',
                       'slug': 'universal-consciousness-11'},
        'remixers': [],
        'sale_type': {   'id': 1,
                         'name': 'purchase',
                         'url': 'https://api.beatport.com/v4/auxiliary/sale-types/1/'},
        'sample_end_ms': 276600,
        'sample_start_ms': 156600,
        'sample_url': 'https://geo-samples.beatport.com/track/d457d9b9-70f9-4226-8870-4b15102bdccc.LOFI.mp3',
        'slug': 'sometimes-im-blind',
        'sub_genre': None,
        'url': 'https://api.beatport.com/v4/catalog/tracks/1079662/'},
    {   'artists': [   {   'id': 108690,
                           'image': {   'dynamic_uri': 'https://geo-media.beatport.com/image_size/{w}x{h}/0dc61986-bccf-49d4-8fad-6b147ea8f327.jpg',
                                        'id': 5539565,
                                        'uri': 'https://geo-media.beatport.com/image_size/590x404/0dc61986-bccf-49d4-8fad-6b147ea8f327.jpg'},
                           'name': 'Alexander Ross',
                           'slug': 'alexander-ross',
                           'url': 'https://api.beatport.com/v4/catalog/artists/108690/'}],
        'available_worldwide': True,

Another debug output, this time at the very end you'll see that number is not found in the data that BeatportTrack receives for initializing the object:
   prod  (beets)    ~/git/beets-beatport4   main ●  beet import -I -t /remote/data/music-archive/0-andi/Februar\ 2011/Alexander\ Ross\ -\ 01\ -\ Room.mp3
In Beatport4Client.get_release, this is response:
{   'artists': [   {   'id': 8642,
                       'image': {   'dynamic_uri': 'https://geo-media.beatport.com/image_size/{w}x{h}/ba586072-e78a-4eac-a393-567101c46f7f.jpg',
                                    'id': 31657857,
                                    'uri': 'https://geo-media.beatport.com/image_size/590x404/ba586072-e78a-4eac-a393-567101c46f7f.jpg'},
                       'name': 'Argy',
                       'slug': 'argy',
                       'url': 'https://api.beatport.com/v4/catalog/artists/8642/'},
                   {   'id': 108690,
                       'image': {   'dynamic_uri': 'https://geo-media.beatport.com/image_size/{w}x{h}/0dc61986-bccf-49d4-8fad-6b147ea8f327.jpg',
                                    'id': 5539565,
                                    'uri': 'https://geo-media.beatport.com/image_size/590x404/0dc61986-bccf-49d4-8fad-6b147ea8f327.jpg'},
                       'name': 'Alexander Ross',
                       'slug': 'alexander-ross',
                       'url': 'https://api.beatport.com/v4/catalog/artists/108690/'}],
    'bpm_range': {'max': 125, 'min': 120},
    'catalog_number': 'TD05',
    'desc': '',
    'enabled': True,
    'encoded_date': '2010-01-21T05:35:39-07:00',
    'exclusive': False,
    'grid': None,
    'id': 217494,
    'image': {   'dynamic_uri': 'https://geo-media.beatport.com/image_size/{w}x{h}/2230dabd-8bf5-48dc-a206-37fe57a67808.jpg',
                 'id': 578334,
                 'uri': 'https://geo-media.beatport.com/image_size/500x500/2230dabd-8bf5-48dc-a206-37fe57a67808.jpg'},
    'is_available_for_streaming': True,
    'is_hype': False,
    'label': {   'id': 12526,
                 'image': {   'dynamic_uri': 'https://geo-media.beatport.com/image_size/{w}x{h}/371f84a5-f950-4065-ae7f-de6baf197d82.jpg',
                              'id': 353166,
                              'uri': 'https://geo-media.beatport.com/image_size/60x60/371f84a5-f950-4065-ae7f-de6baf197d82.jpg'},
                 'name': 'These Days',
                 'slug': 'these-days'},
    'name': 'Universal Consciousness 1.1',
    'new_release_date': '2010-02-03',
    'override_price': None,
    'pre_order': False,
    'pre_order_date': None,
    'price': {'code': 'EUR', 'display': '€2.58', 'symbol': '€', 'value': 2.58},
    'price_override_firm': False,
    'publish_date': '2010-02-03',
    'remixers': [],
    'slug': 'universal-consciousness-11',
    'track_count': 2,
    'tracks': [   'https://api.beatport.com/v4/catalog/tracks/1079663/',
                  'https://api.beatport.com/v4/catalog/tracks/1079662/'],
    'type': {'id': 1, 'name': 'Release'},
    'upc': '827170091184',
    'updated': '2015-04-12T20:51:43-06:00'}
In Beatport4Client.get_release_tracks, this is response:
[   {   'artists': [   {   'id': 8642,
                           'image': {   'dynamic_uri': 'https://geo-media.beatport.com/image_size/{w}x{h}/ba586072-e78a-4eac-a393-567101c46f7f.jpg',
                                        'id': 31657857,
                                        'uri': 'https://geo-media.beatport.com/image_size/590x404/ba586072-e78a-4eac-a393-567101c46f7f.jpg'},
                           'name': 'Argy',
                           'slug': 'argy',
                           'url': 'https://api.beatport.com/v4/catalog/artists/8642/'}],
        'available_worldwide': True,
        'bpm': 120,
        'catalog_number': 'TD05',
        'current_status': {   'id': 10,
                              'name': 'General Content',
                              'url': 'https://api.beatport.com/v4/auxiliary/current-status/10/'},
        'encoded_date': '2010-01-21T05:35:39-07:00',
        'exclusive': False,
        'free_download_end_date': None,
        'free_download_start_date': None,
        'free_downloads': [],
        'genre': {   'id': 6,
                     'name': 'Techno (Peak Time / Driving)',
                     'slug': 'techno-peak-time-driving',
                     'url': 'https://api.beatport.com/v4/catalog/genres/6/'},
        'id': 1079662,
        'image': {   'dynamic_uri': 'https://geo-media.beatport.com/image_size/{w}x{h}/83529a43-1875-41df-b05d-3f8455b19d30.png',
                     'id': 1445824,
                     'uri': 'https://geo-media.beatport.com/image_size/1500x250/83529a43-1875-41df-b05d-3f8455b19d30.png'},
        'is_available_for_streaming': True,
        'is_hype': False,
        'isrc': 'DECY50923051',
        'key': {   'camelot_letter': 'A',
                   'camelot_number': 5,
                   'chord_type': {   'id': 1,
                                     'name': 'Minor',
                                     'url': 'https://api.beatport.com/v4/catalog/chord-types/1/'},
                   'id': 5,
                   'is_flat': False,
                   'is_sharp': False,
                   'letter': 'C',
                   'name': 'C Minor',
                   'url': 'https://api.beatport.com/v4/catalog/keys/5/'},
        'label_track_identifier': '',
        'length': '6:31',
        'length_ms': 391500,
        'mix_name': 'Original Mix',
        'name': "Sometimes I'm Blind",
        'new_release_date': '2010-02-03',
        'pre_order': False,
        'pre_order_date': None,
        'price': {   'code': 'EUR',
                     'display': '€1.29',
                     'symbol': '€',
                     'value': 1.29},
        'publish_date': '2010-02-03',
        'publish_status': 'published',
        'release': {   'id': 217494,
                       'image': {   'dynamic_uri': 'https://geo-media.beatport.com/image_size/{w}x{h}/2230dabd-8bf5-48dc-a206-37fe57a67808.jpg',
                                    'id': 578334,
                                    'uri': 'https://geo-media.beatport.com/image_size/500x500/2230dabd-8bf5-48dc-a206-37fe57a67808.jpg'},
                       'label': {   'id': 12526,
                                    'image': {   'dynamic_uri': 'https://geo-media.beatport.com/image_size/{w}x{h}/371f84a5-f950-4065-ae7f-de6baf197d82.jpg',
                                                 'id': 353166,
                                                 'uri': 'https://geo-media.beatport.com/image_size/60x60/371f84a5-f950-4065-ae7f-de6baf197d82.jpg'},
                                    'name': 'These Days',
                                    'slug': 'these-days'},
                       'name': 'Universal Consciousness 1.1',
                       'slug': 'universal-consciousness-11'},
        'remixers': [],
        'sale_type': {   'id': 1,
                         'name': 'purchase',
                         'url': 'https://api.beatport.com/v4/auxiliary/sale-types/1/'},
        'sample_end_ms': 276600,
        'sample_start_ms': 156600,
        'sample_url': 'https://geo-samples.beatport.com/track/d457d9b9-70f9-4226-8870-4b15102bdccc.LOFI.mp3',
        'slug': 'sometimes-im-blind',
        'sub_genre': None,
        'url': 'https://api.beatport.com/v4/catalog/tracks/1079662/'},
    {   'artists': [   {   'id': 108690,
                           'image': {   'dynamic_uri': 'https://geo-media.beatport.com/image_size/{w}x{h}/0dc61986-bccf-49d4-8fad-6b147ea8f327.jpg',
                                        'id': 5539565,
                                        'uri': 'https://geo-media.beatport.com/image_size/590x404/0dc61986-bccf-49d4-8fad-6b147ea8f327.jpg'},
                           'name': 'Alexander Ross',
                           'slug': 'alexander-ross',
                           'url': 'https://api.beatport.com/v4/catalog/artists/108690/'}],
        'available_worldwide': True,
        'bpm': 125,
        'catalog_number': 'TD05',
        'current_status': {   'id': 10,
                              'name': 'General Content',
                              'url': 'https://api.beatport.com/v4/auxiliary/current-status/10/'},
        'encoded_date': '2010-01-21T05:35:39-07:00',
        'exclusive': False,
        'free_download_end_date': None,
        'free_download_start_date': None,
        'free_downloads': [],
        'genre': {   'id': 6,
                     'name': 'Techno (Peak Time / Driving)',
                     'slug': 'techno-peak-time-driving',
                     'url': 'https://api.beatport.com/v4/catalog/genres/6/'},
        'id': 1079663,
        'image': {   'dynamic_uri': 'https://geo-media.beatport.com/image_size/{w}x{h}/84006767-84df-4873-af0f-4f67a5f594d2.png',
                     'id': 1445645,
                     'uri': 'https://geo-media.beatport.com/image_size/1500x250/84006767-84df-4873-af0f-4f67a5f594d2.png'},
        'is_available_for_streaming': True,
        'is_hype': False,
        'isrc': 'DECY50923052',
        'key': {   'camelot_letter': 'A',
                   'camelot_number': 7,
                   'chord_type': {   'id': 1,
                                     'name': 'Minor',
                                     'url': 'https://api.beatport.com/v4/catalog/chord-types/1/'},
                   'id': 7,
                   'is_flat': False,
                   'is_sharp': False,
                   'letter': 'D',
                   'name': 'D Minor',
                   'url': 'https://api.beatport.com/v4/catalog/keys/7/'},
        'label_track_identifier': '',
        'length': '8:12',
        'length_ms': 492887,
        'mix_name': 'Original Mix',
        'name': 'Room',
        'new_release_date': '2010-02-03',
        'pre_order': False,
        'pre_order_date': None,
        'price': {   'code': 'EUR',
                     'display': '€1.29',
                     'symbol': '€',
                     'value': 1.29},
        'publish_date': '2010-02-03',
        'publish_status': 'published',
        'release': {   'id': 217494,
                       'image': {   'dynamic_uri': 'https://geo-media.beatport.com/image_size/{w}x{h}/2230dabd-8bf5-48dc-a206-37fe57a67808.jpg',
                                    'id': 578334,
                                    'uri': 'https://geo-media.beatport.com/image_size/500x500/2230dabd-8bf5-48dc-a206-37fe57a67808.jpg'},
                       'label': {   'id': 12526,
                                    'image': {   'dynamic_uri': 'https://geo-media.beatport.com/image_size/{w}x{h}/371f84a5-f950-4065-ae7f-de6baf197d82.jpg',
                                                 'id': 353166,
                                                 'uri': 'https://geo-media.beatport.com/image_size/60x60/371f84a5-f950-4065-ae7f-de6baf197d82.jpg'},
                                    'name': 'These Days',
                                    'slug': 'these-days'},
                       'name': 'Universal Consciousness 1.1',
                       'slug': 'universal-consciousness-11'},
        'remixers': [],
        'sale_type': {   'id': 1,
                         'name': 'purchase',
                         'url': 'https://api.beatport.com/v4/auxiliary/sale-types/1/'},
        'sample_end_ms': 317154,
        'sample_start_ms': 197154,
        'sample_url': 'https://geo-samples.beatport.com/track/65d7357c-60e4-4894-947e-a7c907fe9d6f.LOFI.mp3',
        'slug': 'room',
        'sub_genre': None,
        'url': 'https://api.beatport.com/v4/catalog/tracks/1079663/'}]
'number' was not found in BeatportTrack.__init__
This is self.number in BeatportTrack.__init__: None
'number' was not found in BeatportTrack.__init__
This is self.number in BeatportTrack.__init__: None
In _get_track_info, this is track.number: None
In _get_track_info, this is track.number: None

I tried looking up by track ID using the "Try it" button in the beatport api docs:
Random track id: 12345
https://api.beatport.com/v4/docs/catalog/tracks/{id}/
"number": 1,
A track id as in the debug print stuff above: 1079663
"number": 2,
Interesting! number key is there! But not in the response we actually get from the api.... !?!

Ok, so now some debugging only inside the init method of BeatportTracks and BeatportRelease class. It reveals that a newer tracks api endpoint exists that we should probably use instead of "manually" crafting a track endpoint url from data['slug]:
   prod  (beets)    ~/git/beets-beatport4   main ●  beet import -I -t /remote/data/music-archive/0-andi/Februar\ 2011/Alexander\ Ross\ -\ 01\ -\ Room.mp3
This is self.url in BeatportRelease.__init__: https://beatport.com/release/universal-consciousness-11/217494
'url' key is not available from the api response!

'number' was not found in BeatportTrack.__init__
This is self.url in BeatportRelease.__init__: https://beatport.com/release/universal-consciousness-11/217494
'url' key is not available from the api response!

This is self.number in BeatportTrack.__init__: None
This is self.url in BeatportTrack.__init__: https://beatport.com/track/sometimes-im-blind/1079662
This would be url key in the api response: https://api.beatport.com/v4/catalog/tracks/1079662/

'number' was not found in BeatportTrack.__init__
This is self.url in BeatportRelease.__init__: https://beatport.com/release/universal-consciousness-11/217494
'url' key is not available from the api response!

This is self.number in BeatportTrack.__init__: None
This is self.url in BeatportTrack.__init__: https://beatport.com/track/room/1079663
This would be url key in the api response: https://api.beatport.com/v4/catalog/tracks/1079663/

In _get_track_info, this is track.number: None
In _get_track_info, this is track.number: None

And here I added a debug print that reveals what the actual url is we use to fetch, right inside the lowlevel _get method:
$ beet import -I -t /remote/data/music-archive/0-andi/Februar\ 2011/Alexander\ Ross\ -\ 01\ -\ Room.mp3
In Beatport4Client._get this is the url we are fetching from:
https://api.beatport.com/v4/my/account
In Beatport4Client._get this is the url we are fetching from:
https://api.beatport.com/v4/catalog/search
In Beatport4Client._get this is the url we are fetching from:
https://api.beatport.com/v4/catalog/releases/217494/
This is self.url in BeatportRelease.__init__: https://beatport.com/release/universal-consciousness-11/217494
'url' key is not available from the api response!

In Beatport4Client._get this is the url we are fetching from:
https://api.beatport.com/v4/catalog/releases/217494/tracks/
'number' was not found in BeatportTrack.__init__
This is self.url in BeatportRelease.__init__: https://beatport.com/release/universal-consciousness-11/217494
'url' key is not available from the api response!

This is self.number in BeatportTrack.__init__: None
This is self.url in BeatportTrack.__init__: https://beatport.com/track/sometimes-im-blind/1079662
This would be url key in the api response: https://api.beatport.com/v4/catalog/tracks/1079662/

'number' was not found in BeatportTrack.__init__
This is self.url in BeatportRelease.__init__: https://beatport.com/release/universal-consciousness-11/217494
'url' key is not available from the api response!

This is self.number in BeatportTrack.__init__: None
This is self.url in BeatportTrack.__init__: https://beatport.com/track/room/1079663
This would be url key in the api response: https://api.beatport.com/v4/catalog/tracks/1079663/

In _get_track_info, this is track.number: None
In _get_track_info, this is track.number: None
This is self.url in BeatportRelease.__init__: https://beatport.com/release/universal-consciousness-11/217494
This would be url key in the api response: https://api.beatport.com/v4/catalog/releases/217494/

In Beatport4Client._get this is the url we are fetching from:
https://api.beatport.com/v4/catalog/releases/371304/
This is self.url in BeatportRelease.__init__: https://beatport.com/release/consciousness-ep/371304
'url' key is not available from the api response!

In Beatport4Client._get this is the url we are fetching from:
https://api.beatport.com/v4/catalog/releases/371304/tracks/
'number' was not found in BeatportTrack.__init__
This is self.url in BeatportRelease.__init__: https://beatport.com/release/consciousness-ep/371304
'url' key is not available from the api response!

This is self.number in BeatportTrack.__init__: None
This is self.url in BeatportTrack.__init__: https://beatport.com/track/orient-express/1914120
This would be url key in the api response: https://api.beatport.com/v4/catalog/tracks/1914120/

'number' was not found in BeatportTrack.__init__
This is self.url in BeatportRelease.__init__: https://beatport.com/release/consciousness-ep/371304
'url' key is not available from the api response!

This is self.number in BeatportTrack.__init__: None
This is self.url in BeatportTrack.__init__: https://beatport.com/track/walking-on-stars/1914121
This would be url key in the api response: https://api.beatport.com/v4/catalog/tracks/1914121/

'number' was not found in BeatportTrack.__init__
This is self.url in BeatportRelease.__init__: https://beatport.com/release/consciousness-ep/371304
'url' key is not available from the api response!

This is self.number in BeatportTrack.__init__: None
This is self.url in BeatportTrack.__init__: https://beatport.com/track/dreams/1914122
This would be url key in the api response: https://api.beatport.com/v4/catalog/tracks/1914122/

'number' was not found in BeatportTrack.__init__
This is self.url in BeatportRelease.__init__: https://beatport.com/release/consciousness-ep/371304
'url' key is not available from the api response!

This is self.number in BeatportTrack.__init__: None
This is self.url in BeatportTrack.__init__: https://beatport.com/track/shadows-drop/1914123
This would be url key in the api response: https://api.beatport.com/v4/catalog/tracks/1914123/

In _get_track_info, this is track.number: None
In _get_track_info, this is track.number: None
In _get_track_info, this is track.number: None
In _get_track_info, this is track.number: None
This is self.url in BeatportRelease.__init__: https://beatport.com/release/consciousness-ep/371304
This would be url key in the api response: https://api.beatport.com/v4/catalog/releases/371304/

In Beatport4Client._get this is the url we are fetching from:
https://api.beatport.com/v4/catalog/releases/1149518/
This is self.url in BeatportRelease.__init__: https://beatport.com/release/1-city-1-song/1149518
'url' key is not available from the api response!

In Beatport4Client._get this is the url we are fetching from:
https://api.beatport.com/v4/catalog/releases/1149518/tracks/
'number' was not found in BeatportTrack.__init__
This is self.url in BeatportRelease.__init__: https://beatport.com/release/1-city-1-song/1149518
'url' key is not available from the api response!

This is self.number in BeatportTrack.__init__: None
This is self.url in BeatportTrack.__init__: https://beatport.com/track/1-city-1-song/4670926
This would be url key in the api response: https://api.beatport.com/v4/catalog/tracks/4670926/

In _get_track_info, this is track.number: None
This is self.url in BeatportRelease.__init__: https://beatport.com/release/1-city-1-song/1149518
This would be url key in the api response: https://api.beatport.com/v4/catalog/releases/1149518/

In Beatport4Client._get this is the url we are fetching from:
https://api.beatport.com/v4/catalog/releases/2539993/
This is self.url in BeatportRelease.__init__: https://beatport.com/release/memorias-vol-1-bugandan-sacred-places/2539993
'url' key is not available from the api response!

In Beatport4Client._get this is the url we are fetching from:
https://api.beatport.com/v4/catalog/releases/2539993/tracks/
'number' was not found in BeatportTrack.__init__
This is self.url in BeatportRelease.__init__: https://beatport.com/release/memorias-vol-1-bugandan-sacred-places/2539993
'url' key is not available from the api response!

This is self.number in BeatportTrack.__init__: None
This is self.url in BeatportTrack.__init__: https://beatport.com/track/jinja-on-the-nile/11744564
This would be url key in the api response: https://api.beatport.com/v4/catalog/tracks/11744564/

'number' was not found in BeatportTrack.__init__
This is self.url in BeatportRelease.__init__: https://beatport.com/release/memorias-vol-1-bugandan-sacred-places/2539993
'url' key is not available from the api response!

This is self.number in BeatportTrack.__init__: None
This is self.url in BeatportTrack.__init__: https://beatport.com/track/wood-of-the-lusamba/11744565
This would be url key in the api response: https://api.beatport.com/v4/catalog/tracks/11744565/

'number' was not found in BeatportTrack.__init__
This is self.url in BeatportRelease.__init__: https://beatport.com/release/memorias-vol-1-bugandan-sacred-places/2539993
'url' key is not available from the api response!

This is self.number in BeatportTrack.__init__: None
This is self.url in BeatportTrack.__init__: https://beatport.com/track/mbale/11744566
This would be url key in the api response: https://api.beatport.com/v4/catalog/tracks/11744566/

'number' was not found in BeatportTrack.__init__
This is self.url in BeatportRelease.__init__: https://beatport.com/release/memorias-vol-1-bugandan-sacred-places/2539993
'url' key is not available from the api response!

This is self.number in BeatportTrack.__init__: None
This is self.url in BeatportTrack.__init__: https://beatport.com/track/fresh-ice-cream/11744567
This would be url key in the api response: https://api.beatport.com/v4/catalog/tracks/11744567/

'number' was not found in BeatportTrack.__init__
This is self.url in BeatportRelease.__init__: https://beatport.com/release/memorias-vol-1-bugandan-sacred-places/2539993
'url' key is not available from the api response!

This is self.number in BeatportTrack.__init__: None
This is self.url in BeatportTrack.__init__: https://beatport.com/track/albert-plays-the-flute/11744568
This would be url key in the api response: https://api.beatport.com/v4/catalog/tracks/11744568/

'number' was not found in BeatportTrack.__init__
This is self.url in BeatportRelease.__init__: https://beatport.com/release/memorias-vol-1-bugandan-sacred-places/2539993
'url' key is not available from the api response!

This is self.number in BeatportTrack.__init__: None
This is self.url in BeatportTrack.__init__: https://beatport.com/track/binding-the-arched-harp/11744569
This would be url key in the api response: https://api.beatport.com/v4/catalog/tracks/11744569/

'number' was not found in BeatportTrack.__init__
This is self.url in BeatportRelease.__init__: https://beatport.com/release/memorias-vol-1-bugandan-sacred-places/2539993
'url' key is not available from the api response!

This is self.number in BeatportTrack.__init__: None
This is self.url in BeatportTrack.__init__: https://beatport.com/track/the-story-of-walumbe/11744570
This would be url key in the api response: https://api.beatport.com/v4/catalog/tracks/11744570/

'number' was not found in BeatportTrack.__init__
This is self.url in BeatportRelease.__init__: https://beatport.com/release/memorias-vol-1-bugandan-sacred-places/2539993
'url' key is not available from the api response!

This is self.number in BeatportTrack.__init__: None
This is self.url in BeatportTrack.__init__: https://beatport.com/track/boda-has-no-radio/11744571
This would be url key in the api response: https://api.beatport.com/v4/catalog/tracks/11744571/

In _get_track_info, this is track.number: None
In _get_track_info, this is track.number: None
In _get_track_info, this is track.number: None
In _get_track_info, this is track.number: None
In _get_track_info, this is track.number: None
In _get_track_info, this is track.number: None
In _get_track_info, this is track.number: None
In _get_track_info, this is track.number: None
This is self.url in BeatportRelease.__init__: https://beatport.com/release/memorias-vol-1-bugandan-sacred-places/2539993
This would be url key in the api response: https://api.beatport.com/v4/catalog/releases/2539993/

In Beatport4Client._get this is the url we are fetching from:
https://api.beatport.com/v4/catalog/releases/386930/
This is self.url in BeatportRelease.__init__: https://beatport.com/release/every-night-pt-1/386930
'url' key is not available from the api response!

In Beatport4Client._get this is the url we are fetching from:
https://api.beatport.com/v4/catalog/releases/386930/tracks/
'number' was not found in BeatportTrack.__init__
This is self.url in BeatportRelease.__init__: https://beatport.com/release/every-night-pt-1/386930
'url' key is not available from the api response!

This is self.number in BeatportTrack.__init__: None
This is self.url in BeatportTrack.__init__: https://beatport.com/track/every-night/1991846
This would be url key in the api response: https://api.beatport.com/v4/catalog/tracks/1991846/

'number' was not found in BeatportTrack.__init__
This is self.url in BeatportRelease.__init__: https://beatport.com/release/every-night-pt-1/386930
'url' key is not available from the api response!

This is self.number in BeatportTrack.__init__: None
This is self.url in BeatportTrack.__init__: https://beatport.com/track/every-night/1991847
This would be url key in the api response: https://api.beatport.com/v4/catalog/tracks/1991847/

In _get_track_info, this is track.number: None
In _get_track_info, this is track.number: None
This is self.url in BeatportRelease.__init__: https://beatport.com/release/every-night-pt-1/386930
This would be url key in the api response: https://api.beatport.com/v4/catalog/releases/386930/

/remote/data/music-archive/0-andi/Februar 2011/Alexander Ross - 01 - Room.mp3 (1 items)
Correcting tags from:
    Alexander Ross - Universal Consciousness 1.1
To:
    Argy, Alexander Ross - Universal Consciousness 1.1
URL:
    https://beatport.com/release/universal-consciousness-11/217494
(Similarity: 78.1%) (media, missing tracks, artist, source, tracks) (Beatport, Digital, 2010, These Days, TD05)
 * Room (#1) -> Room (#None) (source)
Missing tracks (1/2 - 50.0%):
 ! Sometimes I'm Blind (#None) (6:31)
[A]pply, More candidates, Skip, Use as-is, as Tracks, Group albums,
Enter search, enter Id, aBort? b

Hey @JOJ0, thank you for help with debugging it!
Looks like the issue lays in the fact, that beatport does not return the number field in /catalog/releases/{releaseId}/tracks call. To fix this, I just added additional call to /catalog/tracks/{trackId} where the number is already present. This commit fixes this:  https://github.com/Samik081/beets-beatport4/commit/9071e60c0730990fc7ca039a0bae0324bb6ac613, released in https://github.com/Samik081/beets-beatport4/releases/tag/v0.2.6


net: tcp: fix spurious TCP retries

Spurious TCP retries were observed using Wireshark while continuously
sending TCP packets at an interval faster than the initial RTO.
If the send list is empty and CONFIG_NET_TCP_TIME_WAIT_DELAY is used,
the retry timer will not be correctly stopped when receiving a valid
ACK. As a consequence, the timer might be running when a new packet is
queued, but the logics in net_tcp_queue_data() will not restart the
timer as it is already running. This will make the retry timer to expire
prematurely, potentially while sending packets.
The nested condition is merged into a single condition, allowing the
final else clause to be reached when a valid ACK is received.
Signed-off-by: Florian Vaussard<EMAIL_ADDRESS>

@vaussard : Looks interesting. Are you aware that there're more issues in that stuff, e.g. https://github.com/zephyrproject-rtos/zephyr/issues/8188 . And sent_list itself looks like might be able to be corrupted due to concurrency issues, as weird debug traces in https://github.com/zephyrproject-rtos/zephyr/issues/5857 show.

I do not think that I already encountered #5857. Maybe #8188 already happened because I know that we had unexpected TCP disconnections sometime. My first explanation was to blame the retry logics because one of the hardware setup is a very nice collision generator. That's why I looked more carefully into the retry mechanisms. I will see if #8188 pops up in our tests.

@vaussard

I do not think that I already encountered #5857. Maybe #8188 already happened because I know that we had unexpected TCP disconnections sometime. My first explanation was to blame the retry logics because one of the hardware setup is a very nice collision generator. That's why I looked more carefully into the retry mechanisms. I will see if #8188 pops up in our tests.

Thanks. First step is indeed getting a confirmation from other folks that they see same/similar problematic behavior. Myself, I couldn't find time to concentrate on further investigating them, with floodwork of refactoring and new features Zephyr has :-(. For your reference, #5857 was spotted while playing with MicroPython's Zephyr port. Using interactive interpreted language makes playing with networking easy, and it doesn't take long to drive the IP stack into havoc.

From commit message:

Spurious TCP retries were observed using Wireshark while continuously
sending TCP packets at an interval faster than the initial RTO.

Based on this description, it's unclear, whether packets were send from or to Zephyr.

Based on this description, it's unclear, whether packets were send from or to Zephyr.

Packets were sent from Zephyr to Zephyr :) The sending node resent some packets without letting the other node enough time to ACK the packets. Do you want me to update the description to be more clear?

Do you want me to update the description to be more clear?

Up to you, and thanks for considering that option ;-). In this case, it's probably not too important, as the rest of the commit message explains the situation with the code.


Convert examples to using the NLSE.pulseData.plot function

This will make the examples more concise.

taken care of by #41.


better handle non eks optimized amis

Description of changes:
When running the log collector script on OS images other than the standard AL2/23 eks optimized ami and without the vpc-cni, there are a few potentially confusing log messages due to missing files/execs.  This adds:

command -v checks around specific bins, lvs pvs vgs ipvsadm ipset conntrack aws-eks-na-cli similar to others within this script.
Skips saving ipam information if /var/run/aws-node/ipam.json does not exist.

The one somewhat functional change is to handle Ubuntu nodes.  Ubuntu is INIT_TYPE snap in the script and when kubelet logs are retrieve the snap kubelet-eks is assumed to exist. The change checks for existence of the kubelet-eks snap and if it does not exist, falls back to the normal kubelet log collecting. Without this change, the kubelet logs would not be collected for these kinds of nodes.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Testing Done

Successfully ran on the al23 optimized ami
Successfully ran on Ubuntu EKS Optimized ami to ensure snap still functions as before
Successfully ran on Ubuntu and Rhel nodes with the standard set of bins

Old Output:
        This is version 0.7.8. New versions can be found at https://github.com/awslabs/amazon-eks-ami/blob/main/log-collector-script/

Trying to collect common operating system logs...
Trying to collect kernel logs...
Trying to collect modinfo... modinfo: ERROR: Module lustre not found.
Trying to collect mount points and volume information...
Trying to collect SELinux status...
Trying to collect iptables information... Trying to collect ipvs information... script.sh: line 358: ipvsadm: command not found
sed: -e expression #1, char 18: unknown option to `s'
script.sh: line 359: ipvsadm: command not found

script.sh: line 361: ipvsadm: command not found
script.sh: line 362: ipset: command not found

script.sh: line 364: ipset: command not found

Trying to collect installed packages...
Trying to collect active system services...
Trying to Collect Containerd daemon information...
Trying to Collect Containerd running information...
Trying to Collect Docker daemon information...

        Warning: The Docker daemon is not running.

Trying to collect kubelet information... error: snap "kubelet-eks" not found

Trying to collect nodeadm information...

        Warning: The current operating system is not supported.

Trying to collect L-IPAMD introspection information... Trying to collect L-IPAMD prometheus metrics... Trying to collect L-IPAMD checkpoint... cp: cannot stat '/var/run/aws-node/ipam.json': No such file or directory

Trying to collect Multus logs if they exist...
Trying to collect sysctls information...
Trying to collect networking infomation... timeout: failed to run command 'conntrack': No such file or directory
timeout: failed to run command 'conntrack': No such file or directory
timeout: failed to run command 'conntrack': No such file or directory
timeout: failed to run command 'ifconfig': No such file or directory

Trying to collect CNI configuration information...
Trying to collect CNI Configuration Variables from Docker...

        Warning: The Docker daemon is not running.
Trying to collect CNI Configuration Variables from Containerd...        Timed out, ignoring "cni configuration variables output "

Trying to collect network policy ebpf loaded data... script.sh: line 567: /opt/cni/bin/aws-eks-na-cli: No such file or directory

Trying to collect Docker daemon logs...
Trying to Collect sandbox-image daemon information...
Trying to Collect CPU Throttled Process Information...
Trying to Collect IO Throttled Process Information...
Trying to Collect reboot history...
Trying to Collect Nvidia Bug report... No Nvidia drivers found, nothing to do.

Trying to archive gathered information...

        Done... your bundled logs are located in /var/log/eks_i-0be40b1aeed0c7986_2024-11-21_1755-UTC_0.7.8.tar.gz

New Output:
	        This is version 0.7.8. New versions can be found at https://github.com/awslabs/amazon-eks-ami/blob/main/log-collector-script/

Trying to collect common operating system logs...
Trying to collect kernel logs...
Trying to collect modinfo... Trying to collect mount points and volume information...
Trying to collect SELinux status...
Trying to collect iptables information...
Trying to collect installed packages...
Trying to collect active system services...
Trying to Collect Containerd daemon information...
Trying to Collect Containerd running information...
Trying to Collect Docker daemon information...

        Warning: The Docker daemon is not running.

Trying to collect kubelet information...
Trying to collect nodeadm information...

        Warning: The current operating system is not supported.

Trying to collect L-IPAMD introspection information... Trying to collect L-IPAMD prometheus metrics... Trying to collect L-IPAMD checkpoint...
Trying to collect Multus logs if they exist...
Trying to collect sysctls information...
Trying to collect networking infomation...
Trying to collect CNI configuration information...
Trying to collect CNI Configuration Variables from Docker...

        Warning: The Docker daemon is not running.
Trying to collect CNI Configuration Variables from Containerd...        Timed out, ignoring "cni configuration variables output "

Trying to collect network policy ebpf loaded data...
Trying to collect Docker daemon logs...
Trying to Collect sandbox-image daemon information...
Trying to Collect CPU Throttled Process Information...
Trying to Collect IO Throttled Process Information...
Trying to Collect reboot history...
Trying to Collect Nvidia Bug report... No Nvidia drivers found, nothing to do.

Trying to archive gathered information...

        Done... your bundled logs are located in /var/log/eks_i-0be40b1aeed0c7986_2024-11-21_1753-UTC_0.7.8.tar.gz

See this guide for recommended testing for PRs. Some tests may not apply. Completing tests and providing additional validation steps are not required, but it is recommended and may reduce review time and time to merge.

@jaxesn linter found some trailing spaces: https://github.com/awslabs/amazon-eks-ami/actions/runs/11981467999/job/33628858679?pr=2073#step:5:82
/ci


EDU-1086 simplify translation handling

Closes https://educandu.atlassian.net/browse/EDU-1086

It is indeed simplified, nice! 👏


Remove the use of utils in cora lp demo

This doesn't remove the actual utils directory yet, since it contains the metapath2vec demo code which needs to be notebook-ified, but this does remove the use of utils in the node2vec demo. Some of the markdown in the notebook was also updated to explain more bits of code that have ended up in the notebook.
See #896

I've made a few more adjustments to make it more closely follow what was there before... but the results seem to have gotten worse actually - I'll look into it a bit further, but what I've gathered so far:

dataset loading

Previously used networkx to obtain largest connected component which gave Largest subgraph statistics: 2485 nodes, 5069 edges
With dataset.load(largest_connected_component_only=True, str_node_ids=True) we get more edges: Nodes: 2485, Edges: 5209

edge splitter

previously used edge splitter with networkx graphs
now used with StellarGraph objects

Training Word2Vec model

Previously used some kind of custom node2vec code to run random walks (in utils/node2vec)
Now using BiasedRandomWalk class in the library

we get more edges

I think there's multi-edges in the Cora dataset, which get eliminated with a nx.Graph but not with the multigraph StellarGraph.

previously used edge splitter with networkx graphs, now used with StellarGraph objects

The StellarGraph "implementation" is just doing conversions, so hopefully this shouldn't affect it too much:
https://github.com/stellargraph/stellargraph/blob/d9e49e1810a7f999e04ad29283aae4222b2ce28c/stellargraph/data/edge_splitter.py#L64-L71

I tried a switch to compare results using BiasedRandomWalk vs. the biased random walk in utils, and still getting the same (different from original) results, so will try reducing the multi-edges now to see if that helps. Seems a bit odd to me that a citation network contains multi-edges?

Also tried:

removing the multi-edges, which does make the number of edges match the original, but doesn't improve the scores.
tried using the entire set of data/graph loading cells from old notebook, but still no luck

The problem has to be somewhere before the link classifier training, since the embedding visualisation still doesn't look correct, and that only uses the node2vec features + binary operator
So now going to open both notebooks side by side and go through each cell...

Turned out to be a silly typo.. walk_length was set incorrectly. Now the plot looks good:

@PantelisElinas It'd be good to get your review on this one, since I'm not entirely sure if there's enough explanation for the train-test splitting in the notebook... the splits are:

we create graph_test and test examples/labels using edge splitter on graph
we create graph_train and training examples/labels using edge splitter on graph_test, with graph as the master so that the negative edge sampling doesn't accidentally sample a positive edge from graph.
Using a node2vec model trained on graph_train we:

split the training examples/labels further into train/test sets within this set
train the classifier on the train-train examples/labels
evaluate the classifier on the train-test examples/labels

Using a node2vec model trained on graph_test we:

evaluate the classifier on these test examples/labels

I'm not sure if I got my head around why we need to evaluate do another split on the training set to evaluate the classifier, instead of just evaluating on the original test set? What roughly made sense to me was that we want scores using embeddings from both graph_train and graph_test, so that's the explanation I've included as a comment in the notebook, but not sure if that's the full/correct story.

@kjun9

I'm not sure if I got my head around why we need to evaluate do another split on the training set to evaluate the classifier, instead of just evaluating on the original test set?

The reason you need the extract split is for selecting the binary operator. You shouldn't use your test data to compare operators and decide on the best one to use.
So, given the train graph and the training edge data, you do a 75/25 split, you use 75% of the data to train you binary classifier and 25% of the data to evaluate its performance for a given binary_operator. This value you store in score_graph_train. Using the same split of data you perform the same operation on the next available binary_operator and so on. Now, you can compare performance between the operators and select the best performing one. Then you evaluate your model's performance on the test set. The idea here is that your test data should not be used for model selection, i.e., in this case selecting the binary_operator. You can further select a better model tuning any of the Node2Vec parameters but we don't show that in this notebook; again, for this you would need that additional 75/25 split of your training data.
The "error" in your run_link_prediction(binary_operator) is that it should just return score_graph_train and not proceed with calculating score_graph_test. Once you have the score_graph_train values for all binary_operators then you can select the best performing one and calculate and report score_graph_test. The reason I said "error" and not error is that there is nothing wrong with evaluating all operators on the test data, unless you want to do model selection which I think is what this notebook should demonstrate and hence should be updated accordingly.
One important detail is when you do the 75/25 split, make sure that it is done once and all operators are trained and evaluated on the same data. Otherwise, it is not exactly correct to use the calculated score_graph_train values to select the best operator; just in case one operator receives a split of the data that is easier to classify.
Let me know if this doesn't clarify things.
BTW, the rest of the notebook looks really good.

The reason you need the extract split is for selecting the binary operator ... The idea here is that your test data should not be used for model selection, i.e., in this case selecting the binary_operator. You can further select a better model tuning any of the Node2Vec parameters but we don't show that in this notebook

Ah I see! Okay I'll update the notebook to mention more explicitly that the 75/25 split is for model selection and that in this notebook we are demonstrating model selection with a single variable parameter being the binary operator. Then only use the best operator for calculating score_graph_test.

One important detail is when you do the 75/25 split, make sure that it is done once and all operators are trained and evaluated on the same data.

Makes sense 👍  I'll update the notebook

@huonw @PantelisElinas this is ready for review again - I've tried to incorporate more explanation of the splits in the notebook


Unable to place tank in Chemical Crystallizer using automation

The Chemical Crystallizer is unable to accept a tank of clean ore slurry through any means of automation, even though it has a purple item slot that should accept it.  Placing the tank in by hand works fine.
On another note, it would also be extremely useful to be able to condensate clean ore slurries into liquid form.

I think the gas tanks are literally handy :-)
I guess you've separated processes, have a large distance between machinery/contraptions? That's why you choose to use gas tanks instead of direct input/piping?
(I have my chemical machinery lined up next to each other, feeding the next one directly)

Actually, kind of the opposite.  With 5x ore processing is the way that mekanism gas tubes distribute clean slurry to the crystallizers leads to a small amount getting left over in the crystallizer,  jamming the whole system up if a different type of clean slurry comes through.
My plan was to have a bunch of empty tanks basically work in a loop - the tanks fill up with clean slurry, get picked up when full (was gonna use a SFM block gate, but it can't break the tanks apparently), then deposit the tank into a crystallizer.  Since a full tank has an even amount of slurry in it, the crystallizer completely processes all the slurry and is empty.  Once the tank was empty, it would get sucked out and get back into the loop to be refilled with more slurry.
The thing I am trying to avoid is having a different set of crystallizers for each type of ore slurry.  It would be incredibly messy and a huge waste of both in-game and server resources.

Aha, that would be nice indeed if would work like that for that kind of situation.
I think I'm used to set up more Mek machinery for convenience (each ore has it's own line of chemical machines) and/or put up with manually configuring/finetuning at the start or during the process. Also: I'm on single-player w/small modpack. But I'm feeling your need :-)

honestly if your at 5x process time the ammount of "waste" is minimal physically.. now size wise it is.. but honestly there is no way to prevent this unless it can process only one ore at a time ..
my advice is to store the slurry and run it though when you know you have full runs without leftovers ..

Thanks for the pointer, this was definitely not intentional.  Crystalizers can now accept gas tanks.


Fixed flatten array argument name in docs

Side note, what's the reasoning of not being arr in the first place?

Good catch, thanks!

what's the reasoning of not being arr in the first place?

Good question, I've opened up #4 to continue that line of discussion.


Javascript plugins not working properly with turbolinks

I'm using share buttons (javascript plugins). When I open a page it shows that buttons, but when I open other page it disappear. It appear again when i refresh website. It working properly with jquery.turbolinks and without turbolinks gem, but without it website is very slow. How to make website faster? My website, now plugins is working, because I removed turbolinks.
application.js
//= require jquery
//= require jquery.turbolinks
//= require jquery_ujs
//= require lightbox
//= require_tree .
//= require turbolinks

application.html.erb javascript plugins
<script type="text/javascript" src="//s7.addthis.com/js/300/addthis_widget.js#pubid=ra-588e44a9401a6d9f"></script>
<script type="text/javascript" id="st_insights_js" src="http://w.sharethis.com/button/buttons.js?publisher=630cdfb3-b6c5-4da7-a389-1c7d5340827d"></script>
<script type="text/javascript">stLight.options({publisher: "630cdfb3-b6c5-4da7-a389-1c7d5340827d", doNotHash: false, doNotCopy: false, hashAddressBar: false});</script>

Gemfile
gem 'jquery-turbolinks'
gem 'turbolinks', '~> 5.0.1'

I tried to add one plugin in javascript file, but result same
$(document).on('turbolinks:load', function(){
  stLight.options({publisher: "630cdfb3-b6c5-4da7-a389-1c7d5340827d",      doNotHash: false, doNotCopy: false, hashAddressBar: false});
});

Thank you for any help.

#62
In this issue find  compatibility.coffee script and add to project


Update NCEP TDS data access URLs

Replace the old ncei.noaa.gov TDS URLs I found with their new ncdc.noaa.gov counterparts as described in (and closes) #81. Not sure if this is the preferred workflow for python-training or if anything else needs to change to squeeze this in. Let me know!

So I wouldn't describe the NCDC links as "new". NCDC (and a few other centers) were renamed to NCEI. I'll also add that the TDS at the NCDC URLs is a few revisions behind that available at NCEI.
Having said that, the most important aspect is that the NCDC links actually work.


agate: 3.3.9 → 3.3.10

https://github.com/mbrubeck/agate/releases/tag/v3.3.10
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[x] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review pr 354518

x86_64-linux

:white_check_mark: 1 package built:

agate


Added Linear Regression using PyTorch

Please read how to contribute first!
Description
Submission description here
Checklist

[x] I've checked the issue list BEFORE deciding what to submit
[x] I've read the contribution guidelines
[x] I've edited the README.md link to my code

Don't forget to visit our website!
Feel free to add this badge after your submission is accepted!

Awesome .Looks good Thank you

@anushkrishnav  @HarshCasper  Can you add me to the list of contributors ?

@farazkhanfk7 You have been added .  Its automated :) . Have a nice day

@anushkrishnav  I thought it was using allcontributors bot.

Yes it is. I meant we will automaticaly add it DW abt it . Have  a great day

@all-contributors please add @farazkhanfk7 for code


known Parcel/build issues

issues bundling client on watch mode, so right now we're resorting to restarting client everytime server is bundled (https://github.com/parcel-bundler/parcel/issues/1286)
code splitting isn't working yet (https://github.com/parcel-bundler/parcel/issues/905)
need to configure static dir for production

Is this related to the following?
Browser:

Console:

Yeah I think so
client doesn't seem to be rebundling, so HMR isn't working and that's causing a client-server mistmatch. I'll have to fix this


Can we use passive event listeners for a bump in scroll performance?

The video on this site is a nice comparison, although these may be geared more for touch devices
https://medium.com/@devlucky/about-passive-event-listeners-224ff620e68c

For the main scroll scenario - scrolling a Window in Neovim - we actually handle that really outside the browser infrastructure. So the passive event listeners unfortunately won't be helpful there.
There are a few proposals in #88. The bottleneck at the moment is rendering - we basically re-render every line that changes, which since most lines move in a scroll, is a lot of rendering!
There are definitely some ways we can improve that, though. Canvas even supports a way to translate a section of the canvas, which would be worth trying out.


Sometimes fiat value appear as 0 but should show none info or other kind of rendering

Describe the bug
Sometimes I open Alby and I look to see how much the sats I have is representing in fiat. But sometimes maybe because it take some seconds to load or maybe because the fiat quote provider service is offline, the value appear as 0.
Screenshots of the changes [optional]
This screenshot is working ok, I couldn't take a screenshot reproducing the problem

To Reproduce
The problem is hard to reproduce, maybe when fiat quote provider is slow or offline
Expected behavior
Not show 0 but instead show none information or some other kind of rendering.
Alby information

Alby Version: 1.17

Are you working on this issue?
No

I feel it shows 0 because initialState of currencyRate is set to be 0
https://github.com/getAlby/lightning-browser-extension/blob/49705ab42ed5f9efaeb4c1d21c7832610f0d1e2e/src/app/context/SettingsContext.tsx#L30
Initial stage of fiatBalance :
https://github.com/getAlby/lightning-browser-extension/blob/5156710bce3f21ce543a0f2fda89f0db2e724318/src/app/context/AccountContext.tsx#L53

@bitcoinuser is the 0 usually replaced by a fiat value after some time/seconds or does it stay?
If this just shows during loading we could i.e. just show ... if fiat is 0 as a first fix. No need to add a spinner I think.

yes, esp. in slow connections this happens and is visible.

I feel we can change this with below mentioned changes in the image
Current Code:
https://github.com/getAlby/lightning-browser-extension/blob/5156710bce3f21ce543a0f2fda89f0db2e724318/src/app/components/AccountMenu/index.tsx#L95-L97
Proposed Changes:

Hm, currency depends on the settings. So checking for i.e. $ will not cover all cases, right?
Play around with the currencies in "settings".
Wanna check what solution might work better here?

Hm, currency depends on the settings. So checking for i.e. $ will not cover all cases, right?

Oh I forgot that one. Will make it dynamic.

@escapedcat I have opened a PR for this issue with the proposed changes. Can assign this issue to me?


Netlify form native submission does not work

I noticed that I cannot seem to get native Netlify forms to work. I do not get a default response message either from the native form.
        <form
          name="Frontend Monday"
          method="post"
          class="HeroForm"
          data-netlify="true"
          data-netlify-honeypot="bot-field"
        >
          <input type="hidden" name="bot-field" />
          <input type="text" name="name" placeholder="Name" />
          <input type="email" name="email" placeholder="" />
          <button type="submit">Send</button>
        </form>

Any thoughts?

Would love a response on this. Have had the same issue and haven't found a solution for several days.

Hi @willlma,
Try creating a component for your form and changing the form to this:
<form
    name="HeroFormX"
    method="post"
    action="/success"
    data-netlify="true"
    data-netlify-honeypot="bot-field"
    className="HeroForm"
  >
    <input type="hidden" name="bot-field" />

      Name
      <input type="text" name="name" id="name" required />

      Email
      <input type="text" name="email" id="email" required />

      <input type="submit" value="Sign up" className="HeroFormBtn" />

  </form>


Simplify installation of needed resources

We can provide a jsonnet package to install all related resources in order to get user started (while for advanced user they can still use their own without running this installation). The package can include followings:

secrets (if specified): github token, gcp creds
nfs service: include an nfs server and corresponding service
volume: nfs pv & pvc

/priority p1

/assign


*** buffer overflow detected ***: ./ultra_simple terminated

Hello~
I use order ./ultra_simple --channel -s ttyUSB0 256000 to test the RPLIDAR S1 on jetson nano, but en error occurred that
Ultra simple LIDAR data grabber for SLAMTEC LIDAR.
Version: SL_LIDAR_SDK_VERSION
*** buffer overflow detected ***: ./ultra_simple terminated
Aborted (core dumped)

l have had use the order 'sudo chmod a+rw /dev/ttyUSB0' to set the UART permission
There is any solution about this problem?

I am using the S2 and had to run it as sudo and also specify the baudrate as 1000000 -- this is specific to the LiDAR you're using.

Hi tcexeexe,
Did you manage to resolve the problem?
I am facing the same problem with A2.
Using ./ultra_simple --channel --serial ttyUSB0 115200 in my case.
Throwing the same error as you have.
Anyone with solution?

@LDec-AlmostHuman I'm jumping into the thread because I have an A2M12 as well and have been getting the buffer overflow error. Although, I seemed to have reached a state where I am back to a "cannot bind to the specified serial port /dev/ttyUSB0" error that I had been experiencing before.
If I'm still having issues after I get through that, can I call on you to dig up the solution you found?

@LDec-AlmostHuman I'm jumping into the thread because I have an A2M12 as well and have been getting the buffer overflow error. Although, I seemed to have reached a state where I am back to a "cannot bind to the specified serial port /dev/ttyUSB0" error that I had been experiencing before.
If I'm still having issues after I get through that, can I call on you to dig up the solution you found?

I will do that on Sunday, the latest if you don't mind. My setup was Ubuntu 20.04 and not in docker. Having problems with docker exposing /dev even with --privileged, but when I will fix that issue I will post as well docker version of it.

I was able to get this working on my raspberry pi 4 running ubuntu 22.04 without having to chmod /dev/ttyUSB0 or add myself to the dialout group. I did have to run  "./ultra_simple --channel --serial /dev/ttyUSB0 256000" as sudo though as a result.
I'm still having issues getting it to work on my jetson nano, which is running xubuntu 20.04. I'll have to look into where I went wrong there.

@totenkatze
Please see below fix for Ubuntu 20.04 & Ubuntu 22.04.
I have tested it on Jetson NX with Ubuntu 20.04 and AMD PC with both Ubuntu 20.04 & Ubuntu 22.04.
Tested on Ubuntu 20.04 without problems.
====================================================
On Ubuntu 22.04, after getting
git clone https://github.com/Slamtec/rplidar_sdk.git
you will need to correct line 175 in net_socket.cpp, which is in rplidar_sdk/sdk/src/arch/linux
Replace:
return ans<=0?RESULT_OPERATION_FAIL:RESULT_OK;
with:
return ans == nullptr ? RESULT_OPERATION_FAIL : RESULT_OK;
save the file, get back to rplidar_sdk directory and now should compile.
====================================================
I have not tested it to see if it works on MacOS (Silicone)
I have not tested it to see if it works on any Windows flavors.
====================================================
RPLidar A2M12, which has 256000 communication speed.
Change the speed to 256000 with the switch on the device.
Check if you can see CP2102 USB to UART Bridge Controller
Run:
usb-devices
It should return something like this, that means that the driver is loaded.
If not, look a few lines below for installation instructions from Slamtec and start from the beginning.
T:  Bus=05 Lev=01 Prnt=01 Port=02 Cnt=01 Dev#=  4 Spd=12  MxCh= 0
D:  Ver= 1.10 Cls=00(>ifc ) Sub=00 Prot=00 MxPS=64 #Cfgs=  1
P:  Vendor=10c4 ProdID=ea60 Rev=01.00
S:  Manufacturer=Silicon Labs
S:  Product=CP2102 USB to UART Bridge Controller
S:  SerialNumber=0001
C:  #Ifs= 1 Cfg#= 1 Atr=80 MxPwr=100mA
I:  If#= 0 Alt= 0 #EPs= 2 Cls=ff(vend.) Sub=00 Prot=00 Driver=cp210x
E:  Ad=01(O) Atr=02(Bulk) MxPS=  64 Ivl=0ms
E:  Ad=81(I) Atr=02(Bulk) MxPS=  64 Ivl=0ms
Run below line to see ownership and which ttyUSBx you see
ls -l /dev | grep ttyUSB
It should return something like this:
crw-rw----   1 root  dialout 188,     0 Mar 11 06:44 ttyUSB0
Change permissions to that device (others are suggesting using 666, but for me only 777 worked):
sudo chmod 666 /dev/ttyUSB0
sudo chmod 777 /dev/ttyUSB0
Run again:
ls -l /dev | grep ttyUSB
Now should return changed permission:
crwxrwxrwx   1 root  dialout 188,     0 Mar 11 06:44 ttyUSB0
Add current user to dialout:
sudo adduser $USER dialout
Installation from Slamtec to install drivers
If usd-devices doesn't return any info on Product=CP2102 USB to UART Bridge Controller - to install drivers.
You will need to start the above procedure from the beginning.
git clone https://github.com/Slamtec/rplidar_sdk.git
cd rplidar_sdk
Install ament_cmake if you don't have one installed already - my installation was missing ament-cmake and was throwing errors
sudo apt-get update
sudo apt-get -y install ament-cmake
make
If no errors we can run what was compiled:
cd output/Linux/Release/
Now test what the actual speed of the interface is working correctly.
./custom_baudrate  /dev/ttyUSB0 256000
In return you should get something like below.
Baudrate negotiation demo for SLAMTEC LIDAR.
Try to establish communication to the LIDAR using the baudrate at /dev/ttyUSB0@256000 ...
The baudrate detected by the pair is 257175 bps. Error: 0.459 %
Wow, we just communicate with the LIDAR using a non-standard baudrate : 256000!.
SLAMTEC LIDAR S/N: (32 characters serial number)
Firmware Ver: 1.32
Hardware Rev: 6
As you can see it throws an Error: 0.459 % in my case.
As baudrate is 257175 bps in my case, so we will correct the line.
Please use your baudrate for all below tests.
./custom_baudrate /dev/ttyUSB0 257175
Now it returned without any errors:
Baudrate negotiation demo for SLAMTEC LIDAR.
Try to establish communication to the LIDAR using the baudrate at /dev/ttyUSB0@257175 ...
The baudrate detected by the pair is 257175 bps. Error: 0.000 %
Wow, we just communicate with the LIDAR using a non-standard baudrate : 257175!.
SLAMTEC LIDAR S/N: (32 characters serial number)
Firmware Ver: 1.32
Hardware Rev: 6
Now let's test simple_grabber
./simple_grabber --channel --serial /dev/ttyUSB0 257175
It should return something like this:
========================================
Lidar health status : OK. (errorcode: 0)
waiting for data...
                             *       *
                   * * *     *       *
                   * * *     *       *
                   * * *     *       *
                   *** **    *       *
                   *** **    *       * *

Do you want to see all the data? (y/n)
For more data you can press y and will show some more data and will stop.
To stop press n and will stop.
Now let's test ultra_simple
./ultra_simple --channel --serial /dev/ttyUSB0 257175
It should return something like this:
========================================
theta: 0.00 Dist: 00000.00 Q: 0
theta: 0.23 Dist: 00000.00 Q: 0
theta: 0.47 Dist: 00000.00 Q: 0
theta: 0.70 Dist: 00000.00 Q: 0
theta: 0.93 Dist: 00000.00 Q: 0
theta: 1.17 Dist: 00000.00 Q: 0
theta: 1.40 Dist: 00000.00 Q: 0
....
theta: 358.59 Dist: 00000.00 Q: 0
theta: 358.83 Dist: 00000.00 Q: 0
theta: 359.06 Dist: 00000.00 Q: 0
theta: 359.30 Dist: 00000.00 Q: 0
theta: 359.53 Dist: 00000.00 Q: 0
theta: 359.76 Dist: 00000.00 Q: 0
I hope all works:)
If you are interested I can post as well corrected python files for git clone https://github.com/Slamtec/sllidar_ros2.git

@totenkatze
I was testing it with Jetson Nano with JetPack 4.6.1 and I have managed to spin it, but couldn't see results as Nano froze and since I have a problem with powering via jack, only powering via USB seems to work. PSU is 5.2V 4A under full load and 5.22V without any load. No idea what happened, but will have time to work on it by the end of the month so will post as well Nano instructions.
BTW are you thinking of switching to Orin Nano or Orin NX 😄? Cannot wait to get my hands on it. I was thinking of AGX Orin dev kit, but it's a bit too pricey for my current budget. Despite Orin NX with carrier board will be around 1.000€ for 16GB model. I'm still in the dark with PCIE 4 on carrier board. All available now seem to be PCIE 3. All my NVME drives are PCIE 4. I know they will work, but ... 😄.

@LDec-AlmostHuman Thank you for sharing the instructions above. I'll give them a try on my nano this weekend. I have tried most of the items in the instructions without success, but I'm curious to see if adjusting the baud rate would help me out.
Getting things running so effortlessly on my Pi with 22.04 while still having such difficulty on the nano makes be feel a bit less crazy.
I haven't looked into the Orin yet. I'm just getting started with robotics as a hobby. I have a few old Raspberry Pis, two Jetson Nanos, and an Orange Pi 5 on the way. With the $XXXX dollars I've spent on robots and parts, I think that I should have just starting doing cocaine to save a bit of money.

@totenkatze
Maybe Nano has not enough juice to spin the Lidar.
Try to connect power to Lidar to other source of power. I can try to install Docker image of Xubuntu 20.04 on Jetson NX and try to see if it works. I don't have second Nano. And will have time to fix Nano in circa 2 weeks.
You are not the only one with $xxxx spent on robotics hobby projects 😄. And I actually like doing such projects.
Next week I was thinking of running both A2M12 and OAK-D S2 to see how would that setup work.

@totenkatze
Just tested on Jetson Nano with Xubuntu 20.04 and works without problems. I have followed instructions on installing Xubuntu from forums.developer.com
Can send you link to it. All together took me a bit over an hour, but all works. Lidar is spinning etc.

Anyone succeed with my guidance? If not please let me know.

@totenkatze
They are confusing B01 😄. Boots from SDCard without issues. It hangs on USB, but I think I know what's the problem is. I have all other devices plugged in and it seems I need to boot for the first time with only USB drive plugged in. Didn't do it yet, just found out that from one of the posts. Let me know if you succeeded with RPLidar.

@totenkatze
Someone wrote on NVidia forum that there might be some issues on B01 when you are using A01 in extlinux. Maybe this is the problem.

@LDec-AlmostHuman I started from fresh install of xubuntu and installed git, make, g++, etc. and after the compile, it worked when running as sudo. I chmod /dev/ttyUSB0 to 666 (any form of XX6 works) and it worked without sudo privileges. Now I have everything working on both my Pi and Nano =).


Organizing GDCR 17 in Santiago, Chile

The Software Crafters Comunity is organizing a GDCR on Santiago, Chile

Welcome ☺🎉


Speech-to-Text sample

Hi,
I tried running the speech-to-text sample but get an error (SPXERR_GSTREAMER_NOT_FOUND_ERROR) on the mp3 file. It seems a plugin needs to be installed to read compressed files.
I can't see it addressed in the prerequisites. It would be good to add instructions on this for Databricks and Azure Synapse as described in the how-to, to be able to run these samples.
I see there's a guide here https://learn.microsoft.com/en-us/azure/cognitive-services/speech-service/how-to-use-codec-compressed-audio-input-streams?tabs=windows%2Cdebian%2Cjava-android%2Cterminal&pivots=programming-language-python but it's using azure.cognitiveservices.speech package not mmlspark.cognitive and on local machines, not Azure Synapse / Databricks.
I'd be grateful especially on Azure Synapse as that's what I'm using.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 21731831-f5a9-c779-48dd-42f760f0809e
Version Independent ID: fc748952-1a5a-41b7-ffb9-00649bf26cf8
Content: Cognitive Services for big data - Python Samples - Azure Cognitive Services
Content Source: articles/cognitive-services/big-data/samples-python.md
Service: cognitive-services
GitHub Login: @mhamilton723
Microsoft Alias: marhamil

@iww9
Thanks for your feedback! We will investigate and update as appropriate.

@iww9
use version 1.14.5 of Gstreamer https://gstreamer.freedesktop.org/pkg/windows/1.14.5/gstreamer-1.0-x86_64-1.14.5.msi - complete setup
use version 1.13 of Microsoft.CognitiveServices.Speech (Nuget package)
Go to environment variables and add path the following C:\gstreamer\1.0\x86_64\bin
then add a system variable called "GSTREAMER_ROOT_X86_64" (without the quotes) and the value to "C:\gstreamer\1.0\x86_64"
you may need to reboot if still having issues.

Hi, Thank you very much for your response.
How can I do this on Azure Synapse?

I think this isnt the fix. This hsould work OOTB on synapse, Can you try this example, keeping the wav file intact:
https://github.com/microsoft/SynapseML/blob/master/notebooks/features/cognitive_services/CognitiveServices - Overview.ipynb
If the following example does not work. Then we have a bigger problem. If it does work but the example fails when you use an mp3, then we need to run this script on the synapse workers:
#!/bin/bash
sudo add-apt-repository ppa:jonathonf/ffmpeg-4 -y
sudo apt-get update
sudo apt-get install ffmpeg -y
sudo apt-get install libgstreamer1.0-0 gstreamer1.0-plugins-base gstreamer1.0-plugins-good -y

@iww9
use version 1.14.5 of Gstreamer https://gstreamer.freedesktop.org/pkg/windows/1.14.5/gstreamer-1.0-x86_64-1.14.5.msi - complete setup
use version 1.13 of Microsoft.CognitiveServices.Speech (Nuget package)
Go to environment variables and add path the following C:\gstreamer\1.0\x86_64\bin then add a system variable called "GSTREAMER_ROOT_X86_64" (without the quotes) and the value to "C:\gstreamer\1.0\x86_64"
you may need to reboot if still having issues.
if there are any further questions regarding the documentation, please tag me in your reply and we will be happy to continue the conversation.

Thanks @mhamilton723.
The wav file example runs successfully.
How can you run that script on the synapse workers?

Hi @mhamilton723
Could you please review and add your comments on this, update as appropriate.

@iww9
Thanks for your feedback! I've assigned this issue to the author who will investigate and update as appropriate.

Thanks for your dedication to our documentation. Unfortunately, at this time we have been unable to resolve the issue in a timely manner and we sincerely apologize for the delayed response. The requested updates have not been made since the creation of this issue, and the timeline for resolution may vary based on resourcing, so we've created an internal work item to incorporate your suggestions. We are closing this issue for now, but feel free to comment here as necessary.
#please-close


[setup] Conda image could be improved

In the setup, at the step telling you to make a new conda env, the explanitory image assets/vs_code_create_environment.jpg highlights venv instead of conda, which could cause someone not paying enough attention *cough* to click on venv
Old image

Suggested new image

(although my colour scheme is monokai, not sure what the original one is)

Updated README.md.
See 1bd96c2.


pass .packages path to snapshot invocation

Description
While a snapshot can generally run without a .packages, it prevents using some Isolate package resolution APIs we need to use to integrate with webdev

Tested: manual
:)


Nuget package library for interaction with SSRS Rest API not available

Please provide a nuget package for easy install SSRS Rest API interactions.
Right now the only option is Swagger.
But when using Swagger, how to keep library updated as Microsoft improves the swagger library?
In nuget you get package update notifications?
This requires setting up a whole big .NET project and compiling in our 100 applications by adding reference to that project.
Would be easier if just in package.config and it automatically restores on checkout.

Put this feature request in: https://ideas.powerbi.com/forums/265200-power-bi-ideas/suggestions/40742506-provide-a-nuget-for-power-bi-report-server-rest-ap


[BUG] flagd accepts invalid flag config with just a warning

Observed behavior
When using a flag config that does not conform to the schema, flagd shows the following warning in the logs but accepts the config nevertheless without error:
2024-12-19T10:26:53.980+0100	warn	evaluator/json.go:114	flag definition does not conform to the schema; validation errors:  1:flags.is-enabled: Must validate one and only one schema (oneOf)  2:flags.is-enabled.variants.off: Invalid type. Expected: number, given: boolean  3:flags.is-enabled: Must validate all the schemas (allOf)
This leads to inconsistent behavior during evaluations (https://github.com/open-feature/flagd/issues/1481)
Expected Behavior
There are a couple of behaviors that could make sense:

when flag config validation fails, return an error and don't accept the updated config, essentially keep working with whatever was already in the flag store (preferred)
possibly, accept only parts of the config that conform to the schema and return a warning / error (probably not a good idea)

Steps to reproduce
Use the following flag config. The variants have mismatched types (number and boolean).
{
    "$schema": "https://flagd.dev/schema/v0/flags.json",
    "flags": {
      "is-enabled": {
        "defaultVariant": "off",
        "state": "ENABLED",
        "targeting": {
          "if": [
            {
              "<": [
                {
                  "%": [
                    {
                      "var": "request_id"
                    },
                    1000
                  ]
                },
                100
              ]
            },
            "on",
            "off"
          ]
        },
        "variants": {
          "on": 1,
          "off": false
        }
      }
    }
  }

Hey @cupofcat, I agree that the first option sounds like the right approach. We should start by defining the expected behavior in the provider lifecycle spec.
I believe we'll need to define the expected behavior during:

Initialization: what happens when flagd (or an in-process provider) starts with an invalid config. The current behavior is to put flagd in a fatal state.
Invalid config updates: should the whole flag set update be rejected or only the invalid flags? I believe that rejecting the whole change would be the best approach.

FYI, @toddbaert @lukas-reining @aepfli

Hey @cupofcat, I agree that the first option sounds like the right approach. We should start by defining the expected behavior in the provider lifecycle spec.
I believe we'll need to define the expected behavior during:

Initialization: what happens when flagd (or an in-process provider) starts with an invalid config. The current behavior is to put flagd in a fatal state.
Invalid config updates: should the whole flag set update be rejected or only the invalid flags? I believe that rejecting the whole change would be the best approach.

FYI, @toddbaert @lukas-reining @aepfli

I agree on both counts.


After I deploy it with 'bash workspace/service_docker_up.sh' how may I call it on other python code?

I would like to use this deployment as a server that all python code could call it as an api. Is it possible to do that? Thanks a lot!

You can try the API in lmdeploy/serve/turbomind/chatbot.py.
lmdeploy/serve/client.py is an example about how to use the API


docs: add new environment variable to disable Hugging Face warning when spaces persistant storage is disabled

Description
This PR is associated to changes implemented on https://github.com/argilla-io/argilla-server/pull/124 adding a new environment variable to force disable a warning when Argilla is running on Hugging Face Spaces and persistant storage is disabled.
Closes #4736
Type of change
(Please delete options that are not relevant. Remember to title the PR according to the type of change)

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Refactor (change restructuring the codebase without changing functionality)
[ ] Improvement (change adding some improvement to an existing functionality)
[x] Documentation update

How Has This Been Tested

[x] Checking that the markdown is correctly rendered.

Checklist

[x] I added relevant documentation
[ ] follows the style guidelines of this project
[ ] I did a self-review of my code
[ ] I made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I filled out the contributor form (see text above)
[ ] I have added relevant notes to the CHANGELOG.md file (See https://keepachangelog.com/)

As it is added in HF, maybe a reference should be added here too? https://docs.argilla.io/en/latest/getting_started/installation/deployments/huggingface-spaces.html#setting-up-secret-environment-variables

I think makes more sense adding under the section  Setting up the persistent storage

As it is added in HF, maybe a reference should be added here too? https://docs.argilla.io/en/latest/getting_started/installation/deployments/huggingface-spaces.html#setting-up-secret-environment-variables

I think makes more sense adding under the section Setting up the persistent storage

I will add a tip block there.

Added a note on Hugging Face setting storage section about the new environment variable.

@jfcalvo Thanks, it should be a bit clearer now.


istio cni pod can not ready

Is this the right place to submit this?

[X] This is not a security vulnerability or a crashing bug
[X] This is not a question about how to use Istio

Bug Description
2024-02-24T08:26:20.705824Z     info    install CNI config file /host/etc/cni/net.d/05-cilium.conflist exists. Proceeding.
2024-02-24T08:26:20.710211Z     info    install Created CNI config /host/etc/cni/net.d/05-cilium.conflist
2024-02-24T08:26:20.710236Z     info    install Istio CNI configuration and binaries validated/reinstalled.
2024-02-24T08:26:20.710296Z     info    file watcher skipping watch on non-existent path: /host/secondary-bin-dir/istio-cni
2024-02-24T08:26:20.717437Z     info    file modified: /host/etc/cni/net.d/.05-cilium.conflist293125112
2024-02-24T08:26:20.717458Z     info    file modified: /host/etc/cni/net.d/.05-cilium.conflist293125112
2024-02-24T08:26:20.729396Z     info    install Detected changes to the node-level CNI setup, checking to see if configs or binaries need redeploying
2024-02-24T08:26:21.489439Z     info    install Copied istio-cni to /host/opt/cni/bin.
2024-02-24T08:26:21.489481Z     info    install Directory /host/secondary-bin-dir is not writable, skipping.
2024-02-24T08:26:21.490294Z     info    install missing (or invalid) configuration detected, (re)writing CNI config file at /host/etc/cni/net.d/05-cilium.conflist
2024-02-24T08:26:21.490303Z     info    install Using CNI config template from CNI_NETWORK_CONFIG environment variable.
2024-02-24T08:26:21.490310Z     info    install CNI config: {
  "cniVersion": "0.3.1",
  "name": "istio-cni",
  "type": "istio-cni",
  "log_level": "debug",
  "log_uds_address": "/var/run/istio-cni/log.sock",

  "kubernetes": {
      "kubeconfig": "/etc/cni/net.d/ZZZ-istio-cni-kubeconfig",
      "cni_bin_dir": "/opt/cni/bin",
      "exclude_namespaces": [ "istio-system", "kube-system" ]
  }
}
2024-02-24T08:26:21.490632Z     info    install CNI config file /host/etc/cni/net.d/05-cilium.conflist exists. Proceeding.
2024-02-24T08:26:21.494864Z     info    install Created CNI config /host/etc/cni/net.d/05-cilium.conflist
2024-02-24T08:26:21.494883Z     info    install Istio CNI configuration and binaries validated/reinstalled.
2024-02-24T08:26:21.494927Z     info    file watcher skipping watch on non-existent path: /host/secondary-bin-dir/istio-cni
2024-02-24T08:26:21.502079Z     info    file modified: /host/etc/cni/net.d/.05-cilium.conflist2066457858
2024-02-24T08:26:21.502102Z     info    file modified: /host/etc/cni/net.d/.05-cilium.conflist2066457858
2024-02-24T08:26:21.517402Z     info    install Detected changes to the node-level CNI setup, checking to see if configs or binaries need redeploying

root@istio-01-a:~# kubectl get pod -n istio-system -w
NAME                                       READY   STATUS    RESTARTS      AGE
elasticsearch-master-0                     1/1     Running   1 (13m ago)   4d1h
elasticsearch-master-1                     1/1     Running   1 (13m ago)   4d1h
elasticsearch-master-2                     1/1     Running   1 (13m ago)   4d1h
istio-cni-node-5z88l                       0/1     Running   0             8m32s
istio-cni-node-rj8jt                       0/1     Running   0             8m32s
istio-cni-node-rl9zq                       0/1     Running   0             8m32s
istio-cni-node-svwsd                       0/1     Running   0             8m33s
istio-egressgateway-c878fd6c5-4pdbs        1/1     Running   0             8m35s
istio-ingressgateway-86446666f9-kfnfw      1/1     Running   0             8m35s
istiod-75d8d56b68-9x6mh                    1/1     Running   0             8m42s
opentelemetry-collector-6c979d7998-dm8lv   1/1     Running   1 (13m ago)   3d

Version
Croot@istio-01-a:~# istioctl version
client version: 1.20.3
control plane version: 1.20.3
data plane version: 1.20.3 (10 proxies)

root@istio-01-a:~# kubectl version
Client Version: v1.28.1
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.28.1

Additional Information
No response

It looks like it's conflicting with cilium cni, can you post the contents of this file：/etc/cni/net.d/05-cilium.conflist （the node host path)  ?

Cilium must be installed with cni.exclusive disabled.

Cilium must be installed with cni.exclusive disabled.

Correct, cilium will actively purge all other CNI config and plugins (as in, literally delete them) unless you tell it not to.
This also creates problems with cloud-provided CNIs, which is why Cilium docs mention the use of cni.exclusive=false there as well: https://docs.cilium.io/en/stable/installation/cni-chaining-aws-cni/


Daemon exits on load if user has custom downloads path - Windows

Bug
If a new user isn't using the default downloads path, the app fails to load after installing (see below error message).
To Reproduce
Steps to reproduce the behavior:

Change default download path
Start application.
See error

Expected behavior
Check system defaults for the correct download path.
Screenshots

System Configuration
Windows 10
Latest version of the app.
Additional Info
I am unsure if the user had changed the download path on their system after installing. However, after uninstalling and then reinstalling, the problem continued.

This is a bug on the SDK side, thought we should surface the errors to here when possible. Believe there are issues for it, and if not, I'll file.
Added the workaround to the startup troubleshooting steps, thank you!

Sorry. I always get confused which one it should go in.
I had a quick check but didn't find any other issues listed. Maye because I was looking in the wrong place.

I think it's kind of filed here:
https://github.com/lbryio/lbry-sdk/issues/1970#issuecomment-480417753


add install instructions and warning for netlify adapter

hi! i used to work at netlify.
i figured i would better document the netlify adapter install instructions because it tripped me up as well as Rich Harris at Svelte Summit so it definitely needs some documenting

All of the adapters and other packages are currently set up so that you should be using the next tag. I think these should all be so documented in one place. Currently that's in the FAQ but it could be there's a better place.

ya. just adding it where i normally look for this warning (http://npm.im/@sveltejs/adapter-netlify)

thanks — just curious about the inclusion of the prerender options, which mostly match the defaults but add force: true which is generally unrecommended — is this intentional? If so crawl, enabled and pages could be usefully omitted i think

no, not intentional - was copied from another template. i'll remove


build environment: check on version of dependent software

On Linux ( SemaphoreCI ) we check for the version of the dependent software (stack, z3, cvc4).
On Windows (appveyor) we don't check this!
Make consistent by also checking on windows!

Instance of the more general issue #709


[MAIN-2] Add Author and Publisher fields to the Add Book Page

Pull Request
Story ID: MAIN-2
Title: Add Author and Publisher fields to the Add Book Page
Card URL: https://trello.com/c/hbkzZyE2
Author: @flukekd


Update detector.go - doubled word in sentence

What this PR does / why we need it:
There is a wrong spelling in dectector.go
Which issue(s) this PR fixes:
Fixes #
Does this PR introduce a user-facing change?:

NONE

/cla sign

Thank you. Nice catch! 👍
/approve

/trigger presubmits


Sensor readout while wave_tx_busy()

Dear all,
first of all I want to thank you joan for creating this amazing PIGPIO lib!
I am using it to create a nice ramp for my stepper motor, using a TMC2208 (Trinamics) motor driver.
After some troubleshooting I managed to get the PIGPIO work as I want it to.
Now I want to read out my "homing" sensor while the stepper is moving to check if a limit was reached.
I am using the following code to read out the sensor while the chain is in progress.
The function "checkLimits will return true if a sensor is active.
# Check if limits are reached during movement
        while (pi.wave_tx_busy()): # While transmitting.
            if (checkLimits(axis,steps,PUL)):
                pi.wave_tx_stop()
                break
            sleep(0.1)

The problem is, that my PWM is looking really bad when this while loop is active and the motor is doing noisy movement.
Even if I just enter:
# Check if limits are reached during movement
        while (pi.wave_tx_busy()): # While transmitting.
		sleep(0.1)

It behaves the same. It feels like the while loop is disturbing the wave chain progress.
Is there any better way to read out the sensor after hitting the pi.wave_chain(chain) command?
Thank you very much, I really appreciate your help!
Philipp

When you just play the waveform, without any looping, is your stepper motor performing noise/jitter free?
Heavy network traffic on the pi will induce some additional jitter - on the order of 3-4 microseconds - on top of a baseline of 2-3 us jitter.  (These were my observations I made quite some time ago.)

When just playing the wave, the stepper motor is doing fine without any problems.

The problem is, that my PWM is looking really bad when this while loop is active...

Is there any way you can quantify the problem and provide a script that we can use to reproduce?

So this is a direct comparison of noice of the two movements. The first is with the "while" part and the second is without it.
The motor is also not at the correct position at the end.
https://user-images.githubusercontent.com/74316765/119476267-31785300-bd4e-11eb-89e8-880b580643b5.mov
https://user-images.githubusercontent.com/74316765/119476239-2a514500-bd4e-11eb-89ae-f225904cfae0.mov

Since I don't have a stepper motor handy, I will be looking at the output pin with an oscilloscope to see any distortion that might be the cause of the motor noise.  If you have such a scope trace it would nice to see it for comparison.
Also, why the pi.set_PWM_dutycycle(PUL, 128) before transmitting the wave chain?  PWM will be turned off whenever waves are transmitting.

Closing due to lack of response.


Top Charts component Updated

I have updated the Top Charts component. Check it now

also need to work on page overflow now #39

#28