High memory usage when embedding large texts

Hi i am experiencing high memory usages which caused my pod to be killed because of exeeding its limits. after some experiments i found its related to text length that i am trying to embed.
when using models such as e5 large or paraphrase base v2  the program starts with about 1.5GB of ram usage which is expected for this model, but after some iterations this process uses  16GB of virtual memory and 6GB of ram. which is a lot
I tried other smaller models as well and its happening for them too but in smaller scale. So my intuiton was that i am just using texts that are too long so i cut all to max 100 chars. which slowed the ram usage increase, but still if not using constant batch of text it kept growing untill the program was eventually killed.
For more context i am building vector api that returns embeddings for texts. so model sessions is active from the start.
To reproduce you can use this array of texts with e5 large model. in default example with infinite loop
[
            "Spain",
            "Spain,Barcelona,Avignon,Maspalomas,Disfrutar,Bohemia Suites & Spa,Mandarin Oriental, Barcelona,Restaurant Milano",
            "",
            "Seattle's best vegan restaurants",
            "Seattle,The Wayward Vegan,Lin Handmade Dumplings & Noodles,The Sushi Samurai (100% Vegan),Harvest Beat,Cafe Flora,Araya's Place,Plum Bistro,Georgetown Liquor Company,Pi Vegan Pizzeria,Cycle Dogs",
            "Must try vegan restaurants when visiting Seattle!",
            "10 must visit breweries in Seattle!",
            "Seattle,Great Notion Brewing - Ballard,Bizarre Brewing,Stoup Brewing - Ballard,Reuben's Brews - The Ballard Taproom,Urban Family Brewing Co.,Fremont Brewing's Urban Beer Garden,Seapine Brewing Company,Obec Brewing,Cloudburst Brewing,Holy Mountain Brewing Company",
            "Seattle is a mecca for incredible beer, here are 10 must visit breweries!",
            "Newport",
            "Newport,Elizabeth Oceanfront Suites,Rogue Bayfront Public House,Local Ocean Seafoods",
            "",
            "Iceland",
            "Iceland,Seljalandsfoss,Skaftafell",
            "",
            "Paris",
            "Paris,Eiffel Tower,Place de la Concorde,Jardin des Plantes",
            "",
            "Weekend in Tokyo",
            "Tokyo,Tokyo,Tokyo Metropolitan Government Building,Circus Tokyo,Asakura Museum of Sculpture,Kanae,Hotel Sunroute Plaza Shinjuku,Inshotei,Nata de Cristiano,GYOPAO Gyoza Roppongi,d47 SHOKUDO,Shibuya Scramble Crossing,Little Nap Coffee Stand,Shinjuku Golden Gai,Sarutahiko Coffee,Mag's Park Rooftop Shibuya Crossing,Ueno Toshogu Shrine,Kayaba Coffee,Sakurai Japanese Tea Experience,Ōkuma Shōkai,Meiji Jingu",
            "",
            "The Best Cheap Eats in Tokyo",
            "Tokyo,Tokyo,Yamatarō,Karaage Hitosuji,Butagumi Shokudo,Torikatsu Chicken Shibuya,Niigata Katsudon Tarekatsu Shibuya,Matsuya,YAKINIKU LIKE Shibuya Udagawacho,Ginza Shabutsu,Hakata Furyu Akihabara flagship shop,Pork Vindaloo Taberu Fukudaitoryo",
            "",
            "Port Clyde",
            "Port Clyde,Saint George,Boothbay,Bristol,Rockland,Rockland Café,Monhegan Boat Line,Pemaquid Point Lighthouse,Ocean Point Walk,Marshall Point Lighthouse",
            "",
            "Las Vegas",
            "Las Vegas,Las Vegas,Shin Lim - Limitless,Tacos El Gordo,High Roller,Fountains of Bellagio,M&M’S Las Vegas,SAHARA Las Vegas,Bavette's Steakhouse & Bar,Delmonico Steakhouse,Skybar,Shark Reef Aquarium at Mandalay Bay,Fremont Street Experience,Las Vegas Hilton at Resorts World,Secret Pizza,Allegiant Stadium,Eataly",
            "",
            "Hershey",
            "Hershey,The Mill in Hershey,Hershey Gardens,The Hotel Hershey",
            "",
            "Paia - Best beaches ",
            "Paia,Paia Inn,Paia Fish Market Restaurant,Milagros Food Co.,Paia Surf Lessons,Paia Bay,Mana Foods,Baldwin Beach Park,Café Des Amis,Flatbread Company",
            "",
            "24 Wild Hours on the Las Vegas Strip",
            "Las Vegas Strip,Las Vegas,Garden of the Gods Pool Oasis,High Roller,The Shops at Crystals,MGM Grand,Marquee Nightclub,Shin Lim - Limitless,Ghost Donkey,Tacos El Gordo,Momofuku,Eggslut,WAKUDA,Sphere",
            "",
            "10 Artsy Attractions in Barcelona",
            "Barcelona,Fundació Antoni Tàpies,Museu Nacional d'Art de Catalunya,Picasso Museum,Casa Batlló,CaixaForum Barcelona,El Gat de Botero,Moco Museum Barcelona,La Pedrera-Casa Milà,Barcelona Museum of Contemporary Art,Joan Miró Foundation",
            "",
            "Las Vegas",
            "Las Vegas,Las Vegas,Red Rock Canyon National Conservation Area,Fremont Street Experience,Fountains of Bellagio",
            "",
            "5 Incredible Dubrovnik Eats with Epic Views",
            "Dubrovnik,Komolac,Bistro Izvor,Nautika,Restaurant 360,Komarda,Restaurant Panorama",
            "",
            "Midweek Mammoth Ski Trip",
            "Mammoth Lakes,Bishop,Mammoth Mountain,The Stove,Erick Schat's Bakery,Mammoth Mountain Inn,Convict Lake,Roberto's Cafe,Holy Smoke Texas Style BBQ",
            "",
            "Oregon",
            "Oregon,Eugene,Columbia River Gorge National Scenic Area,Multnomah Falls,Hult Center for the Performing Arts",
            "",
            "Maldives",
            "Maldives,Malé,Hulhumalé,Fihalholi,Hulhule Island,Velassaru,Naladhu Private Island Maldives,Ithaa Undersea Restaurant,Seagull Café,Fish Market,Hulhumale,Sea House Café,MU Beach Bar & Grill",
            "",
            "10 must try breweries in Seattle!",
            "Seattle,Fremont Brewing's Urban Beer Garden,Holy Mountain Brewing Company,Cloudburst Brewing,Standard Brewing,Great Notion Brewing - Ballard,Seapine Brewing Company,Bizarre Brewing,Obec Brewing,Stoup Brewing - Ballard,Reuben's Brews - The Ballard Taproom",
            "Seattle is a mecca for incredible beer! Here are the breweries that are a must visit while here!",
            "Japan",
            "Japan,Sapporo,Kyoto,Hikone,Lake Biwa,Funaoka Onsen,Hōheikyō Hot Spring",
            "",
            "The 10 Best Outdoor Adventures in Dubrovnik",
            "Dubrovnik,Studenci,Dubrovnik Game of Thrones Tour,Dubrovnik Local Guides - City Walls Walking Tour,Lokrum,Dubrovnik Local Guides - Old Town Walking Tour,Blue Cave Dubrovnik | Official tours,Plaža Sveti Jakov,Dubrovnik Elafiti Cruise,반예비치,Betina Cave beach,Kravica Waterfall",
            "",
            "The Perfect Barcelona Weekend",
            "Barcelona,Moog,SlowMov,Banksy museum - Barcelona,Can Paixano,Entrepans Díaz,Bo de B,Barceloneta Beach,Palau de la Música Catalana,Funky Eatery,Nomad Coffee - Lab & Shop,Majestic Hotel & Spa Barcelona,La Sagrada Familia,Mercado de La Boqueria,Park Güell,La Pepita,Three Marks Coffee",
            "",
            "Japan",
            "Japan,Tokyo,Osaka,Kyoto,Urayasu,Nisekohirafu 2 Jo,Arashiyama Bamboo Forest,Aman Tokyo,Tokyo Disneyland,Afuri,OKO - Fun Okonomiyaki Bar,Dotonbori River,Sukiyabashi Jiro,Imperial Palace",
            "",
            "Aruba",
            "Aruba,Noord,Oranjestad,West Deck,Buccaneer Restaurant,Eagle Beach,Hyatt Regency Aruba Resort Spa And Casino,Palm Beach,Malmok Beach",
            "",
            "Czechia drone map",
            "Czechia,Štěchovice,Pňovany,Brno,Most,Nalžovské Hory,Velké Karlovice,Kovářov,Velemín,Bartošovice v Orlických horách,Kdyně,Seč,Dvůr Králové nad Labem,Dolní Morava,Chrudim,Jílové,Nechanice,Skalná,Třemošnice,Černý Důl,Turnov,Chudenice,Oslov,Libošovice,Velká Jesenice,Jílové u Prahy,Svojanov,Rabyně,Místo,Přítluky,Žleby,Nové Dvory,Točník,Podolí I,Heřmanice,Klapý,Nedvědice,Bříství,Klenová,Ráby,Zřícenina hradu Ballymote,Kost,Čarták lookout (Súkenická viewtower),Most u Červené nad Vltavou,Milešovka,Royal Forest Dam,Vyhlídka na přehradu,Veveří Castle,castle Svojanov,Nové Mlýny,Hospital Kuks,Bridge Pňovany,Hrádek u Nechanic,Vyhlídka Bednář,Ruins of the Chapel Exaltation. Crisis,Vyhlídka Neratov,Točník Castle,Nákladní lanovka Černý důl,Podolský most,Žampach viaduct,lookout Rýzmberk,Hněvín,Lichnice Castle ruins,Viewpoint Máj,Obora Žleby,Klenová - castle, chateau, gallery,lookout Bolfánek,Pernštejn Castle,Rock lookout Hlavatice,Žďákov Bridge,Seč Dam,Vildštejn Castle,Kunětice Mountain,Hazmburk,Hasištejn Castle,Rozkoš Lake,Velhartice,Železniční viadukt,Stezka v oblacích,Děčínský Sněžník,Kačina,Rozhledna Bára",
            "flying from castle park is prohibited, but never had problem to get permission from the castle,tip fromhttps://www.facebook.com/groups/letamesdrony/permalink/3295302460761543,tip fromhttps://www.facebook.com/groups/letamesdrony/posts/3295168087441647/,tip fromhttps://www.facebook.com/groups/letamesdrony/posts/3294486957509760/,https://www.youtube.com/watch?v=uh5Jv12hn7c&ab_channel=siamifpv",
            "10 Iconic Portland Eats ",
            "Portland,Hat Yai,Departure Restaurant + Lounge,Stumptown Coffee Roasters,Le Pigeon,Lil' America,Pine State Biscuits | NW 23rd,Great Notion Brewing - NW28th,Nong's Khao Man Gai (Downtown),Matt's BBQ,Voodoo Doughnut - Old Town",
            "",
            "Hottest new restaurants in Seattle!",
            "Seattle,Kirkland,Burien,Sap Sap Lao Deli & Cafe,Sushi Suzuki,Layers Green Lake,El Encanto,Ballard Beer Box,Ananas Pizzeria,Watson's Counter,Donna's,Stevie's Famous Burien,Miss Pho,Ben's Bread Co.,Sammich Seattle,G.H PASTA Co.,The Boat,Gold Coast Ghal Kitchen,FamilyFriend",
            "Come check out the hottest new restaurants in Seattle.",
            "1 Week in Los Angeles",
            "Los Angeles,Avalon,Hermosa Beach,Pasadena,Long Beach,Beverly Hills,Manhattan Beach,Central LA,San Marino,ROW DTLA,Stahl House,Mt Ada,Griffith Park,Tail O' the Pup,The Westin Long Beach,In-N-Out Burger,Catalina Express,Beverly Hills Park,Book Soup,Descanso Beach Club,Pages: a bookstore,Rose Bowl Flea Market,curious...,Pizzana,Uncorked The Wine Shop,The Shay - Destination by Hyatt,Avalon Hotel Beverly Hills,Gyu-Kaku Japanese BBQ,Anderton Court Shops - Frank Lloyd Wright,Airplane Landing View Point,Gum Tree Shop & Café,Catalina Museum For Art & History,Los Angeles County Museum of Art,Martha’s Hermosa Beach,Griffith Observatory,Catalina Casino,The Langham Huntington, Pasadena,Veterans Parkway (Hermosa Valley Greenbelt),Hollywood Walk of Fame,Clark Street Diner,Sunset Boulevard,Vroman's Bookstore,Dialog Cafe,HomeState,The Huntington Library, Art Museum, and Botanical Gardens,Avalon Grille,Los Angeles International Airport,Vista",
            "With 7 days you can see LA the way many first-time visitors do, as well as dig a little deeper. Hope you enjoy!This is a shopping day! Start at Gum Tree and if you need coffee, get it here before browsing their shop. I recommend a Dirty Chai or one of their seasonal specialty drinks.,Reward yourself for the early exercise with a White Corn Omelette and Bloody Mary at Martha's. Since it is mere steps from the beach, walk down to it to put your feed in the sand afterward.,You'll find all the quirky and super fun gifts at this LGBTQIA-friendly shop that always has a wonderful display in their window for PRIDE.,Go here before you tuck into bed for the night. Urban Light in the evening has its own special vibe and since it's outside the museum entrance, it's accessible 24/7/365.,Leave time today or tomorrow for the pool at this boutique hotel with interior design by Kelly Wearstler. ,After taking the ferry back to the mainland, get a final taste of California. The In-N-Out by LAX brings the bonus of being right under a landing zone so you watch planes land while enjoying your cheeseburger animal style. ,While watching planes land, take off your shoes. Walking on grass with bare feet is supposed to help you before and after flying.,Safe travels and enjoy your flight home!,Time your visit to arrive on a Saturday before the Rose Bowl Flea Market the next day. It takes place every second Sunday of the month. Head straight to The Langham and check-in. Take advantage of their amazing Chuan Spa to unwind after your flight.,Wake up super early and go shopping at the Flea. The early bird truly gets the worm and the best finds. It's like an LA right of passage and a great way to pick up a souvenir you literally can't find anywhere else.,Reward yourself for waking up early by heading here after the Flea. Nosh on a chorizo taco, queso and a Ranchwater or Paloma for maximum satisfaction. They have vegetarian options too!,Spend the balance of your day in the stacks finding your next read before you start the [possibly long - LA traffic!] drive to your next hotel.,This will be a packed day! So pick up some caffeine, if that's your thing, at the Alfred Coffee on the corner of Beverly Drive and Santa Monica to fuel up. Then walk across the street and stroll the park, headed in the direction of the Beverly Hills Sign.,Look for the angled circular staircase and tower by Frank Lloyd Wright that's hiding in plain sight here. If it's open, take a walk up to the top for a nice view of Rodeo Drive.,A cold Asahi or Sapporo goes well with their BBQ.,Try to catch sunset here in the park or at the top by the Griffith Observatory. Top tip if you have a rental car: since there is never enough parking up by the Observatory, look for the shuttle that allows you to park near the Greek Theater and get a ride up the hill. If not, Uber straight to the top.,Make sure to head inside and see the Tesla Coil if science is your thing!,Enjoy Happy Hour here where they have wine flights and a great selection of bottles you could take home to your hotel to enjoy before leaving California.,Start the day early with a wonderful walk on this greenbelt that goes through the beach communities of the South Bay.,If you want to be so close to the beach you could touch it for Happy Hour, or dinner, go to Vista instead of Uncorked. Or start at Uncorked and finish your day with dinner at Vista!,Pick up a book to read at the beach club tomorrow!,Stay here for the amazing happy hour, views over Catalina, the included golf carts so you can explore the island at leisure and the wonderfully sumptuous guest rooms.,Take the casino tour where you will understand why there's no gambling in this island landmark!,If you don't yet have a book for the plane ride home, this is a great place to pick one up. Or a second one!,Grab lunch at this landmark that harkens back to Hollywood's golden eras. ,You have to get tickets to tour this home before you arrive in Los Angeles but it's worth planning ahead for!,You can't leave Los Angeles without trying the pizza. It gives New York a run for its money and this is one of the best spots for it in LA.,Because you have to visit the stars when you're in Los Angeles, right?,Stay at The Westin so you're close to the Long Beach ferry to Catalina Island which you will want to board early the next day! It has a Travel Sustainability Rating of Level 3 according to booking.com!,Take an early ferry to Catalina Island to spend the day and night at California's version of The Riviera.,Visit this museum so you understand the connection the Chicago Cubs and Marilyn Monroe have to the island.,Enjoy dinner here before heading up the hill to Mt. Ada for a wonderful night's rest.,Lay out on the beach or enjoy the delicious tacos at their restaurant. If you're feeling the lux vibe, reserve a cabana. You can also snorkel or kayak in the bay.,Sit down for brinner, aka: breakfast for dinner, at this locally-loved but Tik Tok-friendly diner that's on the way from the Walk of Fame to Griffith Park.,Grab breakfast or brunch at this West Hollywood classic.,If you're not too tired from walking the Flea, head to this garden which is one of the most beautiful in all of Los Angeles.,If you can't get tickets to Stahl House or you want to experience another part of LA, check out ROW which is shops and restaurants that represent the best of creative LA. You can also feast on excellent pizza here, instead of Pizzana, at Pizzeria Bianco.,Take in the The Sunset Strip and do some shopping before returning to Book Soup. ,This hotel is incredibly situated whether you're driving yourself, taking public transportation or grabbing ubers during your visit to Los Angeles. It's also across the street from The Platform which is a super hot outdoor shopping and events complex.",
            "Beach Trip on Paros Island",
            "Paros,Santa Maria,Parasporos,Naousa,Chrisi Akti,Pounta,Kolimpithres,Kolympethres Beach,Parasporos Beach Club,Kalogeros beach,Faragas,Santa Maria,Paros Park,Port of Pounda,Παραλία Παρασπόρος,Golden Beach Paros,Piperi Beach,Santa Maria Beach Bar",
            "Famous for being the best overall beach on Paros!,Cool rocks and views.,Be sure to check out the healing mud.,Considered the best beach in Naoussa.,Rocks, caves, hiking and more!,Parasporos Beach is the best beach near Parikia.,Great spot for windsurfing!,Ferry to Antiparos",
            ]

There might be a few things at play here:

Size of the text — we load everything in memory, both the text and it's vectors. This means as vectors get computed, there is more RAM consumption which keeps increasing.
Data Parallel — we use Python's multi-processing under the hood to split large lists of strings into smaller and process in parallel. This can lead to large RAM consumption as well sometimes.

Can you give some estimates of 3 things:

How many tokens are there per sentence?
How many sentences are there?
What are the device RAM and CPU specs?

@NirantK

tokens not shure at the moment but in latest testing i used max of 512 characters for e5 large model
single batch had about 130 items
64GB RAM, AMD Ryzen 9 5950X 16-Core Processor

But the main problem i have is that after processing of batch is done the memory is not released. it only keeps growing untill the process gets killed. i think that the problem is in onnx session when using continuosly without recreating. when i use the model dirrectly from Hugging face, there is no growth of memory and no problem

Thanks for sharing specifics @Barney241 — very helpful.

i think that the problem is in onnx session when using continuosly without recreating.

This'd be among my areas to investigate as well. Thanks for sharing. I'll look into this as energy permits! I'm leaving this issue open and marking it as a bug because of the impact it has on how usable FastEmbed is.

Hi, any updates on this? We've experienced the same issue.

Hi @johnreyev
Could you tell us the version of fastembed you're using and maybe a reproducible code snippet?
We've tried to reproduce the issue with the code above, but unsuccessfully, we haven't been able to observe the memory growth

Hi @joein, you can try this:
fastembed==0.3.6
lorem==0.1.1

from timeit import default_timer as timer
import lorem
from fastembed import TextEmbedding

def run():
    model = TextEmbedding(model_name='BAAI/bge-small-en-v1.5')

    # start timer
    start = timer()

    # Generate a large amount of sample text
    docs = [lorem.text() for _ in range(10000)]

    print(f'Each document is {len(docs[0])} chars long.')
    print(f'Embedding {len(docs)} documents...')

    # embed the documents
    embeddings = model.embed(documents=docs, batch_size=512)

    proto_embeddings = [
        embedding.tolist()
        for embedding in embeddings
    ]

    print(f'len(embedding_list) {len(proto_embeddings)}')
    print('Done!')

    # end timer
    end = timer()

    print(f'Time taken: {end - start:.2f} seconds')

if __name__ == '__main__':
    run()


Running the code with GPU

Hi, Thanks for the great code.
While trying to run the code with CUDA by setting the gpu flag in config.yaml to true, I am getting the following log trace:
[2021-04-08 01:38:20,880][icml2020][INFO] - Running command '_main'
[2021-04-08 01:38:20,928][icml2020][INFO] - Started run with ID "1"
[2021-04-08 01:38:22,528][ignite.engine.engine.Engine][INFO] - Engine run starting with max_epochs=400.
[2021-04-08 01:38:22,530][ignite.engine.engine.Engine][ERROR] - Current run is terminating due to exception:
[2021-04-08 01:38:22,531][ignite.engine.engine.Engine][ERROR] - Engine run is terminating due to exception:
[2021-04-08 01:38:22,533][icml2020][ERROR] - Failed after 0:00:02!
Can you help me on how to proceed for using gpu for the given code?

Hi @sharmapulkit , thank you for the inquiry. I'm sorry I had missed the notification and that the reply is so late (too late, probably?).
I did not use GPU for the experiments at the time I wrote this code, so the code has not been tested for GPU use cases.
Honestly I'm not certain if the code is complete if we need to run it with GPU (e.g., the code may not have sufficient data.to(device) statements). (Though, if the data set in the paper is used, it shouldn't take a terribly long time to train the model.)
I'm sorry that this info is probably of no use, but just wanted to let you know that the code hasn't been used with GPUs before and it may not be supported as is.
Best regards,


CI failures due to issues uploading

Describe the bug
I believe this should have been resolved by #17 however seeing an issue with failed upload requests.  Would be great to not fail the build when upload fails, however the plugin not uploading currently fails the build.
Steps to reproduce

issue is intermittent

Environment

Node version: v20.12.2
NPM version: v10.5.0
Webpack version: 4.47.0

Example code snippet

Connection timed out
   at ClientRequest.<anonymous (/home/user/node_modules_cache/9a0b663f2ebc45a0e058e7660d0fb4fc8346e175/node_modules/@bugsnag/source-maps/dist/Request.js:194:21)
   at Object.onceWrapper (node:events:632:28)
   at ClientRequest.emit (node:events:518:28)
   at TLSSocket.emitRequestTimeout (node:_http_client:845:9)
   at Object.onceWrapper (node:events:632:28)
   at TLSSocket.emit (node:events:530:35)
   at Socket._onTimeout (node:net:589:8)
   at listOnTimeout (node:internal/timers:573:17)
   at process.processTimers (node:internal/timers:514:7)

Hi @zackarychapple
Thank you for reaching out.
We now have an item on our backlog aimed at dropping these failed source maps after failing the initial retries. While I don't currently have an ETA on the release of this functionality, once we have more information to share we will be sure to update you.
For the meantime as a workaround we would recommend removing the automatic source map upload steps from your webpack config and manually upload the source maps using our CLITool as per our documentation here.


Pageserver Python tests should not fail if the server is built with no testing feature

testing feature is disabled by default and some people use make and then run tests, which fails.
We could use pageserver's version string to determine which features are enabled and expect those tests to fail/skip them.

I think this is a good idea. Separately, I wonder if we should have make by default use testing, and something like make release to build without it - just so that the default path runs more of the tests.

Hi! So before doing any PR's i would like to know ur take on this
Please take a look if u have time at the session scope in is_testing_supported and is_profiling_supported, i have done it this way to "cache" the result of the current pageserver's features for all the pytest session, if this is dangerous or not interesting i would go with a simple skipif(is_testing_supported, "blabla") or something else.
Also, the fixture go at first in the python signature test's to skip (or not) fast, without doing anything with NeonEnv.
Thanks!!!

I am a bit sceptical about the approach, since it requires a person, writing the test, to place two things independently: a failpoint and a marker on the test, it's easy to overlook.
I wonder, if NeonPageserverHttpClient::configure_failpoints method could run the same pytest.skip("pageserver was built without 'testing' feature")?
Then we could inject the feature knowledge about pageserver into the HTTP client and disallow all the maintenance HTTP calls.
That requires knowledge about which rust methods are dependent on feature call, but it only requires hardcoding in the client, so should be fine?
Also, I think rather than creating a lot of is_**_supported methods with scope="session", I would create a single scope="session" method that returns a NeonEnv object with pageserver version inside, and a few methods on it, def is_testing_enabled.
This way, we're writing more customisable code, if we decide to change the way service versions are handled.
Otherwise, thanks for the effort and thinking on this: I am now aware of important pytest.skip bit and we can think more on the implementation.

So if we want to check the features inside configure_failpoints to skip and also we want to maintain the pageserver's features under a scope="session" then i think that we would need something similar to this, because we cannot call the fixture directly we need to inject it somewhere, in this case would be _shared_simple_env or neon_env_builder to store the features inside NeonEnv otherwise we would need to pass the fixture from the executing test deep down to configure_failpoints and the spot now would be similar to before.
Why do you want a sessionize NeonEnv? In that case we would need a different fixture and specify it in each test that we want to use the configure_failpoints and i think we don't want that.
So something like:
@pytest.fixture(scope="session")
def pageserver_features:
    yield PageServerFeatures()

class PageServerFeatures:
    def __init__(self):
        # get the features from pageserver --version
    def is_testing_enabled():
        # do things

Then inject PageServerFeatures inside NeonEnv with pageserver_features

WDYT? Sounds too complicated

I imagined, we need to do things:

write a code to parse pageserver version string, at least on the level of "is testing feature is enabled"
NeonPageserverHttpClient is instantiated via NeonPageserver Python class, that has access to whole env
we can call pagesever --version using that env (might require code adjustments), parse the version, add this knowledge to NeonPageserverHttpClient
when configure_failpoints, timeline_gc or whatever else "service" HTTP method is called, do pytest.skip with a warning log or smth.

We could add more checks on top of that, I guess, such as "always be sure that CI regress tests are run with these tests enabled", but it's optional.
To me, seems simple and portable (no pytest scopes involved) enough, am I missing something?

Oh if we can check the pageserver's version each time the NeonEnv is created and forget the fixture/session-scopes thing then yeah, that sounds promising!!! My take was about the same idea of NeonEnv -> NeonPageserver -> NeonPageserverHttpClient but with a sessionize fixture so we would need the fixture builders.
Thanks! I'm on it


[ErrorException]        mkdir(): File exists

- Installing magento/core (<IP_ADDRESS>)
       Cloning 2802bec4c42fb52a12cf328fa243253e9a7ec64d

         - MagentoCoreInstaller: Installing: "magento/core" version: "<IP_ADDRESS>" to: "public"

       [ErrorException]
       mkdir(): File exists

That is weird. There is only 1 instance of mkdir and it is wrapped in a file_exists check: https://github.com/AydinHassan/magento-core-composer-installer/blob/c175c62e29c9ca3672972c9fa52b1a8325bd98ef/src/CoreInstaller.php#L62
Are the permissions incorrect on the folder? Maybe PHP thinks the file does not exist due to the permissions.
Can you run with -vvv eg composer update -vvv and post the out please?

[ErrorException]
       mkdir(): File exists

       Exception trace:
       () at /var/www/happia/happia.ru/shared/vendor/aydin-hassan/magento-core-composer-installer/src/CoreInstaller.php:62
       Composer\Util\ErrorHandler::handle() at n/a:n/a
       mkdir() at /var/www/happia/happia.ru/shared/vendor/aydin-hassan/magento-core-composer-installer/src/CoreInstaller.php:62
       AydinHassan\MagentoCoreComposerInstaller\CoreInstaller->install() at /var/www/happia/happia.ru/shared/vendor/aydin-hassan/magento-core-composer-installer/src/CoreManager.php:189
       AydinHassan\MagentoCoreComposerInstaller\CoreManager->installCore() at n/a:n/a
       call_user_func() at phar:///var/www/happia/happia.ru/shared/composer.phar/src/Composer/EventDispatcher/EventDispatcher.php:152
       Composer\EventDispatcher\EventDispatcher->doDispatch() at phar:///var/www/happia/happia.ru/shared/composer.phar/src/Composer/EventDispatcher/EventDispatcher.php:112
       Composer\EventDispatcher\EventDispatcher->dispatchPackageEvent() at phar:///var/www/happia/happia.ru/shared/composer.phar/src/Composer/Installer.php:618
       Composer\Installer->doInstall() at phar:///var/www/happia/happia.ru/shared/composer.phar/src/Composer/Installer.php:232
       Composer\Installer->run() at phar:///var/www/happia/happia.ru/shared/composer.phar/src/Composer/Command/InstallCommand.php:131
       Composer\Command\InstallCommand->execute() at phar:///var/www/happia/happia.ru/shared/composer.phar/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:257
       Symfony\Component\Console\Command\Command->run() at phar:///var/www/happia/happia.ru/shared/composer.phar/vendor/symfony/console/Symfony/Component/Console/Application.php:874
       Symfony\Component\Console\Application->doRunCommand() at phar:///var/www/happia/happia.ru/shared/composer.phar/vendor/symfony/console/Symfony/Component/Console/Application.php:195
       Symfony\Component\Console\Application->doRun() at phar:///var/www/happia/happia.ru/shared/composer.phar/src/Composer/Console/Application.php:146
       Composer\Console\Application->doRun() at phar:///var/www/happia/happia.ru/shared/composer.phar/vendor/symfony/console/Symfony/Component/Console/Application.php:126
       Symfony\Component\Console\Application->run() at phar:///var/www/happia/happia.ru/shared/composer.phar/src/Composer/Console/Application.php:82
       Composer\Console\Application->run() at phar:///var/www/happia/happia.ru/shared/composer.phar/bin/composer:43
       require() at /var/www/happia/happia.ru/shared/composer.phar:25

I've spent some time on debugging this problem. It fails on creating folder on the same place there is a symlink.

What do you think is the best solution here? Why is one of the core folders a symlink?

I am still in process. it is media

Ah okay. You could use the excludes config to not copy the media folder. See: https://github.com/AydinHassan/magento-core-composer-installer#configuration
Maybe the code should do a check and display a better error message if there is a symlink there.

My bad. It was symlink to non-existing folder.
Your code works well. I can only suggest to catch exception and show problematic folder.
And you can probably add code like below:
if (is_link($destinationFile)) {
    $destinationFile = readlink($destinationFile);
}

NP. I'll have a play around when I get sometime :). Shall we close this off?

up to you :)

@luckyraul I added a check for broken symlinks :)


finish docs for processing view

[x] add wire frame for processing

added wireframe image to desktop processing section


Simple GUI Calculator

Simple GUI Calculator

Make a folder named "GUI_Calculator"
Make a subfolder with your github handle as the name.
For example mine would be pvgupta24

Note

Implement in any language .
E.g ,  in Java by using libraries like Java AWT, Java Swing, Java FX.
Read about basics of  Java Swing, Java AWT.
Include a README.md file with usage instructions and limitations.

Lookup CONTRIBUTING.md document for more instructions.

Give links to read up on Java Swing, AWT, etc....

I Claim

I claim

I Claim

I Claim!

I claim

I Claim

i claim

I would like to work on this issue.

I claim

Issue closed, end of Hacktoberfest!


ADD to CONTRIB: single_image_random_do_stereograms for data analysis

I have written a TensorFlow OP "single_image_random_dot_stereograms" which takes a 2 D array with Z values and outputs a Stereogram image for use in "encode_png/jpg".
it is based upon this paper:
http://www.learningace.com/doc/4331582/b6ab058d1e206d68ab60e4e1ead2fe6e/sirds-paper
I am sure this can be used to output data to TensorBoard as well but haven't tried that yet.
I would like to offer to add this as a contrib for TensorFlow and would like some advice if there is interest on getting the process started.  Right now I compile it into the "user_ops" section, but think it needs to move for contrib.
Basically, this helps display 3D data without hidden lines like a "waterfall" plot.
Here is the basic code as a demo:
img=[[1,2,3,3,2,1],
     [1,2,3,4,5,2],
     [1,2,3,4,5,3],
     [1,2,3,4,5,4],
     [6,5,4,4,5,5]]

session = tf.InteractiveSession()

sirds = single_image_random_dot_stereograms(img,convergence_dots_size=8,number_colors=256,normalize=True)

out = sirds.eval()

png = tf.image.encode_png(out).eval()

with open('picture_out.png', 'wb') as f:
    f.write(png)

The output image then looks like the attached image.  If you can view this image outside of github it will look better.  Try to look beyond the picture so the 2 dots at the bottom appear to be 3 dots.  The image should then converge and see the shape as described in "img" above.

If there is interest, I can breakup the code to better match one of the contrib subdirectories and start a pull request.
Here is a different image, inverted cone, windowed in full color.

Very cool stuff @Mazecreator , thanks for sharing!
@martinwicke Can you give some advice on whether this should go into contrib, or what other options there might be?  Thanks!

contrib is the right place I'd say.

Sounds good, how about adding it to the "/contrib/image/" section?  I can add it into this directory, there is a few ops there already.
If this is okay, I will work on getting a pull request together over the next few days so you have something to look at.

That is the perfect location. Please send a PR.

3rd times a charm.
I have created a quick overview of the Op here:
https://github.com/Mazecreator/TensorFlow-SIRDS

Seems fixed.


[fix] Fix online SPMD algorithms finalize call

Changes proposed in this pull request:

Fix finalize method for all four online SPMD algorithms, now the call does not affect partial results since all necessary tables are copied. It allows to call finalize between calls of partial_fit to obtain intermediate results
Add ONEDAL_DATA_PARALLEL to finalize implementations where they were forgotten

/intelci: run

/intelci: run

http://intel-ci.intel.com/ef65eab3-b487-f188-a43a-a4bf010d0e2e


Fix verified.asn permission setting through peeringdb oauth

fixes #76

Codecov Report

:exclamation: No coverage uploaded for pull request base (prep-release@3b57e47). Click here to learn what that means.
The diff coverage is n/a.

@@               Coverage Diff               @@
##             prep-release      #77   +/-   ##
===============================================
  Coverage                ?   66.90%
===============================================
  Files                   ?       80
  Lines                   ?     1139
  Branches                ?        0
===============================================
  Hits                    ?      762
  Misses                  ?      377
  Partials                ?        0

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


merge package from Kronos-Tools/npm-package-template-minimal

package.json

chore(package<EMAIL_ADDRESS>chore(package): add module from template

Coverage remained the same at 14.543% when pulling 89ec7e3d9a4f561270640768dd310b6ef483358d on template-sync-1 into c120b50de9207e249016c5a6a9e1b2150608e6d8 on master.


[FLINK-6775] [state] Duplicate StateDescriptor's serializer

Duplicate the TypeSerializer before returning it from the StateDescriptor. That way
we ensure that StateDescriptors can be shared by multiple threads.

Thanks for the quick review @aljoscha. Yes my filter condition was *StateDescriptorTest. Will add a test for the AggregatingStateDescriptor.

LGTM, really like the change on the testCorrectClassLoaderUsedOnSnapshot test!

Thanks for the review @tzulitai and @aljoscha. Failing test case is unrelated. Merging this PR.


Add authorizedMint and whitelist Reservoir Relay EOA

Add authorizedMint in 1155 and 721
Whitelisted Reservoir Relay EOA by default

If that EOA is compromised or the relationship with the EOA changes I'm not seeing any protections that can ever remove it. I think the _authorizedMinters mapping needs add/remove methods and a process for managing access to those functions.
What is the motivation for setting the EoA as a constant if it can be passed as a constructor param?

If that EOA is compromised or the relationship with the EOA changes I'm not seeing any protections that can ever remove it. I think the _authorizedMinters mapping needs add/remove methods and a process for managing access to those functions.
What is the motivation for setting the EoA as a constant if it can be passed as a constructor param?

Good suggestion. My initial intention is to make Reservior Relay fixed so it cannot be modified. But after some thoughts I think it's  better to make the list mutable (by owner only).

Should work.
In the future what do you think about something like AuthorizedMinter.sol and inherit/extend but it's not necessary in this PR, I think.

Should work.
In the future what do you think about something like AuthorizedMinter.sol and inherit/extend but it's not necessary in this PR, I think.

I think we could switch to use OZ's Access Control / Role to consolidate owner and authorizer.

worth refactoring later, but yeah, lgtm for now, that we delay the introduction of another Roles dependency


OWLS-91043, added test to Update the WDT version on a running domain with auxiliary image

Update the WDT version on a running domain with common mount.
Create a domain with common mount for domain config and a common mount for WDT only.
Patch the WDT common mount with newer WDT version (by creating a image with latest WDT version for the common mount).
https://build.weblogick8s.org:8443/job/weblogic-kubernetes-operator-kind-new/5794/
https://build.weblogick8s.org:8443/job/weblogic-kubernetes-operator-kind-new/5793/

LGTM


Shader Processor: loop over shader until everything has been processed

Objective

I want to be able to use #ifdef and other processor directives in an imported shader

Solution

Process the shader string in a loop until nothing remains to be processed

I don't think we actually need to loop here

It's to have the possibility of nested imports. It's probably not necessary to loop over everything until nothing changed, but I don't think it should be too long to process text...
I added a test for nested import.

no more loop...

Awesome. Thats exactly what I was thinking of :)

bors r+


New article

Please use this template to ensure easier processing of your pull request
Category

[ ] Content fix
[ ] New article
[x] Example checked item (delete this line)

For the above list, an empty checkbox is [ ] as in [SPACE]. A checked checkbox is [x] with no space between the brackets. Use the PREVIEW tab at the top right to preview the rendering before submitting your issue.
(DELETE THIS PARAGRAPH AFTER READING)

Related issues

fixes #issuenumber
partially #issuenumber
mentioned in #issuenumber

If this fixes (should close) or references an issue, please add the issue number here. When you type the hashtag symbol (#), a list of potential issues will pop up from which you can choose. This helps those maintaining the issue list as it will (1) link the PR to the issue & (2) automatically close the issue when this PR is merged in.
(DELETE THIS PARAGRAPH AFTER READING)

Contents of the Pull Request

Please describe the changes in this PR. Try to give enough details so the person reviewing it to merge it can make a good decision about your content.
(DELETE THIS PARAGRAPH AFTER READING)

Guidance

Please update this PR information accordingly. We'll use this as part of our release notes in monthly communications.
Please target your PR to master branch. Released documents are in live branch.
(DELETE THIS PARAGRAPH AFTER READING)

Docs Build status updates of commit b924e75:
:warning: Validation status: warnings

File
Status
Preview URL
Details

Community/power-automate-send-sharepoint-files-as-attachments.md
:warning:Warning
View
Details

Community/media/power-automate-send-sharepoint-files-as-attachments/powerautomate-sendasattachment01.png
:white_check_mark:Succeeded
View

Community/media/power-automate-send-sharepoint-files-as-attachments/powerautomate-sendasattachment02.png
:white_check_mark:Succeeded
View

Community/media/power-automate-send-sharepoint-files-as-attachments/powerautomate-sendasattachment03.png
:white_check_mark:Succeeded
View

Community/media/power-automate-send-sharepoint-files-as-attachments/powerautomate-sendasattachment04.png
:white_check_mark:Succeeded
View

Community/media/power-automate-send-sharepoint-files-as-attachments/powerautomate-sendasattachment05.png
:white_check_mark:Succeeded
View

Community/media/power-automate-send-sharepoint-files-as-attachments/powerautomate-sendasattachment06.png
:white_check_mark:Succeeded
View

Community/media/power-automate-send-sharepoint-files-as-attachments/powerautomate-sendasattachment07.png
:white_check_mark:Succeeded
View

Community/media/power-automate-send-sharepoint-files-as-attachments/powerautomate-sendasattachment08.png
:white_check_mark:Succeeded
View

Community/media/power-automate-send-sharepoint-files-as-attachments/powerautomate-sendasattachment09.png
:white_check_mark:Succeeded
View

Community/media/power-automate-send-sharepoint-files-as-attachments/powerautomate-sendasattachment10.png
:white_check_mark:Succeeded
View

Community/media/power-automate-send-sharepoint-files-as-attachments/powerautomate-sendasattachment11.png
:white_check_mark:Succeeded
View

Community/media/power-automate-send-sharepoint-files-as-attachments/powerautomate-sendasattachment12.png
:white_check_mark:Succeeded
View

Community/media/power-automate-send-sharepoint-files-as-attachments/powerautomate-sendasattachment13.png
:white_check_mark:Succeeded
View

Community/power-automate-send-sharepoint-files-as-attachments.md

Line 29, Column 0: [Warning-hard-coded-locale] Link 'https://docs.microsoft.com/en-us/power-automate/' contains locale code 'en-us'. For localizability, remove 'en-us' from links to most Microsoft sites.
Line 34, Column 1: [Suggestion-code-block-indented] Indented code blocks aren't allowed. Use a Markdown code block surrounded by triple backticks (```). NOTE: This Suggestion will become a Warning in late September, 2020.
Line 43, Column 1: [Suggestion-duplicate-alt-text] Alt text 'image' is duplicated. Within an article, alt text must be unique. NOTE: This Suggestion will become a Warning in late October, 2020.
Line 47, Column 1: [Suggestion-duplicate-alt-text] Alt text 'image' is duplicated. Within an article, alt text must be unique. NOTE: This Suggestion will become a Warning in late October, 2020.
Line 51, Column 1: [Suggestion-duplicate-alt-text] Alt text 'image' is duplicated. Within an article, alt text must be unique. NOTE: This Suggestion will become a Warning in late October, 2020.
Line 55, Column 1: [Suggestion-duplicate-alt-text] Alt text 'image' is duplicated. Within an article, alt text must be unique. NOTE: This Suggestion will become a Warning in late October, 2020.
Line 59, Column 1: [Suggestion-duplicate-alt-text] Alt text 'image' is duplicated. Within an article, alt text must be unique. NOTE: This Suggestion will become a Warning in late October, 2020.
Line 63, Column 1: [Suggestion-duplicate-alt-text] Alt text 'image' is duplicated. Within an article, alt text must be unique. NOTE: This Suggestion will become a Warning in late October, 2020.
Line 68, Column 1: [Suggestion-code-block-indented] Indented code blocks aren't allowed. Use a Markdown code block surrounded by triple backticks (```). NOTE: This Suggestion will become a Warning in late September, 2020.
Line 74, Column 1: [Suggestion-duplicate-alt-text] Alt text 'image' is duplicated. Within an article, alt text must be unique. NOTE: This Suggestion will become a Warning in late October, 2020.
Line 78, Column 1: [Suggestion-duplicate-alt-text] Alt text 'image' is duplicated. Within an article, alt text must be unique. NOTE: This Suggestion will become a Warning in late October, 2020.
Line 83, Column 1: [Suggestion-duplicate-alt-text] Alt text 'image' is duplicated. Within an article, alt text must be unique. NOTE: This Suggestion will become a Warning in late October, 2020.
Line 97, Column 1: [Suggestion-duplicate-alt-text] Alt text 'image' is duplicated. Within an article, alt text must be unique. NOTE: This Suggestion will become a Warning in late October, 2020.
Line 101, Column 1: [Suggestion-code-block-indented] Indented code blocks aren't allowed. Use a Markdown code block surrounded by triple backticks (```). NOTE: This Suggestion will become a Warning in late September, 2020.
Line 107, Column 1: [Suggestion-duplicate-alt-text] Alt text 'image' is duplicated. Within an article, alt text must be unique. NOTE: This Suggestion will become a Warning in late October, 2020.
Line 112, Column 1: [Suggestion-duplicate-alt-text] Alt text 'image' is duplicated. Within an article, alt text must be unique. NOTE: This Suggestion will become a Warning in late October, 2020.
Line 119, Column 1: [Suggestion-duplicate-alt-text] Alt text 'image' is duplicated. Within an article, alt text must be unique. NOTE: This Suggestion will become a Warning in late October, 2020.

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit e2b41e4:
:warning: Validation status: warnings

File
Status
Preview URL
Details

Community/power-automate-send-sharepoint-files-as-attachments.md
:warning:Warning
View
Details

.github/PULL_REQUEST_TEMPLATE.md
:white_check_mark:Succeeded

Community/media/power-automate-send-sharepoint-files-as-attachments/powerautomate-sendasattachment01.png
:white_check_mark:Succeeded
View

Community/media/power-automate-send-sharepoint-files-as-attachments/powerautomate-sendasattachment02.png
:white_check_mark:Succeeded
View

Community/media/power-automate-send-sharepoint-files-as-attachments/powerautomate-sendasattachment03.png
:white_check_mark:Succeeded
View

Community/media/power-automate-send-sharepoint-files-as-attachments/powerautomate-sendasattachment04.png
:white_check_mark:Succeeded
View

Community/media/power-automate-send-sharepoint-files-as-attachments/powerautomate-sendasattachment05.png
:white_check_mark:Succeeded
View

Community/media/power-automate-send-sharepoint-files-as-attachments/powerautomate-sendasattachment06.png
:white_check_mark:Succeeded
View

Community/media/power-automate-send-sharepoint-files-as-attachments/powerautomate-sendasattachment07.png
:white_check_mark:Succeeded
View

Community/media/power-automate-send-sharepoint-files-as-attachments/powerautomate-sendasattachment08.png
:white_check_mark:Succeeded
View

Community/media/power-automate-send-sharepoint-files-as-attachments/powerautomate-sendasattachment09.png
:white_check_mark:Succeeded
View

Community/media/power-automate-send-sharepoint-files-as-attachments/powerautomate-sendasattachment10.png
:white_check_mark:Succeeded
View

Community/media/power-automate-send-sharepoint-files-as-attachments/powerautomate-sendasattachment11.png
:white_check_mark:Succeeded
View

Community/media/power-automate-send-sharepoint-files-as-attachments/powerautomate-sendasattachment12.png
:white_check_mark:Succeeded
View

Community/media/power-automate-send-sharepoint-files-as-attachments/powerautomate-sendasattachment13.png
:white_check_mark:Succeeded
View

Community/power-automate-send-sharepoint-files-as-attachments.md

Line 29, Column 0: [Warning-hard-coded-locale] Link 'https://docs.microsoft.com/en-us/power-automate/' contains locale code 'en-us'. For localizability, remove 'en-us' from links to most Microsoft sites.
Line 34, Column 1: [Suggestion-code-block-indented] Indented code blocks aren't allowed. Use a Markdown code block surrounded by triple backticks (```). NOTE: This Suggestion will become a Warning in late September, 2020.
Line 68, Column 1: [Suggestion-code-block-indented] Indented code blocks aren't allowed. Use a Markdown code block surrounded by triple backticks (```). NOTE: This Suggestion will become a Warning in late September, 2020.
Line 101, Column 1: [Suggestion-code-block-indented] Indented code blocks aren't allowed. Use a Markdown code block surrounded by triple backticks (```). NOTE: This Suggestion will become a Warning in late September, 2020.

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel


FP style

Spect may come useful in audio-js as:
import output from 'audio-speaker'
import oscillate from 'audio-oscillator'
import spect from 'spect'

spect.use(output, oscillate)

// initial code
let buf = new AudioBuffer({ channels: 2, sampleRate: 44100, length: 1024 })
;(async function frame() {
  await output(oscillate(buf, { frequency: 440 })))
  frame()
})()

// aspect code
spect(new AudioBuffer({ channels: 2, sampleRate: 44100, length: 1024 })).use(({ oscillate, output, update }) => {
  oscillate(440)
  await output()
  update()
})

Isn't that beautiful to just have linked effects as
use(new AudioBuffer({ channels: 2, sampleRate: 44100, length: 1024 }), (buf) => {
  oscillate(buf, 440)
  await output(buf)
  update(buf)
})

#43

8.0.0 is functional style.


As a home baker with multiple orders, I can add different orders with custom details such as customer, delivery date

so that I can fulfill the orders and i can keep track of orders that i currently have

Can we close this issue?


Fix README and prometheus_metrics_parsing

This PR replaces all the mentions of depricated grpc_connection_forwarder to grpc_load_balancer in README.
It also adjusts logic for prometheus_metrics_parsing function to account for prometheus labels and always take last numeric value from the line. Line iterator has been replaced with regular expression, because on my tests it works 1.6 times faster than line iterator (0.52375s vs 0.32459s on 100000 iterations). This also allows for potentially different regular expressions to be passed during initialization to make the module more customizable in the future.

The entire regular expression rf"^{self.metrics_name}(?:\s*{{[^}}]*}})?\s+(\d+\.\d+)" matches lines starting with the given prefix, followed by optional curly braces containing any characters except closing curly braces, and then captures a floating-point number at the end of the line.


Simplify the naming?

To create a linear scale I run something like -
const x = d3_scale.scaleLinear()

However, I had assumed I would instead do -
const x = d3_scale.linear()

It's my first time playing with the new modules and I know it's easy to go blind to these things, so thought I'd raise the issue. It's obviously not a big deal but thought it might be useful feedback.

I’m aware of this issue, but I’m not sure how to make things any clearer. See mbostock/d3#2749 for one idea which I have rejected, and d3/d3-queue#56 for a similar issue.


feat: improve behavior of HTTP redirects

This commit introduces a new okhttp interceptor that will implement customized behavior for HTTP redirects.
Specifically, "safe" headers will be included in a redirected request if the original and redirected hosts are the same, or are both within IBM's "cloud.ibm.com" domain.

Put this back into draft status until we get an approval from CISO.


Add a "Run Script" phase

Context 🕵♀
I am working on another project that uses SPM, and I want to apply SwiftLint inside of that project. Given that I already have a bash script to re-generate the xcodeproj (and do some other administrative tasks at the same time), what I want to do is include something in that bash script that can go into the generated xcodeproj and include a Run Script phase at the end that will run SwiftLint, so that I can get the warnings in Xcode for that project on build, rather than relying on the command-line.
I figured out how to do this with the Ruby xcodeproj library maintained by the Cocoapods folks. However, I'd love to stick inside Swift, and was checking out this library.
What 🌱
I am unable to figure out how to create a new Run Script build phase, so that I can append it to the currently-existing build phases, so I can accomplish my goal.
Is this something I can even do with the library currently, and if so, can someone point me in the right direction?

Naturally, right after I post this, I find PBXShellScriptBuildPhase. Sigh.

(I am not, however, so far able to figure out how to append that to the phases and save and have it persist)

Naturally, right after I post this, I find PBXShellScriptBuildPhase. Sigh.

You are in the right direction. So what you need to do is:

Create an object of that type.
Add it to the project
Add it to the target.

It should show up pin the list of build phases of the target. In case you don't know how to get the list of targets of the project to find the one that you are interested in, you can traverse this attribute and find it by its name.
Let me know if that worksk @kdbertel. I'm happy to help you figure out how to do it.

Aha, I was missing that I had to add it to both the project and the target. That's a little unintuitive, but probably has to do with the actual underlying project structure?

For what it's worth, my problem has been solved. This issue can be closed unless we want further discussion on future API directions


Updated Installer for 32bit support

Sorry, this repository is long-since deprecated in favor of Docker Toolbox (whose usage is now also discouraged in favor of Docker for Windows and Docker for Mac).


[util] Add functions to update values atomically

This PR adds new utility functions to update their properties atomically instead of acquiring locks. It also does some minor refactoring now that we have functions for updating with a lock and updating atomically.
cc @xichen2020 @cw9 @prateek

Coverage increased (+0.0007%) to 84.233% when pulling d5e500bd8c13c41c40c1f7022f993fc9a56c3c07 on jeromefroe/util/add-atomic-update-functions into cd50ade8178668afc23d7464b9ea5e1f451af89a on master.

LGTM


fix(rating): fix keyboard navigation styles bug on Safari

[x] I have followed (at least) the PR section of the contributing guide.

This PR fixes https://github.com/facebook/react/issues/7407; feels a little hacky but there is no better way to do it as far as my knowledge goes

yes, can be tested via this demo: https://mui.com/material-ui/react-rating/#main-content

Again, how can we reproduce the issue?

Ugh, sorry, linked the wrong issue in description - updated the link, the problem in the  issue at hand is 100% reproducible


update cilium-docker-plugin, cilium-operator to golang 1.12.8

Signed-off by: Ian Vernon<EMAIL_ADDRESS>

This change is

test-me-please

test-me-please

Coverage increased (+0.003%) to 44.191% when pulling 47c9256a84e8c8b0aed9d90aa7029a39b3bacd19 on pr/ianvernon/update-golang-versions-docker-operator into 0c7f066529815f4d51f64b709d6a52fba8ee8b25 on master.

test-me-please

@joestringer no, the builds for Cilium operator and the docket plugin are self contained. I bumped the versions in Cilium’s images (builder and runtime) and for Travis , but missed these. All of our images are up to date now with Golang 1.12.8.


chore(main): release 0.3.0

:robot: I have created a release beep boop
0.3.0 (2024-08-28)
Features

add a retry mechanism for API call (#166) (3d89da3)
Add jinja2 templates, add github workflow (7f53c6a)
Add support for Python 3.11 (#87) (03aa815)
Add tmp_path property on workspace (#83) (c944b6d)
Check that docker exists and is running (#136) (3d4d75f)
CI/CD: adds github action v4 config (ce8f848)
CLI: check if permission_denied is thrown (#129) (2f856fe)
CLI: Use blsq/openhexa-blsq-environment as the image to run the pipeline (#115) (c813e3a)
CLI: User can add --yes to the  command to not have to confirm the creation of the pipeline (85a9098)
CLI: User can provide a name, description and link of the pipel… (#142) (45fd2e7)
Connections: add support for IASO connection (#82) (707b798)
Connections: import and re-export connection classes in openhexa.sdk (#93) (2e3db63)
Dataset: Create dataset with the SDK (#110) (cba3e7e)
DatasetFiles: add helper to check if file exists (c76d890)
Datasets: Add datasets support to SDK (#77) (d8eb50e)
delete pipeline (#91) (549f73d)
Parameter: add support for Dataset parameter type (#120) (783b672)
Pipelines: allow user to define a timeout for pipelines (0eeab40)
Pipelines: Allow user to download the latest version of a pipeline (#128) (2dae69d)
Pipelines: prevent push of version  with params for scheduled pipeline. (#55) (b9de756)
Pipeline: Use the AST to get the specs of the pipeline and its parameters (#130) (9515499)
Run: Run pipelines with the debugger ready to be used (a67d0fe)
Run: Run pipelines with the debugger ready to be used (#157) (754c21b)
Run: Run pipelines with the debugger ready to be used (#157) (6863173)
Update SDK version in Dockerfile (#57) (208d7ea)
Use a new way to load settings (#131) (9619673)
Workspaces: allow user to specify workspace docker image (#95) (2455a3e)
Workspaces: get connections from graphql API (#158) (bba409b)
Workspaces: list datasets in a workspace (#112) (916832a)

Bug Fixes

API: uploading a version now returns a pipelineVersion instead of a version (5f89738)
Build: Manifest.in was missing the .github folder (8afa0eb)
ci: Fix python3.9 tests (3d9aa95)
CLI: Ask the user to create the workspace.yaml if not found (#109) (1da1a04)
CLI: Check if args of decorators are of a primitive type (#146) (3ca0f51)
CLI: Do not ask to create the pipeline if the user has said yes as param (6597876)
CLI: Do not require a config even if it's empty (#50) (8e1098f)
Cli: no error is thrown when workspace doesn't exist (#153) (fbbec07)
Cli: no error is thrown when workspace doesn't exist (#153) (cefc69c)
CLI: Notebook pipelines make the pipelines list to crash (374c8fb)
CLI: Running a pipeline should not require an active workspace (#126) (7618d91)
Conda: Copy stringcase package into our repo (e6345b2)
Conda: Update requirements conditions (7d01ad3)
Connections: hide sensitive informations when printing  connection (#79) (db749a3)
Connections: Rename 'slug' to 'identifier' (7f52ce4)
Connections: Set the slug to lowercase before building the env var (7f52ce4)
Connections: specify connection workspace (#170) (f8b2dda)
Connections: use lowercase for identifier (#184) (a864629)
copies files correctly to workspace directory (f6a19b2)
copy workspace folder to correct path (6a1a6a4)
CurrentRun: check if file exist when adding output (#114) (4dfe8d1)
dataset: Support bytes directly in read_content (#111) (e83eefb)
Do not crash the CLI when some pipelines do not have a current version (3b8cd5c)
Do not make two requests to the server to check if the pipeline exists (#94) (4501bda)
Docker: Always force pull the image (#103) (c538e10)
Docker: Disable healthchecks when running the pipeline locally (c1b8535)
Only add debugging ports when it's enabled (4a9f17d)
Parameter: check if default value is in choices (#123) (726df4e)
Parameters: add support for custom connection (#116) (65db931)
Pipelines: prevent upload of workspace folder content (#68) (957c277)
Pipelines: set timeout when uploading a version (#72) (7703794)
Proper error messages for arguments and missing config file (#51) (07a5027)
read_content: bineary files do not take an encoding (5448287)
remove duplicated instruction (137a90d)
Run: In local runs using docker, the configuration from the workspace.yaml should not be parsed and loaded (e4bb7a3)
Skeleton: Add '--yes' to the push action to create the pipeline by default (0013039)
typing: Add all accepted types (96d4170)
urllib: Use urllib < 2 to avoid problemw with openssl being not uptodate (#160) (bb0c98a)
use proper namespace package setup (#53) (81fc51e)
Utils: run pipeline config with right path (99cd6f2)

Documentation

Add docstrings for @pipeline and @parameter (#70) (6282a50)
update parameter supported types (#90) (b3b6740)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/BLSQ/openhexa-sdk-python/releases/tag/v0.3.0 :sunflower:


Rename MutedTopic to UserTopic

Part of #19272

This lgtm; posted one small comment (and the database migrations needs a rebase to renumber them)

@timabbott Thanks for the review. I've rebased the PR and addressed the above comment.

Merged after some small tweaks to the comments in models.py.  Thanks @abhijeetbodas2001!


Tech Meetup & Chapter Chengdu Kickoff 🚀 🌔😀

We are planning to have tech meetup session(protoschool chengdu kickoff) of proto-school meet-up this month on July 21 . In this session, we are going to cover the following topics:

Paxos,PoW,VDF: a beautiful continuous (https://mp.weixin.qq.com/s/R1FNNJkjtSQNomrd4lIN-Q)
IPFS Camp sharing
IPFS in medical insurance

Please see the meet-up post from: https://www.meetup.com/Shanghai-Decentralized-Systems-Meetup-Group/events/262865153/ for the details.
the attachment is post of this meetup.

recall the meeting (in Chinese Language) :
https://github.com/ProtoSchool/chengdu/blob/master/meetup/07212019/活动回顾--『协议学院 · 成都分院 · 开院典礼』.md


Make connection constant optional

Means connection does not have to be selected when extension is installed.
Permissions still work despite a connection not being associated with the model. The model is happy because it has a constant defined. It does not seem to care that the constant value is empty. Auto install no longer has to define a value for connection.
Suggest we modify data dictionary and lookml diagram. We should let this bake in for a few days before doing so though.

I've got a trivial branch in flight that's prepped for the APIX updates, but someone else could quickly replace what I've done. We need to deploy an updated APIX extension shortly anyway.


Enhanced Dependency Management for Cross-Platform Compatibility.

Purpose
This Pull Request addresses compatibility issues with the installation of dependencies on different operating systems. It modifies the requirements.txt file to ensure seamless installation for both Linux and Windows environments, allowing for better usability across diverse systems.
Fixes for issue #160
Does this introduce a breaking change?
[ ] Yes
[x] No

Pull Request Type
What kind of change does this Pull Request introduce?
[ ] Bugfix
[x] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Documentation content changes
[ ] Other... Please describe:

How to Test
git clone https://github.com/huntingcodes-001/openai-chat-app-quickstart.git
cd openai-chat-app-quickstart
pip install -r requirements.txt

Test the code

1. Run the application using the command: `python app.py`
2. Verify that all features work as expected on both Windows and Linux environments.
3. Check for successful installation of all required dependencies without errors.

What to Check
Verify that the following are valid
-All dependencies are installed without issues on both Windows and Linux systems.
-The application runs smoothly and all functionalities are intact after the changes.
Other Information
Ensure that the development environment reflects the changes made in this PR. It may be helpful to run the tests on both operating systems to verify compatibility.

Thanks @huntingcodes-001 for the PR. The requirements.txt file is generated by this command:
pip-compile --output-file=requirements.txt pyproject.toml
Will that retain the marker for the windows OS, or will it override when we run -U? I thought it would override.

The other option:
requirements.in could bring in just uvicorn, not uvicorn[standard], and then bring in uvloop with the environment marker you have in this PR.
Here's what standard brings in:
https://github.com/encode/uvicorn/blob/6ffaaf7c2f78274889da1572832dd307a8b117d3/pyproject.toml#L39

You're correct that using pip-compile to regenerate the requirements.txt file will override any manual modifications, including the OS markers.
To ensure that the requirements are preserved for both Windows and Linux, we can consider adding a comment in the requirements.in file, indicating that the OS-specific installations should remain intact. Alternatively, we could explore using separate requirement files for different platforms if that fits better with our workflow.

Bringing in just uvicorn in requirements.in and specifying uvloop with an environment marker sounds like a solid approach. This would allow us to keep the installation lean while still accommodating the platform-specific needs.

Thanks for breaking that down!
Based on your suggestions, I’ll update the requirements.in to include the following:
uvicorn without [standard]
uvloop with the platform-specific marker
httptools (since it’s recommended)
watchfiles (for --reload functionality)
I'll leave out colorama, PyYAML, and websockets as discussed. This should keep the dependencies lean while ensuring the necessary functionality remains intact.

Sounds good, thanks!

I just merged the fix that I suggested, so am closing this, thanks for sending the PR in!


Build failure: comby fails to build for aarch64-darwin on 24.11

Steps To Reproduce
Steps to reproduce the behavior:
nix build github:nixos/nixpkgs/nixos-24.11#comby on aarch64-darwin
Build log
$ nix build github:nixos/nixpkgs/nixos-24.11#comby
error: builder for '/nix/store/fc153ggh99ccngylbg6fgjxzqx58pw77-ocaml4.14.2-mirage-crypto-rng-1.1.0.drv' failed with exit code 1;
       last 20 log lines:
       > Running phase: unpackPhase
       > unpacking source archive /nix/store/xvyq60axrflddm81wpx4mqvkxzy76z5n-mirage-crypto-1.1.0.tbz
       > source root is mirage-crypto-1.1.0
       > setting SOURCE_DATE_EPOCH to timestamp<PHONE_NUMBER> of file mirage-crypto-1.1.0/tests/x25519_test.json
       > Running phase: patchPhase
       > Running phase: updateAutotoolsGnuConfigScriptsPhase
       > Running phase: configurePhase
       > no configure script, doing nothing
       > Running phase: buildPhase
       > Running phase: checkPhase
       > File "tests/dune", line 41, characters 7-19:
       > 41 |  (name test_entropy)
       >             ^^^^^^^^^^^^
       > (cd _build/default/tests && ./test_entropy.exe)
       > 1211 bd46 Fatal error: exception Failure("same data from timer at 4")
       > (cd _build/default/tests && ./test_random_runner.exe)
       > .........
       > Ran: 9 tests in: 0.20 seconds.
       > OK
       >
       For full logs, run 'nix-store -l /nix/store/fc153ggh99ccngylbg6fgjxzqx58pw77-ocaml4.14.2-mirage-crypto-rng-1.1.0.drv'.
error: 1 dependencies of derivation '/nix/store/hm71ybrmidmyzj7pmg2wdlxff26fh7nq-ocaml4.14.2-mirage-crypto-ec-1.1.0.drv' failed to build
error: 1 dependencies of derivation '/nix/store/g1qpr20djjj1r22456mjk1wvja17viy2-ocaml4.14.2-tls-1.0.2.drv' failed to build
error: 1 dependencies of derivation '/nix/store/yp1cgm93hn2yjcrqzir8glkknhdjbzpp-ocaml4.14.2-comby-1.8.1.drv' failed to build

Additional context

nix build github:nixos/nixpkgs/nixos-24.05#comby succeeds on aarch64-darwin.
Metadata

 - system: `"aarch64-darwin"`
 - host os: `Darwin 23.4.0, macOS 14.4.1`
 - multi-user?: `yes`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.18.5`
 - channels(root): `"nixpkgs"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixpkgs`

Notify maintainers

Note for maintainers: Please tag this issue in your PR.

Add a :+1: reaction to issues you find important.

this is a bug in https://github.com/mirage/mirage-crypto/issues/216. they use a counter that increments at 24mhz and expect a new value every call.

nix build github:nixos/nixpkgs/nixos-24.11#comby

I see...it seems specific to Darwin though? This builds fine on aarch64-linux.


Revert "fix: allow specifying a base for source maps"

Temporarily reverting this one because it causes source maps to have blank source file names and there's too much to deal with WRT the latest swc upgrade (so let's do a follow up PR).

Reverts denoland/deno_ast#87
cc @kitsonk

I think the base shouldn't generally be the same file as a file in the source map.  Was there a specific scenario where the screen grab was from?

@kitsonk yeah, this is v2::tests::import_map in eszip
It seems they are the same because of this line?
https://github.com/denoland/deno_ast/pull/92/files#diff-c865a1cff44ff278a457dcd63af4492767b0f67655e139b489dd961b5c1b8e3eL182

Ok, I think the fix would be to guard against equality and to just output the basename in such cases.  I guess eszip keeps the same files indexed by their original filename versus an output filename.  Should be a straight forward fix.


chore: set lf eol in .gitattributes

:tada: This PR is included in version 1.2.2 :tada:
The release is available on:

GitHub release
npm package (@latest dist-tag)

Your semantic-release bot :package::rocket:


[1] Vendor upstream charts

Is your feature request related to a problem? Please describe.
We're currently moving towards vendoring upstream charts.
See: https://github.com/elastisys/compliantkubernetes-apps/pull/444
We should do the same in this repository. Specifically rook.
Describe the solution you'd like
Vendor upstream charts.

This was completed with https://github.com/elastisys/compliantkubernetes-kubespray/pull/284


Screenshot get err data in image/png format

I got the bytes by the code below:
var res = await controller.player.screenshot(format: 'image/png');

And I use await file.writeAsBytes(res); to write it,but the picture is purly dark.

+1 on this
My current workaround is to just use player.screenshot() or  player.screenshot(format: 'image/jpeg') and save it into a .png file


Add support for sassOptions.includePaths to configure sass compiler

Which project is this feature idea for?
Turbopack
Describe the feature you'd like to request
Turbo repo does not works if sassOptions.includePaths is present.
I guess this configuration is not used that often by others, but in my workflow, it is very much needed. Here is what my next.config.js looks like
// next.config.js
module.exports = nextObfuscator({
  experimental: { appDir: true },
  sassOptions: { includePaths: [path.join(__dirname, 'styles')] },
})

I use tailwind inside CSS modules, that's the main reason to use sassOptions.includePaths
Describe the solution you'd like
I know nothing about how turbopack works behind the scenes. but you can use previous sass compiler that can understand sassOptions.includePaths only if it is present
Describe alternatives you've considered
My alternative solution would be to add native support for tailwind, 😅

Meanwhile, maybe this can help:
https://nextjs.org/docs/api-reference/next.config.js/turbopack#webpack-loaders

https://github.com/vercel/next.js/pull/49882

Still getting

Next v13.4.5

Still getting

Next `v13.4.5`

Fixed with:
https://github.com/vercel/next.js/pull/51259

The main reason I am looking for sassOptions is to use absolute paths in sass files. Something like this
@import 'styles/components.sass'
...

It seems you don't need really need to customise sassOptions if you are using postcss and tailwind. You can use absolute imports without any comfiguration.


Not able to debug node application

VSCode Version: 1.26
Node Version: 6.9.1

I am new to the vscode. i am able to run my node js application using Debug > "Start Without Debugging" option.

But when i use Debug > "Start Debugging" option then its giving following warning and server is not starting.

Following is my launch.json file.
{
"configurations": [
{
"type": "node",
"request": "launch",
"name": "Launch Program",
"program": "${workspaceFolder}/cluster_app.js"
}
]
}
After reading online solution i have made following changes to launch.json file.
"protocol": "inspector"
So now its not showing any warning but still server not starting.

Please let me know where i am doing wrong.

Install a later node version, you have 6.9
If that does not work than pleas ask on stack overflow

Thanks for response @isidorn .
Is there any workaround if i want to debug with node version 6.9? Because my application build on 6.9 version and we can't change version this point of time.

Found solution.
created script in package.json file as below
"debug": "node --inspect  server.js"
then started server as "npm run debug"
then create launch.json file as below configuration.
{
"version": "0.2.0",
"configurations": [
{
"type": "node",
"request": "attach",
"name": "Attach",
"port": 5859,
"protocol": "inspector"
}
]
}
And run as "Start Debugging".


Add readiness and liveness probes to Alluxio containers in K8s

Is your feature request related to a problem? Please describe.
In K8s, the Alluxio container processes goes to RUNNING state right after the process is started (entrypoint.sh doesn't end in error).
However, the container being in RUNNING state doesn't mean the container can work. It can be waiting for masters to elect a leader. It can wait for a certain process to start and then fail eventually. We need a readiness probe to help us make sure the container is functioning, before we do anything on it.
Going further on this thread, we probably need a liveness probe to help us identify a container failing. For example, when the masters crash, the workers are still in RUNNING state, although they are obviously not connected to any master and cannot work.
Describe the solution you'd like
Part I: Add readiness probes to all Alluxio master and worker containers
For readiness probes, alluxio fsadmin report command can work on masters and workers.
The command returns successful state when the Alluxio masters are running and the leader is elected. Having this as the readiness probe at least guarantees the basic functionality of the Alluxio container. In addition it doesn't take much resource or block the cluster from working.
Part I can be well satisfied with alluxio fsadmin report
Part II: Add liveness probes to all Alluxio master and worker containers
alluxio fsadmin report on the master side guarantees there is a leader.
alluxio fsadmin report on the worker side guarantees the worker is connected with the master(s).
However, the trick thing is, K8s will try to restart the container if its liveness probe returns an error. If the master(s) connection is lost, there is no point restarting the worker containers. There may be better options for a liveness probe on worker.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Urgency
MEDIUM
Additional context
Add any other context or screenshots about the feature request here.

@madanadit Wanted to discuss with you on Part II regarding the liveness probe.
In a HA environment and 1 master goes down, will alluxio fsadmin report return error only for the dead master container (and makes K8s restart it)?
For an Alluxio worker that lost master connection (masters are down), how do we make the liveness probe still return alive? What do you think is a good probe in this case?

@ZacBlanco how does the monitor script check for readiness?

It depends on the process.
Single master

RPC server ping

Multi-master w/ HA

ssh into each node, pgrep for Alluxio(Job)Master and check a PID exists

(Job)Worker

RPC ping

@jiacheliu3 you should look at bin/alluxio-monitor.sh and check the run_monitor method. For the workers, you can use the same health check: alluxio.worker.AlluxioWorkerMonitor

In HA mode I can also just do bin/alluxio-monitor.sh master instead of masters. This can also work.

It's actually recommended to use an http probe instead of executing a command which spawns up a JVM. This is especially true for liveness probes as they can get probed quite often and ends in error state if it times out (and get the container restarted).
So if there currently isn't a localhost:port/health endpoint, I would propose we add one. Or maybe we can use the rest service to do it.
If we have to use the alluxio-monitor.sh as liveness probe, I would say the interval and allowed timeout should be big.

This has been addressed in https://github.com/Alluxio/alluxio/pull/11080


Add support for env vars when deploying ECS service

Needed this to add the release version (i..e the first 8 characters of the commit hash being deployed) to the task definition when deploying alert matcher or raap.
A bunch of files were not added to this PR even though they were auto-modified by go/dep:

all the mock files since the mock version was upgraded to 1.6.0
go.mod
go.sum
Gopkg.lock
Gopkg.toml

Let me know if I should add any of these changes too.
go version: go version go1.16.3 linux/amd64

Once approved, I'll upload the binary to our google drive and update the confluence doc that links to it. I'll also send out a PSA on Slack


Allow retrieving secrets from shared path

Currently summon-cerberus only allows retrieving secrets from /app/ however there is two default paths Applications and Shared.
This will allow all secrets to be retrieved rather than just the Applications path.

this is fixed in - https://github.com/lookout/summon-cerberus/pull/4


[Question] how to visualize routing distribution ?

Question
can you share the code to visualize routing distribution like Fig. 4 and 5 in paper?
thanks!

i found https://github.com/PKU-YuanGroup/MoE-LLaVA/blob/main/docs/VISUALIZATION.md


No notification with Apple watch

Describe the bug
After jail breaking on 5.3.1 uncover, Apple Watch no longer receives any notifications from apps or messages. Phone calls do work. But it no longer mirrors or pushes alerts to the watch.
If phone is not jail broken, or a reboot happens it restores normal functions of the watch. I e tried airplane mode on only the phone, only the watch, both at the same time, unpairing/ re-pairing (before JB / and after JB).
Had another test it and she had functionality until just yesterday. Prior to, she had normal functions till then. Now it’s stopped, almost as if a service had stopped working. Unsure. Will continue to test.
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Screenshots
Device (please complete the following information):

iOS Version: 13.5
iOS Device: XS Max
unc0ver Version: 5.3.1

Place an "x" between the brackets if true:

[?] this is a bug others will be able to reproduce
[x] this issue is present with all tweaks uninstalled(except for default packages) or disabled
[?] this issue is present after a rootfs restore
[x] this issue is present on the latest version of unc0ver

Logs

put the watch into airplane mode and turn off bluetooth before you jailbreak.  turn back on after jailbroken.

@Hamlock-maneuverr, I’ve tried this method. Also have put the phone in airplane mode only. Have also put both the watch and phone in airplane mode as well. I’ve turned off Bluetooth from the settings app and not the CC.
Also have tried turning off the the watch then back on once jailbroken. And have also unpaired then re-paired once jailbroken.
Edit: Watch OS 6.2.8
Apple Watch Series 2

Did you turn off the "watches" bluetooth?  from inside the watch itself?

@Hamlock-maneuverr yes I have tried this. I recently tried to disable all tweaks that were active as well. Went into iCleaner and disabled them. And tried to toggle them all back on one by one. No luck though.

So if anyone has this same issue I think I’ve found a work around.
#1: if you are not receive notifications properly to your Apple Watch. (ie. text messages or any notification will give your phone an audible alert and banner or notification, but not your watch).
#2: you will need iCleaner pro and uncover jailbreak app for this to work.
#3: when you notice the notifications are no longer working, you must reboot your phone to be able to re-jailbreak. (I test using google voice, sending myself a message and then immediately locking my phone to see if it formed to my watch).
#4: once rebooted, you must disable load tweaks in the uncover app. Then re-jailbreak.
#5: once jailbroken, open iCleaner pro, go to advanced, then to Cydia substrate add one and click all and apply. This should make sure all tweaks or add one are disabled. Then you must run iCleaner pro analyze and clean. Or just clean. Respring afterwards.
#6: after this I send myself a test message. Sometimes this is all you need. If it doesn’t work, reboot the phone again. Send a test message to ensure you have notifications. Re-jailbreak as before leaving tweaks disabled. Once jailbroken you run iCleaner pro analyze and clean again. Then respring. I send a test message here to myself. If all is well, and should be working....
#7: if all of this worked. And you now have working notifications going to your watch, I open up iCleaner pro again, this time back to advanced, then Cydia substrate add one, then click all, then apply. I once again run analyze and clean. Or just clean each time up to you. Once I respring the phone after iCleaner is done, I send yet another test message to confirm it’s working.
With this method. I have successfully made sure my watch continues to work properly with my phone.


Package request: Tickr

Project description
GTK-based highly graphically-customizable Feed Ticker
Metadata

homepage URL: http://www.open-tickr.net/
source URL: http://www.open-tickr.net/src/tickr-0.7.1.tar.gz
license: GPLv3
platforms: linux

Add a :+1: reaction to issues you find important.

gtk2 or something or other


Make #[system_param(ignore)] and #[world_query(ignore)] unnecessary

Objective
When using PhantomData fields with the #[derive(SystemParam)] or #[derive(WorldQuery)] macros, the user is required to add the #[system_param(ignore)] attribute so that the macro knows to treat that field specially. This is undesirable, since it makes the macro more fragile and less consistent.
Solution
Implement SystemParam and WorldQuery for PhantomData. Remove the ignore attributes.
This borrows the approach used in #8012 to prevent generated types from conflicting with user-defined types.

Changelog

Implemented SystemParam and WorldQuery for PhantomData<T>.

Removed the attributes #[system_param(ignore)] and #[world_query(ignore)].

Migration Guide
The attribute #[system_param(ignore)] has been removed. If you were using this with PhantomData fields, simply remove the attribute. If you were using this for another type that implements Default, consider wrapping that type in Local<>.
#[derive(SystemParam)]
struct MyParam<'w, 's, Marker> {
    ...
    // Before:
    #[system_param(ignore)
    _marker: PhantomData<Marker>,

    // After:
    _marker: PhantomData<Marker>,

    // Before:
    #[system_param(ignore)]
    value: MyDefaultType, // This will be initialized using `Default` each time `MyParam` is created.

    // After:
    value: Local<MyDefaultType>, // This will be initialized using `Default` the first time `MyParam` is created.
}

The attribute #[world_query(ignore)] has been removed, as it is now unnecessary.
#[derive(WorldQuery)]
struct MyQuery<Marker> {
    ...
    // Before:
    #[world_query(ignore)
    _marker: PhantomData<Marker>,

    // After:
    _marker: PhantomData<Marker>,
}

This also prevents you from using any other weirder types in here. But I've never seen a case for that, and can't imagine one, so I think that's net good.

Not totally sure that using PhantomData is a 1:1 replacement for world_query(ignore) and system_param(ignore) for all T: Default, though removing that functionality seems like a good idea to me to encourage users to use Local<T> instead.

Agreed, but I'm hopeful that they'll let us know if they were using it for anything else and we can find a less cludgy solution to their needs.

I reverted the collision-avoidance changes. I'll keep that in #8012 for now, so we don't have to sync versions of the same code in different PRs.

I figured out how to do this without removing the ignore attributes, so this PR is now non-breaking. I will remove the attributes in a follow-up PR.


Add PHP 8.0 Support

Due to PHPUnit 8 introducing a required return type declaration of void on test case functions this breaks support for PHP 5 and this PR features the removal of support for PHP 5 and adds support for PHP 8.

Remove support for PHP 5.4
Remove support for PHPUnit 6.0
Added support for PHP 8.0
Added support for PHPUnit 8.0 || 9.0
Updated .gitignore to ignore phpunit.result.cache
Relocated Exception to it's own class
Fixed some phpdoc comments
Updated test to use expectException class

Thanks! Sorry for the delay in merging.

Thanks! Sorry for the delay in merging.


Aspaen Tacuri

Aspaen Tacuri

@sarabaronaaspaen First stage of verification passed, official domain for ASPAEN TACURI is aspaen.edu.co as far as I verified. This is just an informational note. Pull request is still in the review. I appreciate your patience while waiting for the final decision.

@sarabaronaaspaen I am sorry, this domain blocks web access from abroad, thus I cannot verify its eligibility for this repository. Please enroll to GitHub Student Developer Program at https://education.github.com/discount_requests/application. Then apply for JetBrains educational subscription with your GitHub account (GitHub tab at https://www.jetbrains.com/shop/eform/students).
I have to close this request without merging.
Thank you for your understanding!


[BUG] fix bug for schema schange

Proposed changes
This commit has an error:
https://github.com/apache/incubator-doris/pull/6791
when you only change the order of column, error will apear.
Types of changes
What types of changes does your code introduce to Doris?
Put an x in the boxes that apply

[x] Bugfix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation Update (if none of the other choices apply)
[ ] Code refactor (Modify the code structure, format the code, etc...)
[ ] Optimization. Including functional usability improvements and performance improvements.
[ ] Dependency. Such as changes related to third-party components.
[ ] Other.

Checklist
Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

[x] I have created an issue on (Fix #6790) and described the bug/feature there in detail
[ ] Compiling and unit tests pass locally with my changes
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] If these changes need document changes, I have updated the document
[ ] Any dependent changes have been merged

Further comments

sorry，I ignore column change question.

What I would suggest is that it might be easier to just do it this way.
    for (Column alterColumn : alterSchema) {
        if (alterColumn.nameEquals(partitionCol.getName(), true)) {
            // 2.1 partition column cannot be modified
            if (needAlterColumns.contains(alterColumn) && !alterColumn.equals(partitionCol) {
                throw new DdlException("Can not modify partition column["
                        + partitionCol.getName() + "]. index["
                        + olapTable.getIndexNameById(alterIndexId) + "]");
            }
            found = true;
            break;
        }
    }


Create a Cryptocurrency News Portal

Pull Request Details:

Title: Create a Cryptocurrency News Portal
PR Link: #39
Issue: #38

Next Steps:
I'm ready for feedback and any necessary revisions. Please review at your earliest convenience, and feel free to provide comments or suggest improvements.
Thank you for your time and consideration!

Hi User,
Thank you for your pull request. I appreciate your contribution to our project.
I have reviewed your changes and everything looks good. Please be patient, and I'll get back to you as soon as possible.
If you have any questions or need further assistance, don't hesitate to ask.
Best regards,
Jayesh


Pipeline: Dynamically set zookeeper, kafka and hbase server locations in respective taskactor's prestart's

Parent task: #400 Kafka -> HBase pipeline
It might be nice to generalize this capability since it seems to be common especially related to classpath, memory, jvm args

closed by #400


Simplify pretty printing code

[d.bhagat@devdas ~]$ cat /tmp/foo.pl
my @array = qw(first second third fourth);
sub pretty_join {
my $a = shift;
my $string = join (', ', @$a);
$string =~ s/(.*),/$1 &/;
return $string;
}
my $s = pretty_join(@array);
print "$s\n";
[d.bhagat@devdas ~]$ perl /tmp/foo.pl
first, second, third & fourth

This fails when the array is ['first', 'second', 'third', 'fourth,'] = )

Updated, and fixed a bug from the older code as well :)

Now it strips the comma off the key, that isn't acceptable either.
Yes, the old code couldn't handle: [1,1,"1,"] e.g.
sub pretty_join($) {
  my ($a) = @_;
  my $str;
  for my $i (0 .. $#{$a}) {
    $str .= ' & ' if $i == $#{$a};
    $str .= @$a[$i];
    $str .= ', ' if $i < $#{$a}-1;
  }
  return $str;
}

is pretty clear and deals with all edge cases (that I know of).
What do you think?

sub pretty_join {
my @a = @_;
my $str = join(', ', @a[0..$#a-1]);
$str = $str.' & '.$a[$#a];
return $str;
}
perhaps?

use strict;
use warnings;
use Benchmark qw(:all);

sub pretty_join3 {
  my ($a) = @_;
  my $str = join(', ', @{$a}[0..$#{$a}-1]);
  $str = $str.' & '.@{$a}[$#{$a}];
  return $str;
}

sub pretty_join5 {
  my ($a) = @_;
  return join(', ', @{$a}[0..$#{$a}-1]).
  ' & '.@{$a}[$#{$a}];
}

sub pretty_join6 {
  my @a = @{$_[0]};
  return join(', ', @a[0..$#a-1]).
  ' & '.$a[$#a];
}

my @array = qw(first second third fourth);
sub main {
  timethese(3_000_000, {
    'pretty_join3' => sub { pretty_join3(\@array); },
    'pretty_join5' => sub { pretty_join5(\@array); },
    'pretty_join6' => sub { pretty_join6(\@array); },
  }
);
}
main() unless caller(0);

Benchmark: timing 3000000 iterations of pretty_join3, pretty_join5, pretty_join6...
pretty_join3:  4 wallclock secs ( 3.17 usr +  0.00 sys =  3.17 CPU) @ 946372.24/s (n=3000000)
pretty_join5:  4 wallclock secs ( 2.65 usr +  0.00 sys =  2.65 CPU) @ 1132075.47/s (n=3000000)
pretty_join6:  4 wallclock secs ( 3.54 usr +  0.00 sys =  3.54 CPU) @ 847457.63/s (n=3000000)

From this I learned:

Not creating a variable is fastest
Using an array reference instead of creating a new one is faster (even when the array only contains five elements!)

Thanks for the PR!


A Few Things We Found Useful :-)

Includes

authmethod for pre-authorizations
card store
reverse transaction
updating transactions
parenttransaction for reusing stored cards.
error handling
some formatting changes.

Coverage decreased (-5.4%) to 92.308% when pulling a2b85f6ce3731a80a9f692589b9c20866ae8fd81 on sharpspring:master into 81ae926e3e6d477542324a8966a6ef96eb5624ed on gajus:master.

Looks good, however, please revert the code style changes, as those are irrelevant to this PR.

Coverage decreased (-5.4%) to 92.308% when pulling 1103c256b55acbe810481d4c8fc619e1937c367d on sharpspring:master into 81ae926e3e6d477542324a8966a6ef96eb5624ed on gajus:master.


[Kagi FastGPT] All searches result in "Something went wrong"

Extension
https://www.raycast.com/http.james/kagi-fastgpt
Description
I cannot successfully query FastGPT with this extension, any query results in a "something went wrong." Additionally, I use Lulu. When that is turned on, I get a 502 Bad Gateway and I do not get as far as seeing the "something went wrong" screen. If that is unrelated, I'll figure that on my own, but thought I'd share just in case it's related.
Error:
TypeError: Cannot read properties of undefined (reading 'output')

ask.tsx:30:39

---
27:       } finally {
28:         setLoading(false)

### Steps To Reproduce

1. Install extension
2. Configure extension with Kagi API key
3. Perform query with Kagi FastGPT, does not seem to change based on what the query is.

### Current Behaviour

Right now, the extension installs correctly and accepts a query, but it is not able to execute that query. I do have an active account and can verify that the API key works with the other Kagi Search extension that's available on the Raycast store.

### Expected Behaviour

The extension should query Kagi FastGPT.

Did you add Api credits to your account?

@httpjamesm Aha I did not! My apologies! I just added credits and it works like a charm. Thank you, I'll close this now.


Add guideline for optimistic locking in REST APIs

After a discussion in the guild chat participants suggested to add a best practice section for optimistic locking to the guidelines. As good starting point the following internal documentation about different options was suggested:

Option 1: Last-Modified / If-Unmodified-Since
Option 2: Hardcore REST with only URLs in list requests
Option 3: ETags in result entities of list requests
Option 4: Version numbers in result entities of list requests

As we already discussed this topic internally at least twice and find more teams being interested in existing approaches I would definitely support the creation of a corresponding section in the guidelines.

From the meeting

new appendix "best practices"
links from relevant sections
list all options apart from two, with pro/con.
help wanted.

I would be happy to help. Will approach @ePaul offline :)

[ ] Add new appendix "best practices"
[ ] List all options with pro/con.
[ ] Add links from relevant sections
[ ] Extend changelog

Opened a pull request: #330 Please could someone have a look?


Use example documentation IP address ranges

Currently we have general IP addresses:
      "ipam": {
        "type": "whereabouts",
        "range": "2001::0/116",
        "gateway": "2001::f:1"
      }

These should be updated to use the official example ranges.

Please use this prefix for IPv6 examples
2001:db8::/32
and use one of these prefixes for IPv4 examples
<IP_ADDRESS>/24
<IP_ADDRESS>/24
<IP_ADDRESS>/24
These are reserved blocks expressly for the purpose of documentation (and hence avoiding clashes with any user assigned addresses).

Did anyone test this example on a pod to make sure the pod can actually reach the gateway?
  "ipam": {
    "type": "whereabouts",
    "range": "2001::0/116",
    "gateway": "2001::f:1"
  }

The gateway is outside of the configured network range.  Shouldn't there be a requirement for the gateway to be within the assigned range?


Provide additional logging

Background
At the moment, Ketcher has a lot of try/catch blocks which catch errors (for example, when parsing files) and show appearing errors in modal. This errors sometimes neither informative for end-user nor developers.
Solution

It is required to add console.error to catch block in order to easy find the places, where error appeared
It is required to prevent non-informative errors to be shown to users (Example error: "cannot read property null of undefined")

Not tested by QA.


[Android] exit app using exitApp() backhandler error "ExceptionsManager.js:63 this.notificationListener is not a function"

when i use fcm and want to close app using exitApp() backhandler, then when app close, will get app stopped, and show error in debugger like this "ExceptionsManager.js:63 this.notificationListener is not a function"

my guess:
you have this.notificationListener(); but it should bethis.notificationListener.remove();

thankssss it's works... before i think if i do notificationListener.remove() then all notification cannot run properly, but it's still works. thanks guru... :)


Fix operation output validation

We need operation output validation for development, but not for production. The flag for turn off/on is needed


Added Support for Roblox built-in ProximityDetectors

*Note: Couldn't figure out how to update ClientAPI, assuming it's not in repo, placed text file at root. This needs to be updated before using ProximityDetectors.

I'm back on Roblox for a bit and working on v1.2.0; you should see this change in the next update shortly!


Enabling Protocol TLSv1.2 does't work for devices running on

Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:

.
..
...
....

Example application to reproduce the issue
A link to a github project/gist to reproduce the issue.
Expected behavior
A clear and concise description of what you expected to happen.
Debug log
The the debug log (set the log level to TRACE) to help explain your problem.
Environment(please complete the following information):

Version used:
Java version:
Operating System and version:
Endpoint Name and version:
Link to your project:

Additional context
Add any other context about the problem here.

Please add all the required information about your issue.