Difference between this and using readability

Hi!
Great library, but I am trying to figure out if we should swap from using readability on which this builds.
Apart from manually checking the quality for our corpus there is no easy way for me to compare performance.
Would you be able to explain the differences between the actual text extraction approaches?
Thanks in advance!
D

Hi, thanks for your interest.
According to my benchmark and this external one trafilatura should work better than readability-lxml on average. This can be explained by the way the library is designed:

It grounds on a cascade of extraction methods: first a series of heuristics for main text discovery and conversion, then two external libraries, then a baseline extraction if nothing worked (basically all text tags such as <p>).
Since it defaults to readability-lxml if extraction fails or appears to have failed, you'll still benefit from the efficiency of this (very good) library, justext is also used as a fallback. In theory (and the benchmarks confirm it) trafilatura has a better website coverage without sacrificing precision. Its series of heuristics and readability-lxml are mostly used, the rest only applies to rare cases.
If you're also interested in extracting comments, this library can do it while readability cannot.
It performs a conversion and can output "homogeneous" (clean) text and tags in XML format, which readability-lxml doesn't, sometimes leaving you with "exotic" HTML tags within the content.

I hope it helps!

@adbar Thank you very much for your reply!
I tried a comparison today, and it seems that the libs differ quite a bit on their extraction, but it is not clear to me when and why.
Some information I think I have is that:

trafilatura seems to pick up footers more often
fallback does not seem to work as I can get no text from your lib, but text from readability

I am not sure if you will find this useful, but I am providing you some URLs on which the two libs differ the most, the following URLs are where your lib extracts more text:
['https://www.awesomeinventions.com/my-house-not-my-cat-photos',
 'https://youhadmeatgardening.com/lemon-tree-from-seed',
 'https://www.dailymail.co.uk/money/markets/article-8805521/BUSINESS-LIVE-Stimulus-hopes-boost-markets.html',
 'https://www.historychronicle.com/view/90s-vintage-pics-hc/',
 'https://pawszilla.com/pop-culture/celebrities/fans-expressed-fears-celine-dions-appearance-savagely-responded/28/',
 'https://www.calcalist.co.il/internet/articles/0,7340,L-3835935,00.html',
 'https://www.onthemarket.com/farms-land/property/cornwall/',
 'https://www.theguardian.com/world/live/2020/oct/08/coronavirus-live-news-brazil-cases-pass-5m-trump-calls-catching-covid-a-blessing-in-disguise',
 'https://www.walesonline.co.uk/news/wales-news/coronavirus-live-updates-lockdown-gething-19049119',
 'https://www.nottinghampost.com/news/local-news/live-nottinghamshire-lockdown-tighter-restrictions-4585870',
 'https://www.graduatez.com/view/scientists-are-in-awe-after-finding-a-bewildering-creature-beached-on-the-californian-shore/',
 'https://www.loksatta.com',
 'https://www.lefigaro.fr',
 'https://www.theyeshivaworld.com',
 'https://www.liverpoolecho.co.uk/sport/football/football-news/everton-transfer-news-live-rodriguez-19087404',
 'https://mydailymagazine.com/post-a1e720df/',
 'https://www.historychronicle.com/view/history-discovery-secrets-hc/',
 'https://www.manchestereveningnews.co.uk/sport/football/football-news/leeds-vs-man-city-live-19043874',
 'https://www.dailymail.co.uk/travel/escape/article-8814663/The-worlds-best-hotels-islands-cities-ski-resorts-revealed-Conde-Nast-Traveller.html',
 'https://www.business-live.co.uk/economic-development/boris-johnson-unveil-lockdown-restrictions-19088918',
 'https://www.dailyrecord.co.uk/news/politics/nicola-sturgeon-coronavirus-update-live-22799031',
 'https://musiccritic.com/equipment/guitars/best-acoustic-electric-guitar',
 'https://www.dailyrecord.co.uk/sport/football/football-transfer-news/transfer-deadline-live-rangers-celtic-22792735',
 'https://www.gumtree.com/dogs/scotland/page6',
 'https://www.moneycontrol.com',
 'https://www.getreading.co.uk/news/reading-berkshire-news/live-homes-west-reading-suffer-19062352',
 'https://www.hulldailymail.co.uk/news/hull-east-yorkshire-news/3-tier-lockdown-live-updates-4597247',
 'https://www.nature.com/articles/ncomms4887',
 'https://www.dailyrecord.co.uk/sport/football/football-news/europa-league-draw-live-celtic-22779308',
 'https://www.gumtree.com/dogs/uk/dogs+northern+ireland/page2',
 'https://alphamom.com',
 'https://www.domesticatedcompanion.com/actors-havent-done-anything-in-years/2/',
 'https://www.liverpoolecho.co.uk/sport/football/football-news/liverpool-transfer-news-rumours-butland-19055637',
 'https://www.insider.com/hard-brainteasers-riddles-for-smart-people-2017-9',
 'https://www.catster.com/lifestyle/bringing-an-adult-cat-into-your-home',
 'https://www.express.co.uk/life-style/science-technology/1340039/iPhone-12-Release-Date-October-13-2020-Apple-Fans-Not-Long-Left-To-Wait-UK/amp',
 'https://www.pinkvilla.com/entertainment/news/saif-ali-khan-talks-about-sara-ali-khan-and-how-if-he-s-hurt-her-taimur-can-t-make-him-feel-better-566895',
 'https://reviews.mtbr.com',
 'https://www.dailymail.co.uk/news/article-8803493/Trump-administrations-fear-leaks-leaves-White-House-staff-dark.html',
 'https://kit.honestjohn.co.uk',
 'https://www.trivia.com/view/a-letter-from-this-womans-biological-mother-changed-her-life-forever/',
 'https://www.gumtree.com/dogs/uk/puppies/page12',
 'https://www.essexlive.news/news/essex-news/live-coronavirus-updates-essex-awaits-4604321',
 'https://www.dailymail.co.uk/sport/football/article-8797665/Europa-League-group-stage-draw-2020-21-LIVE-Result.html',
 'https://www.lefigaro.fr/politique',
 'https://www.chocolate.com/view/life-of-steven-mcqueen-cho/',
 'https://www.gumtree.com/dogs/uk/page2',
 'https://www.examinerlive.co.uk/sport/football/transfer-news/huddersfield-town-transfer-news-live-19112945',
 'https://www.walesonline.co.uk/sport/football/transfer-news/cardiff-city-transfer-news-live-19112387',
 'https://www.gumtree.com/cars/uk/audi/a6/page2',
 'https://www.thespruce.com/ingenious-ikea-billy-bookcase-hacks-4006865',
 'https://iamafoodblog.com/birria-tacos-recipe',
 'https://bestofmachinery.com/best-hearing-protection-for-lawn-mowing',
 'https://www.wallstreetmojo.com/preferred-shares',
 'https://parentinfluence.com',
 'https://www.grimsbytelegraph.co.uk/news/uk-world-news/live-boris-johnson-press-conference-4561866',
 'https://www.autoscout24.de/lst/',
 'https://www.msn.com/en-za',
 'https://www.lefigaro.fr/societes',
 'https://www.devonlive.com/news/devon-news/planning-applications-submitted-devon-week-4506950',
 'https://www.oversixty.com.au',
 'https://www.dailypost.co.uk/news/north-wales-news/coronavirus-live-press-conference-take-19099922.amp',
 'https://www.trivia.com/view/cool-google-earth-finds-trv/',
 'https://www.pinkvilla.com/tv/news-gossip/bigg-boss-6-s-sana-khan-bids-goodbye-showbiz-industry-forever-decides-spend-life-serving-humanity-567296',
 'https://www.dailymail.co.uk/sport/football/article-8772027/Crystal-Palace-vs-Everton-Premier-League-Live-result.html',
 'https://www.hulldailymail.co.uk/sport/rugby-league/rugby-league-live-transfer-news-4560871',
 'https://www.manchestereveningnews.co.uk/sport/football/transfer-news/man-utd-transfer-news-recap-19042171',
 'https://www.yardbarker.com/entertainment/gallery/the_most_talked_about_movie_moments_of_the_2010s/s1__30411269',
 'https://www.historychronicle.com/view/thomas-jeffersons-hidden-rooms-hc/',
 'https://www.gumtree.com/property-to-rent/plymouth/page2',
 'https://www.dailymail.co.uk/sport/football/article-8803121/Transfer-News-LIVE-Latest-Premier-League-European-signings-deals-rumours.html',
 'https://www.hulldailymail.co.uk/news/uk-world-news/live-coronavirus-uk-lockdown-rules-4579067',
 'https://www.buzzfeed.com/quizzes',
 'https://www.liverpoolecho.co.uk/news/liverpool-news/live-coronavirus-boris-vaccine-tests-19049134',
 'https://www.liverpoolecho.co.uk/sport/football/football-news/everton-news-transfer-rumours-godfrey-19025712',
 'https://www.dailyrecord.co.uk/sport/football/football-transfer-news/transfer-news-live-celtic-rangers-22765511',
 'https://www.mabelandmoxie.com/amp/These+Are+The+Smartest+Dog+Breeds+In+The+World,+Ranked',
 'https://www.pinkvilla.com/entertainment/news/kangana-ranaut-reacts-karan-srk-other-productions-uniting-against-media-houses-file-case-me-also-567902',
 'https://www.vielfliegertreff.de',
 'https://www.gumtree.com/cars/uk/left+hand+drive/page7',
 'https://www.gumtree.com/cats/uk/kittens/page2',
 'https://www.chroniclelive.co.uk/news/north-east-news/uk-deaths-coronavirus-live-updates-18935120',
 'https://www.manchestereveningnews.co.uk/news/greater-manchester-news/live-mayor-andy-burnham-holds-19023674',
 'https://www.telegraph.co.uk/news/2020/10/07/travel-latest-covid-passport-greece-italy-turkey-quarantine',
 'https://historicalpost.com/anthropology-and-history/places/niagara-falls-runs-dry-reveals-bodies-1969-united-states-canada/12/',
 'https://www.pinkvilla.com/entertainment/news/shabana-azmi-opens-unrest-bollywood-and-kangana-ranauts-outrageous-remarks-566625',
 'https://www.simbaly.com/view/the-jackson-kids-now-sim/',
 'https://www.dailymail.co.uk/sport/football/article-8784211/Transfer-news-LIVE-Latest-Premier-League-European-club-signings-deals-rumours.html',
 'https://www.gumtree.com/dogs/uk/french+bulldog/page3',
 'https://powerlisting.fandom.com/wiki/List_of_Supernatural_Powers_and_Abilities',
 'https://www.standard.co.uk/news/uk/uk-coronavirus-live-latest-updates-boris-johnson-threetier-lockdown-a4568406.html',
 'https://m.khaleejtimes.com',
 'https://divinityoriginalsin2.wiki.fextralife.com',
 'https://forums.civfanatics.com',
 'https://www.gumtree.com/dogs/uk/page6',
 'https://www.insider.com/what-are-healing-crystals-how-to-use-them-2018-7',
 'https://www.msn.com/fr-fr',
 'https://www.gumtree.com/cars/northern-ireland/page2',
 'https://www.birminghammail.co.uk/sport/football/transfer-news/wolves-deadline-day-transfers-live-19037027',
 'https://www.leeds-live.co.uk/news/uk-world-news/live-coronavirus-sheffield-leeds-rates-19042623',
 'https://itsfoss.com/best-linux-desktop-environments',
 'https://www.historychronicle.com/view/cleaning-lady-shock-prank/',
 'https://flawlessfood.co.uk/olive-herb-focaccia-bread',
 'https://www.luxandlush.com/view/woman-sells-a-house-nobody-expected-to-find-this-inside-lux/',
 'https://www.liverpoolecho.co.uk/sport/football/football-news/live-liverpool-transfer-news-rumours-19074803',
 'https://www.everydayhealth.com/healthy-living/fascinating-facts-about-body-temperature.aspx',
 'https://rus.delfi.lv',
 'https://www.birminghammail.co.uk/sport/football/transfer-news/villa-live-king-transfer-latest-19062357',
 'https://hu.motorsport.com',
 'https://www.foxyfolksy.com/chocolate-buttercream-frosting-without-powdered-sugar-ermine-icing',
 'https://toppoptoday.com',
 'https://bap.navigator.web.de',
 'https://trends.bibamagazine.fr/en/prince-william-and-harry-technically-have-a-sister-and-they-completely-ignore-her-2/20',
 'https://www.telegraph.co.uk/business/2020/10/06/joe-bidens-war-economy-policies-radical-break-status-quo',
 'https://www.gumtree.com/dogs/scotland/page4',
 'https://www.birminghammail.co.uk/sport/football/transfer-news/aston-villa-transfers-rashica-benrahma-19037412',
 'https://www.standard.co.uk/news/uk/coronavirus-live-latest-updates-pubs-close-a4566031.html',
 'https://www.gumtree.com/cars/uk/page5',
 'https://www.euronews.com',
 'https://www.graduatez.com/view/surprising-findings-pipes/',
 'https://www.msn.com/de-ch/',
 'https://pawszilla.com/pop-culture/celebrities/fans-expressed-fears-celine-dions-appearance-savagely-responded/65/',
 'https://www.liverpoolecho.co.uk/sport/football/football-news/live-everton-transfer-news-bernard-19075370',
 'https://www.historychronicle.com/view/ozzy-osbourne-facts-hc/',
 'https://zenherald.com/pop-culture/celebrities/fans-expressed-fears-celine-dions-appearance-savagely-responded/30/',
 'https://www.manchestereveningnews.co.uk/news/uk-news/pmqs-live-prime-minister-boris-19101281',
 'https://www.mirror.co.uk/sport/football/transfer-news/transfer-news-live-smalling-dele-22721256',
 'https://sportowefakty.wp.pl/zuzel/relacja/110331/fogo-unia-leszno-rm-solar-falubaz-zielona-gora',
 'https://www.gumtree.com/cars/scotland/page3',
 'https://guides.gamepressure.com',
 'https://www.msn.com/en-gb',
 'https://www.chroniclelive.co.uk/news/north-east-news/coronavirus-restrictions-uk-live-updates-19026811',
 'https://hipertextual.com',
 'https://www.luxandlush.com/view/celeb-kids-are-not-little-anymore-fb-lux/',
 'https://www.historychronicle.com/view/70s-80s-celebs-kids-fb-hc/',
 'https://m.sportowefakty.wp.pl/pilka-nozna/relacja/106622/polska-finlandia',
 'https://www.gumtree.com/cars/uk/range+rover+sport/page2',
 'https://www.stokesentinel.co.uk/news/stoke-on-trent-news/live-covid-19-updates-across-4588547',
 'https://www.bristolpost.co.uk/sport/football/football-news/bristol-rovers-transfer-questions-garner-4555153',
 'https://www.visitlancashire.com/things-to-do/animal-and-farm-attractions',
 'https://www.mirror.co.uk/sport/football/transfer-news/transfer-news-live-arsenal-sancho-22752560',
 'https://www.lesechos.fr',
 'https://www.gumtree.com/for-sale/uk/page2',
 'https://www.bristolpost.co.uk/news/health/coronavirus-cases-lockdown-bristol-live-4597246',
 'https://www.simbaly.com/view/celeb-daughters-grown-up-co-sim/',
 'https://www.buzzfeed.com/alliehayes/weird-house-rules-parents-reddit',
 'https://www.gumtree.com/dogs/northern-ireland/page4',
 'https://www.autosport.com',
 'https://www.liverpoolecho.co.uk/sport/football/football-news/everton-news-transfer-rumours-live-19025712',
 'https://www.rtl.be',
 'https://www.standard.co.uk/news/uk/coronavirus-live-latest-updates-lockdown-vaccine-a4567091.html',
 'https://www.gazettelive.co.uk/news/teesside-news/coronavirus-live-boris-johnson-set-19088101',
 'https://cookieandkate.com/perfect-quinoa',
 'https://www.chroniclelive.co.uk/sport/football/football-news/newcastle-transfer-news-live-mourinho-19007685',
 'https://www.gumtree.com/cars/uk/land-rover/defender/page3',
 'https://www.lakesideloops.com/watson-cardigan',
 'https://www.chocolate.com/view/woman-sees-a-car-stop-for-pregnant-beggar-this-is-what-she-finds-when-she-follows/',
 'https://www.chroniclelive.co.uk/news/north-east-news/boris-johnson-announcement-live-updates-19090211',
 'https://www.sports.fr/direct-foot/50918/177512/lorient-olympique-lyonnais.html',
 'https://www.pinkvilla.com/entertainment/news/sushant-singh-rajputs-flatmate-siddharth-pithani-states-late-actor-did-not-meet-rhea-chakraborty-june-13-566132',
 'https://www.cornwalllive.com/news/cornwall-news/live-storm-alex-weather-traffic-4567887',
 'https://www.yourrulingplanet.com/view/instagram-golfer-paige/',
 'https://www.dublinlive.ie/news/health/level-5-lockdown-nphet-dublin-19053662',
 'https://www.thelist.com/214894/when-you-take-a-multivitamin-every-day-this-is-what-happens-to-your-body',
 'https://beta.southglos.gov.uk/new-south-gloucestershire-local-plan-2018-2036',
 'https://www.latest-hairstyles.com/mens/thin.html',
 'https://www.healthline.com/nutrition/ways-to-measure-body-fat',
 'https://www.themakeupdummy.com',
 'https://www.audi-sport.net/xf',
 'https://www.cheshire-live.co.uk/news/chester-cheshire-news/live-cheshire-coronavirus-national-lockdown-19049856.amp',
 'https://www.chroniclelive.co.uk/news/north-east-news/north-east-news-live-latest-19112705',
 'https://www.edinburghlive.co.uk/news/edinburgh-news/coronavirus-scotland-live-rules-latest-19046171',
 'https://www.ccna7.com/ccna2-v6-0/ccna2-v6-0-chapter-2-exam-answer-2017',
 'https://www.autocar.co.uk/slideshow/biggest-flops-automotive-history-0',
 'https://www.telegraph.co.uk/news/0/matt-favourite-covid-19-cartoons-think-laughing-something-makes',
 'https://www.pinkvilla.com/tv/news-gossip/exclusive-here-s-why-salman-khan-s-bigg-boss-14-will-air-half-hour-there-good-news-565189',
 'https://www.manchestereveningnews.co.uk/news/uk-news/live-boris-johnson-leads-downing-19091296',
 'https://www.gumtree.com/caravans/uk/page3',
 'https://www.cheshire-live.co.uk/news/chester-cheshire-news/live-cheshire-lockdown-coronavirus-council-19021297',
 'https://happyyouhappyfamily.com',
 'https://thepostpartumparty.com/how-to-set-up-a-baby-nursery-in-a-small-space',
 'https://www.realsimple.com/home-organizing/gardening/outdoor/hydrangea-care',
 'https://www.devonlive.com/news/local-news/live-m5-fire-bristol-avonmouth-4575486',
 'https://www.historychronicle.com/view/priest-sues-millions-hc/',
 'https://www.cadena100.es',
 'https://www.minq.com/lifestyle/2553886/students-recall-the-absolute-dumbest-thing-theyve-ever-heard-a-teacher-say/',
 'https://www.levante-emv.com',
 'https://www.authenticfoodquest.com/popular-portuguese-dishes',
 'https://www.instyle.com/beauty/wigs-for-black-women-cancer',
 'https://www.theprimarymarket.com/view/pizza-delivery-employee-tpm/',
 'http://www.msn.com/en-gb/',
 'https://www.dailymail.co.uk/sport/football/article-8772175/Sheffield-United-vs-Leeds-Premier-League-Live-result.html',
 'https://www.minq.com/lifestyle/2495316/students-share-the-rudest-thing-a-teacher-ever-said-to-them/',
 'https://www.nottinghampost.com/sport/football/football-news/nottingham-forest-transfer-news-live-4597836',
 'https://www.msn.com/es-es',
 'https://www.liverpoolecho.co.uk/sport/football/football-news/liverpool-transfer-news-rumours-kabak-19042594',
 'https://wiadomosci.gazeta.pl',
 'https://www.nationalrail.co.uk/service_disruptions/245738.aspx',
 'https://www.domesticatedcompanion.com/actors-havent-done-anything-in-years/',
 'https://www.caughtoffside.com/2020/09/27/agreement-reached-man-united-look-set-to-sign-defender-as-they-reach-an-agreement-on-the-final-details/',
 'https://www.corkbeo.ie/news/local-news/level-3-live-cork-ireland-19056312',
 'https://www.historychronicle.com/view/single-celebs-hc/',
 'https://www.historychronicle.com/view/donny-osmond-confession-hc/',
 'https://www.somersetlive.co.uk/news/somerset-news/live-storm-alex-weather-traffic-4568051',
 'https://www.theprimarymarket.com/view/elderly-house-sell/',
 'https://www.luxandlush.com/view/behind-the-scenes-with-barbi-benton/',
 'https://magellantimes.com/pop-culture/celebrities/fans-expressed-fears-celine-dions-appearance-savagely-responded/37/',
 'https://www.nerfplz.com',
 'https://www.disneytouristblog.com/disney-world-news-star-wars-permit-frozen-ride-refurbishment-restaurants-returning',
 'https://trendscatchers.co.uk/index.php/en/2020/07/09/prince-harry-claims-that-meghan-markle-might-not-be-the-one-2/31',
 'https://www.ign.com/articles/the-best-ssd-for-gaming',
 'https://www.birminghammail.co.uk/sport/football/transfer-news/villa-live-deadline-day-king-19062357',
 'https://www.telegraph.co.uk/opinion/2020/10/04/lettersgps-have-worked-crisis-often-personal-cost',
 'https://www.liverpoolecho.co.uk/sport/football/football-news/live-everton-transfer-news-rumours-19061794',
 'https://www.liverpoolecho.co.uk/sport/football/football-news/liverpool-transfer-news-rumours-live-19025709',
 'https://cookieandkate.com/mujaddara-recipe',
 'https://trendscatchers.co.uk/index.php/en/2020/07/09/prince-harry-claims-that-meghan-markle-might-not-be-the-one-2/30',
 'https://www.theguardian.com/society',
 'https://www.nature.com/articles/s41467-018-04619-5',
 'https://www.birminghammail.co.uk/sport/football/transfer-news/transfer-news-west-brom-live-19036796',
 'https://www.divein.com',
 'https://www.historychronicle.com/view/real-noahs-ark-hc/',
 'https://www.preloved.co.uk/classifieds/furniture-fittings/kitchen-furniture/all/uk/used+kitchen+cabinets+sale',
 'http://wiadomosci.gazeta.pl',
 'https://www.dailyrecord.co.uk/sport/football/football-news/celtic-vs-rangers-live-score-22860327',
 'https://www.domesticatedcompanion.com/each-state-perfectly-portrayed-in-one-photograph/2/',
 'https://www.gumtree.com/dogs/uk/northern+ireland+pets/page2',
 'https://www.hertfordshiremercury.co.uk/news/hertfordshire-news/live-boris-johnson-speech-updates-4597736',
 'https://www.gazettelive.co.uk/sport/football/transfer-news/middlesbrough-transfers-yannick-bolasie-striker-19111028',
 'https://www.cambridge-news.co.uk/news/uk-world-news/live-updates-boris-johnson-tierlockdown-19117181',
 'http://www.msn.com/',
 'https://www.kentlive.news/news/kent-news/live-updates-covid-infection-rates-4597748',
 'https://www.goodhousekeeping.com/holidays/gift-ideas/g1266/handmade-gifts',
 'https://www.walesonline.co.uk/news/wales-news/wales-breaking-news-traffic-weather-19075607',
 'https://www.visitwiltshire.co.uk/accommodation/self-catering',
 'https://www.historychronicle.com/view/bing-crosby-life-hc/',
 'https://www.liverpoolecho.co.uk/news/liverpool-news/live-uk-coronavirus-updates-merseyside-19002010',
 'https://crinacle.com/2020/02/19/the-audiophiles-perspective-samsung-galaxy-buds',
 'https://www.mirror.co.uk/sport/football/transfer-news/arsenal-transfer-news-live-partey-22790130',
 'https://www.chocolate.com/view/70s-80s-celebs-kids-fb-cho/',
 'https://www.domesticatedcompanion.com/actors-havent-done-anything-in-years/11/',
 'https://www.gumtree.com/dogs/uk/jack+russell/page4',
 'https://www.haaretz.com/israel-news/business/.premium-this-year-israeli-energy-finally-went-green-but-the-revolution-is-just-beginning-1.9214282',
 'https://natashaskitchen.com/chicken-stir-fry-recipe',
 'https://www.hulldailymail.co.uk/news/uk-world-news/coronavirus-live-cases-lockdown-updates-4573922',
 'https://www.primandprep.com',
 'https://www.organizationobsessed.com/20-amazing-organization-hacks-will-transform-bedroom',
 'https://www.bakewithpaws.com',
 'https://www.oprahmag.com/entertainment/g28335655/best-halloween-books',
 'https://www.trivia.com/view/61-of-the-best-red-carpet-mishaps/',
 'https://www.examinerlive.co.uk/news/west-yorkshire-news/live-coronavirus-sheffield-leeds-kirklees-19056079',
 'https://www.telegraph.co.uk/news/2020/10/10/return-students-caused-inevitable-spike-covid-cases',
 'https://www.leeds-live.co.uk/sport/leeds-united/leeds-united-u23s-live-highlights-19036388']

This batch is where readability extracts more text:
['https://redtri.com/best-jokes-for-kids/slide/1',
 'https://www.msn.com/en-gb/news/uknews/uk-university-student-halls-too-full-to-be-safe-experts-warn/ar-BB19DUqK',
 'https://www.babysleepsite.com/schedules/toddler-schedule/',
 'https://www.msn.com/en-gb/foodanddrink/other/slow-cooker-turkey-chilli-is-healthier-but-just-as-comforting/ar-BB19KDYg',
 'https://www.dailymail.co.uk/sport/teampages/leicester.html',
 'https://www.msn.com/en-gb/lifestyle/travel/can-you-guess-these-capital-cities-from-up-high/ss-BB19DxUe',
 'https://www.independent.co.uk/news/uk/politics/coronavirus-false-reporting-contact-fine-penalty-b671230.html',
 'https://www.msn.com/en-gb/lifestyle/style/prince-louis-adorable-striped-knit-is-a-10-john-lewis-bargain/ar-BB19HUbk',
 'https://www.dailymail.co.uk/tvshowbiz/love-island/index.html',
 'https://www.discogs.com/sell/list',
 'https://www.msn.com/en-gb/travel/tripideas/these-are-the-world-s-most-beautiful-national-parks/ss-BB19FnGx',
 'https://www.msn.com/en-gb/travel/tripideas/this-enchanted-fairytale-treehouse-in-kent-is-the-perfect-autumn-staycation/ar-BB19I6Wz',
 'https://www.msn.com/en-gb/lifestyle/style/super-organised-star-closets-we-d-love-to-copy/ss-BB19IOde',
 'https://www.futbin.com/21/player/25735/leroy-sane',
 'https://www.msn.com/en-gb/news/techandscience/scientists-discover-24-planets-with-conditions-even-more-suitable-for-life-than-earth/ar-BB19HZlb',
 'https://www.mirror.co.uk/news/uk-news/breaking-englands-worst-schools-revealed-13899939',
 'https://relieved.co/brilliant-kitchen-tips-cooking-easier/',
 'https://www.standard.co.uk/news/uk/three-tier-lockdown-system-coronavirus-london-medium-risk-a4570041.html',
 'https://www.lifewire.com/screen-mirroring-lg-smart-tvs-4770959',
 'https://minecraft-server-list.com',
 'https://www.standard.co.uk/news/uk/boris-johnson-coronavirus-nation-briefing-100-a4559471.html',
 'https://www.fifplay.com/fifa-21-game-settings',
 'https://www.independent.co.uk/news/uk/politics/brexit-boris-johnson-michael-gove-northern-ireland-border-b669752.html',
 'https://www.theguardian.com/uk-news/2020/oct/09/rishi-sunak-expands-wage-subsidies-to-head-off-winter-surge-in-job-losses',
 'https://www.futbin.com/21/player/25374/thomas-partey',
 'https://www.standard.co.uk/news/uk/turkey-poland-caribbean-islands-added-to-uk-quarantine-list-a4561256.html',
 'https://www.dailymail.co.uk/news/article-8747257/Is-England-heading-HALF-TERM-lockdown.html',
 'https://www.independent.co.uk/news/uk/politics/brexit-no-deal-summit-boris-johnson-eu-fishing-state-aid-von-der-leyen-michel-b1041344.html',
 'https://www.msn.com/en-gb/lifestyle/style/kate-middleton-breaks-the-internet-with-her-35-linen-shirt/ar-BB19AQ57',
 'https://ext.theperspective.com/items-we-never-knew-we-wanted/1/',
 'https://www.futbin.com/21/player/681/ferland-mendy',
 'https://www.popsugar.co.uk/smart-living/Cheap-Homemade-Halloween-Costumes-42432483/amp',
 'https://www.msn.com/en-gb/money/homeandproperty/celebrity-real-estate-in-october-chris-hemsworth-demi-lovato-and-more/ss-BB19BHew',
 'https://spotlightstories.co/50-hilarious-exam-answers-students-given/',
 'https://www.futbin.com/21/player/612/leroy-san%C3%A9',
 'https://www.tomshardware.com/uk/reviews/best-gpus,4380.html',
 'https://harrypotter.fandom.com/wiki/Hufflepuff',
 'https://docplayer.net',
 'https://www.theguardian.com/us-news/2020/jun/23/millions-of-americans-cant-afford-water-bills-rise',
 'https://www.msn.com/en-gb/news/uknews/hospitality-bailout-scheme-branded-a-con-as-firms-struggle-to-get-1-500-grant/ar-BB19HTbc',
 'https://deardesigner.co.uk',
 'https://www.crowdyfan.com/worldwide/wedding-gown/42',
 'https://homehacks.co/50-useful-beauty-hacks/',
 'https://www.theguardian.com/football/live/2020/sep/27/manchester-city-v-leicester-city-premier-league-live',
 'https://www.futbin.com/21/player/573/fabinho',
 'https://www.crowdyfan.com/worldwide/wedding-gown/13',
 'https://www.crowdyfan.com/worldwide/wedding-gown/23',
 'https://www.futbin.com/21/player/612/leroy-sane',
 'https://www.eadt.co.uk/sport/league-one-transfer-guide-2020-21-1-6881725',
 'https://www.t3.com/features/best-running-shoes',
 'https://www.crowdyfan.com/worldwide/wedding-gown/6',
 'https://www.msn.com/pl-pl/styl-zycia/travel/cudowny-%C5%9Bwiat-zapieraj%C4%85ce-dech-w-piersiach-widoki-z-r%C3%B3%C5%BCnych-miejsc-naszej-planety/ss-AAGnBMi',
 'https://ext.theperspective.com/clever-and-creative-graffiti/4/',
 'https://www.standard.co.uk/news/world/donald-trump-coronavirus-pandemic-comments-timeline-a4561601.html',
 'https://www.msn.com/en-gb/foodanddrink/other/james-martin-s-incredible-outdoor-kitchen-will-blow-your-mind/ar-BB19z8ih',
 'https://www.cornwalllive.com/news/cornwall-news/confirmed-coronavirus-cases-fall-across-4614786',
 'https://www.futbin.com/21/player/570/roberto-firmino',
 'https://www.dailymail.co.uk/sport/teampages/southampton.html',
 'https://www.standard.co.uk/news/uk/lockdown-measures-north-east-hancock-a4558186.html',
 'https://www.independent.co.uk/arts-entertainment/comedy/features/sian-gibson-interview-call-centre-worker-peter-kay-s-car-share-a6811026.html',
 'https://www.futbin.com/21/player/25736/nelson-semedo',
 'https://www.independent.co.uk/news/uk/politics/brexit-no-deal-tactic-boris-johnson-david-lidington-b599750.html',
 'https://www.msn.com/en-gb/lifestyle/other/can-i-leave-a-local-lockdown-to-go-on-holiday/ar-BB19wRic',
 'https://www.msn.com/en-gb/entertainment/celebrity/rita-ora-sparks-engagement-speculation-with-ring-on-her-wedding-finger/ar-BB19HfZE',
 'https://www.msn.com/en-gb/entertainment/movies/joaquin-phoenix-from-tragedy-to-happy-fatherhood/ss-BB19uYnN',
 'https://www.msn.com/en-gb/entertainment/tv/gemma-atkinson-wells-up-live-on-air-after-gorka-marquezs-emotional-surprise/ar-BB19InRs',
 'https://www.dailymail.co.uk/news/boris_johnson/index.html',
 'https://www.msn.com/en-gb/news/royal-family/claim-meghan-allowed-details-of-her-private-life-to-be-fed-to-authors-of-finding-freedom/ar-BB19CCAp',
 'https://www.biblegateway.com/passage/',
 'https://www.theguardian.com/football/live/2020/oct/05/transfer-deadline-day-2020-cavani-partey-sancho-and-more-news-live',
 'https://www.msn.com/en-gb/lifestyle/style/kate-middleton-breaks-the-internet-with-her-%c2%a335-linen-shirt/ar-BB19AQ57',
 'https://www.dailymail.co.uk/news/article-8772683/Chef-Jamie-Oliver-joins-Mail-Sundays-war-toxic-food.html',
 'https://stardewvalleywiki.com/Penny',
 'https://www.plymouthherald.co.uk/news/plymouth-news/confirmed-coronavirus-cases-fall-across-4614786',
 'https://www.msn.com/en-gb/money/other/these-amazing-homes-cost-nothing-to-run/ss-BB19CboF',
 'https://www.hertfordshiremercury.co.uk/news/hertfordshire-news/38-breathtaking-hertfordshire-walks-perfect-4263258',
 'https://ext.theperspective.com/clever-and-creative-graffiti/6/',
 'https://www.msn.com/en-gb/entertainment/celebrity/naga-munchetty-looks-wildly-different-with-long-hair-transformation-%e2%80%93-see-photo/ar-BB19uzj3',
 'https://www.standard.co.uk/news/uk/closing-schools-increase-coronavirus-deaths-edinburgh-study-a4566076.html',
 'https://www.standard.co.uk/news/uk/uk-coronavirus-cases-today-deaths-a4572002.html',
 'https://www.futbin.com/21/player/671/david-alaba',
 'https://www.standard.co.uk/news/uk/emergency-lockdown-london-north-england-socialising-household-mixing-a4557601.html',
 'https://www.futbin.com/21/player/25701/ansu-fati',
 'https://www.msn.com/en-gb/travel/tripideas/25-fictional-places-we-d-love-to-visit/ss-BB19EOWK',
 'https://www.standard.co.uk/news/london/london-coronavirus-cases-latest-figures-tier-two-lockdown-a4568481.html',
 'https://www.msn.com/en-gb/news/world/floods-in-france-italy-swept-bodies-out-of-cemeteries/ar-BB19KCeT',
 'https://www.msn.com/en-gb/news/brexit/with-its-legal-action-over-this-uk-bill-the-eu-is-showing-it-means-business/ar-BB19Dh5D',
 'https://www.dailymail.co.uk/news/new_zealand/index.html',
 'https://www.independent.co.uk/life-style/food-and-drink/coronavirus-restrictions-three-tier-lockdown-boris-johnson-restaurants-bars-pubs-hospitality-b990820.html',
 'https://www.standard.co.uk/showbiz/celebrity-news/nicola-adams-lesbian-gay-strictly-come-dancing-a4542561.html',
 'https://www.msn.com/pl-pl/styl-zycia/podroze/na-zdj%C4%99ciach-majestatyczne-g%C3%B3ry-z-ca%C5%82ego-%C5%9Bwiata/ss-BBYaukt',
 'https://www.futbin.com/squad-building-challenges/ALL/21/Give%20Me%20Five',
 'https://www.op.gg/champion/statistics',
 'https://statisticsglobe.com/change-font-size-of-ggplot2-plot-in-r-axis-text-main-title-legend']

Thank you very much for the lists, I believe they are useful! Could you
please clarify the following points?

Which version of trafilatura are you using, with which parameters? I
assume you use the last readability-lxml version? (0.8.1)
Sometimes less text is better (boilerplate elements hopefully
missing), similarly the fact that there is more text isn't necessary
significant. The presence of footers doesn't seem too good, do you have
a few examples at hand?
Do you have a list of URLs for which trafilatura doesn't return any
text whatsoever (although it probably should)?

To sum up, minor differences in text output are not necessarily a
concern, I'd say that discrepancies over 5-10% are meaningful.
On 10/26/20 6:35 PM, Dennis wrote:

@adbar https://github.com/adbar Thank you very much for your reply!
I tried a comparison today, and it seems that the libs differ quite a
bit on their extraction, but it is not clear to me when and why.
Some information I think I have is that:

|trafilatura| seems to pick up footers more often
fallback does not seem to work as I can get no text from your lib,
but text from readability

I am not sure if you will find this useful, but I am providing you some
URLs on which the two libs differ the most, the following URLs are where
your lib extracts more text:
|['https://www.awesomeinventions.com/my-house-not-my-cat-photos',
'https://youhadmeatgardening.com/lemon-tree-from-seed',
'https://www.dailymail.co.uk/money/markets/article-8805521/BUSINESS-LIVE-Stimulus-hopes-boost-markets.html',
'https://www.historychronicle.com/view/90s-vintage-pics-hc/',
'https://pawszilla.com/pop-culture/celebrities/fans-expressed-fears-celine-dions-appearance-savagely-responded/28/',
'https://www.calcalist.co.il/internet/articles/0,7340,L-3835935,00.html', 'https://www.onthemarket.com/farms-land/property/cornwall/',
'https://www.theguardian.com/world/live/2020/oct/08/coronavirus-live-news-brazil-cases-pass-5m-trump-calls-catching-covid-a-blessing-in-disguise',
'https://www.walesonline.co.uk/news/wales-news/coronavirus-live-updates-lockdown-gething-19049119',
'https://www.nottinghampost.com/news/local-news/live-nottinghamshire-lockdown-tighter-restrictions-4585870',
'https://www.graduatez.com/view/scientists-are-in-awe-after-finding-a-bewildering-creature-beached-on-the-californian-shore/',
'https://www.loksatta.com', 'https://www.lefigaro.fr',
'https://www.theyeshivaworld.com',
'https://www.liverpoolecho.co.uk/sport/football/football-news/everton-transfer-news-live-rodriguez-19087404',
'https://mydailymagazine.com/post-a1e720df/',
'https://www.historychronicle.com/view/history-discovery-secrets-hc/',
'https://www.manchestereveningnews.co.uk/sport/football/football-news/leeds-vs-man-city-live-19043874',
'https://www.dailymail.co.uk/travel/escape/article-8814663/The-worlds-best-hotels-islands-cities-ski-resorts-revealed-Conde-Nast-Traveller.html',
'https://www.business-live.co.uk/economic-development/boris-johnson-unveil-lockdown-restrictions-19088918',
'https://www.dailyrecord.co.uk/news/politics/nicola-sturgeon-coronavirus-update-live-22799031',
'https://musiccritic.com/equipment/guitars/best-acoustic-electric-guitar',
'https://www.dailyrecord.co.uk/sport/football/football-transfer-news/transfer-deadline-live-rangers-celtic-22792735',
'https://www.gumtree.com/dogs/scotland/page6',
'https://www.moneycontrol.com',
'https://www.getreading.co.uk/news/reading-berkshire-news/live-homes-west-reading-suffer-19062352',
'https://www.hulldailymail.co.uk/news/hull-east-yorkshire-news/3-tier-lockdown-live-updates-4597247',
'https://www.nature.com/articles/ncomms4887',
'https://www.dailyrecord.co.uk/sport/football/football-news/europa-league-draw-live-celtic-22779308',
'https://www.gumtree.com/dogs/uk/dogs+northern+ireland/page2',
'https://alphamom.com',
'https://www.domesticatedcompanion.com/actors-havent-done-anything-in-years/2/',
'https://www.liverpoolecho.co.uk/sport/football/football-news/liverpool-transfer-news-rumours-butland-19055637',
'https://www.insider.com/hard-brainteasers-riddles-for-smart-people-2017-9',
'https://www.catster.com/lifestyle/bringing-an-adult-cat-into-your-home', 'https://www.express.co.uk/life-style/science-technology/1340039/iPhone-12-Release-Date-October-13-2020-Apple-Fans-Not-Long-Left-To-Wait-UK/amp',
'https://www.pinkvilla.com/entertainment/news/saif-ali-khan-talks-about-sara-ali-khan-and-how-if-he-s-hurt-her-taimur-can-t-make-him-feel-better-566895',
'https://reviews.mtbr.com',
'https://www.dailymail.co.uk/news/article-8803493/Trump-administrations-fear-leaks-leaves-White-House-staff-dark.html',
'https://kit.honestjohn.co.uk',
'https://www.trivia.com/view/a-letter-from-this-womans-biological-mother-changed-her-life-forever/',
'https://www.gumtree.com/dogs/uk/puppies/page12',
'https://www.essexlive.news/news/essex-news/live-coronavirus-updates-essex-awaits-4604321',
'https://www.dailymail.co.uk/sport/football/article-8797665/Europa-League-group-stage-draw-2020-21-LIVE-Result.html',
'https://www.lefigaro.fr/politique',
'https://www.chocolate.com/view/life-of-steven-mcqueen-cho/',
'https://www.gumtree.com/dogs/uk/page2',
'https://www.examinerlive.co.uk/sport/football/transfer-news/huddersfield-town-transfer-news-live-19112945',
'https://www.walesonline.co.uk/sport/football/transfer-news/cardiff-city-transfer-news-live-19112387',
'https://www.gumtree.com/cars/uk/audi/a6/page2',
'https://www.thespruce.com/ingenious-ikea-billy-bookcase-hacks-4006865',
'https://iamafoodblog.com/birria-tacos-recipe',
'https://bestofmachinery.com/best-hearing-protection-for-lawn-mowing',
'https://www.wallstreetmojo.com/preferred-shares',
'https://parentinfluence.com',
'https://www.grimsbytelegraph.co.uk/news/uk-world-news/live-boris-johnson-press-conference-4561866',
'https://www.autoscout24.de/lst/', 'https://www.msn.com/en-za',
'https://www.lefigaro.fr/societes',
'https://www.devonlive.com/news/devon-news/planning-applications-submitted-devon-week-4506950',
'https://www.oversixty.com.au',
'https://www.dailypost.co.uk/news/north-wales-news/coronavirus-live-press-conference-take-19099922.amp',
'https://www.trivia.com/view/cool-google-earth-finds-trv/',
'https://www.pinkvilla.com/tv/news-gossip/bigg-boss-6-s-sana-khan-bids-goodbye-showbiz-industry-forever-decides-spend-life-serving-humanity-567296',
'https://www.dailymail.co.uk/sport/football/article-8772027/Crystal-Palace-vs-Everton-Premier-League-Live-result.html',
'https://www.hulldailymail.co.uk/sport/rugby-league/rugby-league-live-transfer-news-4560871',
'https://www.manchestereveningnews.co.uk/sport/football/transfer-news/man-utd-transfer-news-recap-19042171',
'https://www.yardbarker.com/entertainment/gallery/the_most_talked_about_movie_moments_of_the_2010s/s1__30411269',
'https://www.historychronicle.com/view/thomas-jeffersons-hidden-rooms-hc/',
'https://www.gumtree.com/property-to-rent/plymouth/page2',
'https://www.dailymail.co.uk/sport/football/article-8803121/Transfer-News-LIVE-Latest-Premier-League-European-signings-deals-rumours.html',
'https://www.hulldailymail.co.uk/news/uk-world-news/live-coronavirus-uk-lockdown-rules-4579067',
'https://www.buzzfeed.com/quizzes',
'https://www.liverpoolecho.co.uk/news/liverpool-news/live-coronavirus-boris-vaccine-tests-19049134',
'https://www.liverpoolecho.co.uk/sport/football/football-news/everton-news-transfer-rumours-godfrey-19025712',
'https://www.dailyrecord.co.uk/sport/football/football-transfer-news/transfer-news-live-celtic-rangers-22765511',
'https://www.mabelandmoxie.com/amp/These+Are+The+Smartest+Dog+Breeds+In+The+World,+Ranked',
'https://www.pinkvilla.com/entertainment/news/kangana-ranaut-reacts-karan-srk-other-productions-uniting-against-media-houses-file-case-me-also-567902',
'https://www.vielfliegertreff.de',
'https://www.gumtree.com/cars/uk/left+hand+drive/page7',
'https://www.gumtree.com/cats/uk/kittens/page2',
'https://www.chroniclelive.co.uk/news/north-east-news/uk-deaths-coronavirus-live-updates-18935120',
'https://www.manchestereveningnews.co.uk/news/greater-manchester-news/live-mayor-andy-burnham-holds-19023674',
'https://www.telegraph.co.uk/news/2020/10/07/travel-latest-covid-passport-greece-italy-turkey-quarantine',
'https://historicalpost.com/anthropology-and-history/places/niagara-falls-runs-dry-reveals-bodies-1969-united-states-canada/12/',
'https://www.pinkvilla.com/entertainment/news/shabana-azmi-opens-unrest-bollywood-and-kangana-ranauts-outrageous-remarks-566625',
'https://www.simbaly.com/view/the-jackson-kids-now-sim/',
'https://www.dailymail.co.uk/sport/football/article-8784211/Transfer-news-LIVE-Latest-Premier-League-European-club-signings-deals-rumours.html',
'https://www.gumtree.com/dogs/uk/french+bulldog/page3',
'https://powerlisting.fandom.com/wiki/List_of_Supernatural_Powers_and_Abilities',
'https://www.standard.co.uk/news/uk/uk-coronavirus-live-latest-updates-boris-johnson-threetier-lockdown-a4568406.html',
'https://m.khaleejtimes.com',
'https://divinityoriginalsin2.wiki.fextralife.com',
'https://forums.civfanatics.com',
'https://www.gumtree.com/dogs/uk/page6',
'https://www.insider.com/what-are-healing-crystals-how-to-use-them-2018-7',
'https://www.msn.com/fr-fr',
'https://www.gumtree.com/cars/northern-ireland/page2',
'https://www.birminghammail.co.uk/sport/football/transfer-news/wolves-deadline-day-transfers-live-19037027',
'https://www.leeds-live.co.uk/news/uk-world-news/live-coronavirus-sheffield-leeds-rates-19042623',
'https://itsfoss.com/best-linux-desktop-environments',
'https://www.historychronicle.com/view/cleaning-lady-shock-prank/',
'https://flawlessfood.co.uk/olive-herb-focaccia-bread',
'https://www.luxandlush.com/view/woman-sells-a-house-nobody-expected-to-find-this-inside-lux/',
'https://www.liverpoolecho.co.uk/sport/football/football-news/live-liverpool-transfer-news-rumours-19074803',
'https://www.everydayhealth.com/healthy-living/fascinating-facts-about-body-temperature.aspx',
'https://rus.delfi.lv',
'https://www.birminghammail.co.uk/sport/football/transfer-news/villa-live-king-transfer-latest-19062357',
'https://hu.motorsport.com',
'https://www.foxyfolksy.com/chocolate-buttercream-frosting-without-powdered-sugar-ermine-icing',
'https://toppoptoday.com', 'https://bap.navigator.web.de',
'https://trends.bibamagazine.fr/en/prince-william-and-harry-technically-have-a-sister-and-they-completely-ignore-her-2/20',
'https://www.telegraph.co.uk/business/2020/10/06/joe-bidens-war-economy-policies-radical-break-status-quo',
'https://www.gumtree.com/dogs/scotland/page4',
'https://www.birminghammail.co.uk/sport/football/transfer-news/aston-villa-transfers-rashica-benrahma-19037412',
'https://www.standard.co.uk/news/uk/coronavirus-live-latest-updates-pubs-close-a4566031.html',
'https://www.gumtree.com/cars/uk/page5', 'https://www.euronews.com',
'https://www.graduatez.com/view/surprising-findings-pipes/',
'https://www.msn.com/de-ch/',
'https://pawszilla.com/pop-culture/celebrities/fans-expressed-fears-celine-dions-appearance-savagely-responded/65/',
'https://www.liverpoolecho.co.uk/sport/football/football-news/live-everton-transfer-news-bernard-19075370',
'https://www.historychronicle.com/view/ozzy-osbourne-facts-hc/',
'https://zenherald.com/pop-culture/celebrities/fans-expressed-fears-celine-dions-appearance-savagely-responded/30/',
'https://www.manchestereveningnews.co.uk/news/uk-news/pmqs-live-prime-minister-boris-19101281',
'https://www.mirror.co.uk/sport/football/transfer-news/transfer-news-live-smalling-dele-22721256',
'https://sportowefakty.wp.pl/zuzel/relacja/110331/fogo-unia-leszno-rm-solar-falubaz-zielona-gora',
'https://www.gumtree.com/cars/scotland/page3',
'https://guides.gamepressure.com', 'https://www.msn.com/en-gb',
'https://www.chroniclelive.co.uk/news/north-east-news/coronavirus-restrictions-uk-live-updates-19026811',
'https://hipertextual.com',
'https://www.luxandlush.com/view/celeb-kids-are-not-little-anymore-fb-lux/',
'https://www.historychronicle.com/view/70s-80s-celebs-kids-fb-hc/',
'https://m.sportowefakty.wp.pl/pilka-nozna/relacja/106622/polska-finlandia',
'https://www.gumtree.com/cars/uk/range+rover+sport/page2',
'https://www.stokesentinel.co.uk/news/stoke-on-trent-news/live-covid-19-updates-across-4588547',
'https://www.bristolpost.co.uk/sport/football/football-news/bristol-rovers-transfer-questions-garner-4555153',
'https://www.visitlancashire.com/things-to-do/animal-and-farm-attractions',
'https://www.mirror.co.uk/sport/football/transfer-news/transfer-news-live-arsenal-sancho-22752560',
'https://www.lesechos.fr', 'https://www.gumtree.com/for-sale/uk/page2',
'https://www.bristolpost.co.uk/news/health/coronavirus-cases-lockdown-bristol-live-4597246',
'https://www.simbaly.com/view/celeb-daughters-grown-up-co-sim/',
'https://www.buzzfeed.com/alliehayes/weird-house-rules-parents-reddit',
'https://www.gumtree.com/dogs/northern-ireland/page4',
'https://www.autosport.com',
'https://www.liverpoolecho.co.uk/sport/football/football-news/everton-news-transfer-rumours-live-19025712',
'https://www.rtl.be',
'https://www.standard.co.uk/news/uk/coronavirus-live-latest-updates-lockdown-vaccine-a4567091.html',
'https://www.gazettelive.co.uk/news/teesside-news/coronavirus-live-boris-johnson-set-19088101',
'https://cookieandkate.com/perfect-quinoa',
'https://www.chroniclelive.co.uk/sport/football/football-news/newcastle-transfer-news-live-mourinho-19007685',
'https://www.gumtree.com/cars/uk/land-rover/defender/page3',
'https://www.lakesideloops.com/watson-cardigan',
'https://www.chocolate.com/view/woman-sees-a-car-stop-for-pregnant-beggar-this-is-what-she-finds-when-she-follows/',
'https://www.chroniclelive.co.uk/news/north-east-news/boris-johnson-announcement-live-updates-19090211',
'https://www.sports.fr/direct-foot/50918/177512/lorient-olympique-lyonnais.html',
'https://www.pinkvilla.com/entertainment/news/sushant-singh-rajputs-flatmate-siddharth-pithani-states-late-actor-did-not-meet-rhea-chakraborty-june-13-566132',
'https://www.cornwalllive.com/news/cornwall-news/live-storm-alex-weather-traffic-4567887',
'https://www.yourrulingplanet.com/view/instagram-golfer-paige/',
'https://www.dublinlive.ie/news/health/level-5-lockdown-nphet-dublin-19053662',
'https://www.thelist.com/214894/when-you-take-a-multivitamin-every-day-this-is-what-happens-to-your-body',
'https://beta.southglos.gov.uk/new-south-gloucestershire-local-plan-2018-2036',
'https://www.latest-hairstyles.com/mens/thin.html',
'https://www.healthline.com/nutrition/ways-to-measure-body-fat',
'https://www.themakeupdummy.com', 'https://www.audi-sport.net/xf',
'https://www.cheshire-live.co.uk/news/chester-cheshire-news/live-cheshire-coronavirus-national-lockdown-19049856.amp',
'https://www.chroniclelive.co.uk/news/north-east-news/north-east-news-live-latest-19112705',
'https://www.edinburghlive.co.uk/news/edinburgh-news/coronavirus-scotland-live-rules-latest-19046171',
'https://www.ccna7.com/ccna2-v6-0/ccna2-v6-0-chapter-2-exam-answer-2017', 'https://www.autocar.co.uk/slideshow/biggest-flops-automotive-history-0',
'https://www.telegraph.co.uk/news/0/matt-favourite-covid-19-cartoons-think-laughing-something-makes',
'https://www.pinkvilla.com/tv/news-gossip/exclusive-here-s-why-salman-khan-s-bigg-boss-14-will-air-half-hour-there-good-news-565189',
'https://www.manchestereveningnews.co.uk/news/uk-news/live-boris-johnson-leads-downing-19091296',
'https://www.gumtree.com/caravans/uk/page3',
'https://www.cheshire-live.co.uk/news/chester-cheshire-news/live-cheshire-lockdown-coronavirus-council-19021297',
'https://happyyouhappyfamily.com',
'https://thepostpartumparty.com/how-to-set-up-a-baby-nursery-in-a-small-space',
'https://www.realsimple.com/home-organizing/gardening/outdoor/hydrangea-care',
'https://www.devonlive.com/news/local-news/live-m5-fire-bristol-avonmouth-4575486',
'https://www.historychronicle.com/view/priest-sues-millions-hc/',
'https://www.cadena100.es',
'https://www.minq.com/lifestyle/2553886/students-recall-the-absolute-dumbest-thing-theyve-ever-heard-a-teacher-say/',
'https://www.levante-emv.com',
'https://www.authenticfoodquest.com/popular-portuguese-dishes',
'https://www.instyle.com/beauty/wigs-for-black-women-cancer',
'https://www.theprimarymarket.com/view/pizza-delivery-employee-tpm/',
'http://www.msn.com/en-gb/',
'https://www.dailymail.co.uk/sport/football/article-8772175/Sheffield-United-vs-Leeds-Premier-League-Live-result.html',
'https://www.minq.com/lifestyle/2495316/students-share-the-rudest-thing-a-teacher-ever-said-to-them/',
'https://www.nottinghampost.com/sport/football/football-news/nottingham-forest-transfer-news-live-4597836',
'https://www.msn.com/es-es',
'https://www.liverpoolecho.co.uk/sport/football/football-news/liverpool-transfer-news-rumours-kabak-19042594',
'https://wiadomosci.gazeta.pl',
'https://www.nationalrail.co.uk/service_disruptions/245738.aspx',
'https://www.domesticatedcompanion.com/actors-havent-done-anything-in-years/',
'https://www.caughtoffside.com/2020/09/27/agreement-reached-man-united-look-set-to-sign-defender-as-they-reach-an-agreement-on-the-final-details/',
'https://www.corkbeo.ie/news/local-news/level-3-live-cork-ireland-19056312',
'https://www.historychronicle.com/view/single-celebs-hc/',
'https://www.historychronicle.com/view/donny-osmond-confession-hc/',
'https://www.somersetlive.co.uk/news/somerset-news/live-storm-alex-weather-traffic-4568051',
'https://www.theprimarymarket.com/view/elderly-house-sell/',
'https://www.luxandlush.com/view/behind-the-scenes-with-barbi-benton/',
'https://magellantimes.com/pop-culture/celebrities/fans-expressed-fears-celine-dions-appearance-savagely-responded/37/',
'https://www.nerfplz.com',
'https://www.disneytouristblog.com/disney-world-news-star-wars-permit-frozen-ride-refurbishment-restaurants-returning',
'https://trendscatchers.co.uk/index.php/en/2020/07/09/prince-harry-claims-that-meghan-markle-might-not-be-the-one-2/31',
'https://www.ign.com/articles/the-best-ssd-for-gaming',
'https://www.birminghammail.co.uk/sport/football/transfer-news/villa-live-deadline-day-king-19062357',
'https://www.telegraph.co.uk/opinion/2020/10/04/lettersgps-have-worked-crisis-often-personal-cost',
'https://www.liverpoolecho.co.uk/sport/football/football-news/live-everton-transfer-news-rumours-19061794',
'https://www.liverpoolecho.co.uk/sport/football/football-news/liverpool-transfer-news-rumours-live-19025709',
'https://cookieandkate.com/mujaddara-recipe',
'https://trendscatchers.co.uk/index.php/en/2020/07/09/prince-harry-claims-that-meghan-markle-might-not-be-the-one-2/30',
'https://www.theguardian.com/society',
'https://www.nature.com/articles/s41467-018-04619-5',
'https://www.birminghammail.co.uk/sport/football/transfer-news/transfer-news-west-brom-live-19036796',
'https://www.divein.com',
'https://www.historychronicle.com/view/real-noahs-ark-hc/',
'https://www.preloved.co.uk/classifieds/furniture-fittings/kitchen-furniture/all/uk/used+kitchen+cabinets+sale',
'http://wiadomosci.gazeta.pl',
'https://www.dailyrecord.co.uk/sport/football/football-news/celtic-vs-rangers-live-score-22860327',
'https://www.domesticatedcompanion.com/each-state-perfectly-portrayed-in-one-photograph/2/',
'https://www.gumtree.com/dogs/uk/northern+ireland+pets/page2',
'https://www.hertfordshiremercury.co.uk/news/hertfordshire-news/live-boris-johnson-speech-updates-4597736',
'https://www.gazettelive.co.uk/sport/football/transfer-news/middlesbrough-transfers-yannick-bolasie-striker-19111028',
'https://www.cambridge-news.co.uk/news/uk-world-news/live-updates-boris-johnson-tierlockdown-19117181',
'http://www.msn.com/',
'https://www.kentlive.news/news/kent-news/live-updates-covid-infection-rates-4597748',
'https://www.goodhousekeeping.com/holidays/gift-ideas/g1266/handmade-gifts',
'https://www.walesonline.co.uk/news/wales-news/wales-breaking-news-traffic-weather-19075607',
'https://www.visitwiltshire.co.uk/accommodation/self-catering',
'https://www.historychronicle.com/view/bing-crosby-life-hc/',
'https://www.liverpoolecho.co.uk/news/liverpool-news/live-uk-coronavirus-updates-merseyside-19002010',
'https://crinacle.com/2020/02/19/the-audiophiles-perspective-samsung-galaxy-buds',
'https://www.mirror.co.uk/sport/football/transfer-news/arsenal-transfer-news-live-partey-22790130',
'https://www.chocolate.com/view/70s-80s-celebs-kids-fb-cho/',
'https://www.domesticatedcompanion.com/actors-havent-done-anything-in-years/11/',
'https://www.gumtree.com/dogs/uk/jack+russell/page4',
'https://www.haaretz.com/israel-news/business/.premium-this-year-israeli-energy-finally-went-green-but-the-revolution-is-just-beginning-1.9214282',
'https://natashaskitchen.com/chicken-stir-fry-recipe',
'https://www.hulldailymail.co.uk/news/uk-world-news/coronavirus-live-cases-lockdown-updates-4573922',
'https://www.primandprep.com',
'https://www.organizationobsessed.com/20-amazing-organization-hacks-will-transform-bedroom',
'https://www.bakewithpaws.com',
'https://www.oprahmag.com/entertainment/g28335655/best-halloween-books',
'https://www.trivia.com/view/61-of-the-best-red-carpet-mishaps/',
'https://www.examinerlive.co.uk/news/west-yorkshire-news/live-coronavirus-sheffield-leeds-kirklees-19056079',
'https://www.telegraph.co.uk/news/2020/10/10/return-students-caused-inevitable-spike-covid-cases',
'https://www.leeds-live.co.uk/sport/leeds-united/leeds-united-u23s-live-highlights-19036388']
|
This batch is where readability extracts more text:
|['https://redtri.com/best-jokes-for-kids/slide/1',
'https://www.msn.com/en-gb/news/uknews/uk-university-student-halls-too-full-to-be-safe-experts-warn/ar-BB19DUqK',
'https://www.babysleepsite.com/schedules/toddler-schedule/',
'https://www.msn.com/en-gb/foodanddrink/other/slow-cooker-turkey-chilli-is-healthier-but-just-as-comforting/ar-BB19KDYg',
'https://www.dailymail.co.uk/sport/teampages/leicester.html',
'https://www.msn.com/en-gb/lifestyle/travel/can-you-guess-these-capital-cities-from-up-high/ss-BB19DxUe',
'https://www.independent.co.uk/news/uk/politics/coronavirus-false-reporting-contact-fine-penalty-b671230.html',
'https://www.msn.com/en-gb/lifestyle/style/prince-louis-adorable-striped-knit-is-a-10-john-lewis-bargain/ar-BB19HUbk',
'https://www.dailymail.co.uk/tvshowbiz/love-island/index.html',
'https://www.discogs.com/sell/list',
'https://www.msn.com/en-gb/travel/tripideas/these-are-the-world-s-most-beautiful-national-parks/ss-BB19FnGx',
'https://www.msn.com/en-gb/travel/tripideas/this-enchanted-fairytale-treehouse-in-kent-is-the-perfect-autumn-staycation/ar-BB19I6Wz',
'https://www.msn.com/en-gb/lifestyle/style/super-organised-star-closets-we-d-love-to-copy/ss-BB19IOde',
'https://www.futbin.com/21/player/25735/leroy-sane',
'https://www.msn.com/en-gb/news/techandscience/scientists-discover-24-planets-with-conditions-even-more-suitable-for-life-than-earth/ar-BB19HZlb',
'https://www.mirror.co.uk/news/uk-news/breaking-englands-worst-schools-revealed-13899939',
'https://relieved.co/brilliant-kitchen-tips-cooking-easier/',
'https://www.standard.co.uk/news/uk/three-tier-lockdown-system-coronavirus-london-medium-risk-a4570041.html',
'https://www.lifewire.com/screen-mirroring-lg-smart-tvs-4770959',
'https://minecraft-server-list.com',
'https://www.standard.co.uk/news/uk/boris-johnson-coronavirus-nation-briefing-100-a4559471.html',
'https://www.fifplay.com/fifa-21-game-settings',
'https://www.independent.co.uk/news/uk/politics/brexit-boris-johnson-michael-gove-northern-ireland-border-b669752.html',
'https://www.theguardian.com/uk-news/2020/oct/09/rishi-sunak-expands-wage-subsidies-to-head-off-winter-surge-in-job-losses',
'https://www.futbin.com/21/player/25374/thomas-partey',
'https://www.standard.co.uk/news/uk/turkey-poland-caribbean-islands-added-to-uk-quarantine-list-a4561256.html',
'https://www.dailymail.co.uk/news/article-8747257/Is-England-heading-HALF-TERM-lockdown.html',
'https://www.independent.co.uk/news/uk/politics/brexit-no-deal-summit-boris-johnson-eu-fishing-state-aid-von-der-leyen-michel-b1041344.html',
'https://www.msn.com/en-gb/lifestyle/style/kate-middleton-breaks-the-internet-with-her-35-linen-shirt/ar-BB19AQ57',
'https://ext.theperspective.com/items-we-never-knew-we-wanted/1/',
'https://www.futbin.com/21/player/681/ferland-mendy',
'https://www.popsugar.co.uk/smart-living/Cheap-Homemade-Halloween-Costumes-42432483/amp',
'https://www.msn.com/en-gb/money/homeandproperty/celebrity-real-estate-in-october-chris-hemsworth-demi-lovato-and-more/ss-BB19BHew',
'https://spotlightstories.co/50-hilarious-exam-answers-students-given/',
'https://www.futbin.com/21/player/612/leroy-sané',
'https://www.tomshardware.com/uk/reviews/best-gpus,4380.html',
'https://harrypotter.fandom.com/wiki/Hufflepuff',
'https://docplayer.net',
'https://www.theguardian.com/us-news/2020/jun/23/millions-of-americans-cant-afford-water-bills-rise',
'https://www.msn.com/en-gb/news/uknews/hospitality-bailout-scheme-branded-a-con-as-firms-struggle-to-get-1-500-grant/ar-BB19HTbc',
'https://deardesigner.co.uk',
'https://www.crowdyfan.com/worldwide/wedding-gown/42',
'https://homehacks.co/50-useful-beauty-hacks/',
'https://www.theguardian.com/football/live/2020/sep/27/manchester-city-v-leicester-city-premier-league-live',
'https://www.futbin.com/21/player/573/fabinho',
'https://www.crowdyfan.com/worldwide/wedding-gown/13',
'https://www.crowdyfan.com/worldwide/wedding-gown/23',
'https://www.futbin.com/21/player/612/leroy-sane',
'https://www.eadt.co.uk/sport/league-one-transfer-guide-2020-21-1-6881725',
'https://www.t3.com/features/best-running-shoes',
'https://www.crowdyfan.com/worldwide/wedding-gown/6',
'https://www.msn.com/pl-pl/styl-zycia/travel/cudowny-świat-zapierające-dech-w-piersiach-widoki-z-różnych-miejsc-naszej-planety/ss-AAGnBMi',
'https://ext.theperspective.com/clever-and-creative-graffiti/4/',
'https://www.standard.co.uk/news/world/donald-trump-coronavirus-pandemic-comments-timeline-a4561601.html',
'https://www.msn.com/en-gb/foodanddrink/other/james-martin-s-incredible-outdoor-kitchen-will-blow-your-mind/ar-BB19z8ih',
'https://www.cornwalllive.com/news/cornwall-news/confirmed-coronavirus-cases-fall-across-4614786',
'https://www.futbin.com/21/player/570/roberto-firmino',
'https://www.dailymail.co.uk/sport/teampages/southampton.html',
'https://www.standard.co.uk/news/uk/lockdown-measures-north-east-hancock-a4558186.html',
'https://www.independent.co.uk/arts-entertainment/comedy/features/sian-gibson-interview-call-centre-worker-peter-kay-s-car-share-a6811026.html',
'https://www.futbin.com/21/player/25736/nelson-semedo',
'https://www.independent.co.uk/news/uk/politics/brexit-no-deal-tactic-boris-johnson-david-lidington-b599750.html',
'https://www.msn.com/en-gb/lifestyle/other/can-i-leave-a-local-lockdown-to-go-on-holiday/ar-BB19wRic',
'https://www.msn.com/en-gb/entertainment/celebrity/rita-ora-sparks-engagement-speculation-with-ring-on-her-wedding-finger/ar-BB19HfZE',
'https://www.msn.com/en-gb/entertainment/movies/joaquin-phoenix-from-tragedy-to-happy-fatherhood/ss-BB19uYnN',
'https://www.msn.com/en-gb/entertainment/tv/gemma-atkinson-wells-up-live-on-air-after-gorka-marquezs-emotional-surprise/ar-BB19InRs',
'https://www.dailymail.co.uk/news/boris_johnson/index.html',
'https://www.msn.com/en-gb/news/royal-family/claim-meghan-allowed-details-of-her-private-life-to-be-fed-to-authors-of-finding-freedom/ar-BB19CCAp',
'https://www.biblegateway.com/passage/',
'https://www.theguardian.com/football/live/2020/oct/05/transfer-deadline-day-2020-cavani-partey-sancho-and-more-news-live',
'https://www.msn.com/en-gb/lifestyle/style/kate-middleton-breaks-the-internet-with-her-£35-linen-shirt/ar-BB19AQ57',
'https://www.dailymail.co.uk/news/article-8772683/Chef-Jamie-Oliver-joins-Mail-Sundays-war-toxic-food.html',
'https://stardewvalleywiki.com/Penny',
'https://www.plymouthherald.co.uk/news/plymouth-news/confirmed-coronavirus-cases-fall-across-4614786',
'https://www.msn.com/en-gb/money/other/these-amazing-homes-cost-nothing-to-run/ss-BB19CboF',
'https://www.hertfordshiremercury.co.uk/news/hertfordshire-news/38-breathtaking-hertfordshire-walks-perfect-4263258',
'https://ext.theperspective.com/clever-and-creative-graffiti/6/',
'https://www.msn.com/en-gb/entertainment/celebrity/naga-munchetty-looks-wildly-different-with-long-hair-transformation-–-see-photo/ar-BB19uzj3',
'https://www.standard.co.uk/news/uk/closing-schools-increase-coronavirus-deaths-edinburgh-study-a4566076.html',
'https://www.standard.co.uk/news/uk/uk-coronavirus-cases-today-deaths-a4572002.html',
'https://www.futbin.com/21/player/671/david-alaba',
'https://www.standard.co.uk/news/uk/emergency-lockdown-london-north-england-socialising-household-mixing-a4557601.html',
'https://www.futbin.com/21/player/25701/ansu-fati',
'https://www.msn.com/en-gb/travel/tripideas/25-fictional-places-we-d-love-to-visit/ss-BB19EOWK',
'https://www.standard.co.uk/news/london/london-coronavirus-cases-latest-figures-tier-two-lockdown-a4568481.html',
'https://www.msn.com/en-gb/news/world/floods-in-france-italy-swept-bodies-out-of-cemeteries/ar-BB19KCeT',
'https://www.msn.com/en-gb/news/brexit/with-its-legal-action-over-this-uk-bill-the-eu-is-showing-it-means-business/ar-BB19Dh5D',
'https://www.dailymail.co.uk/news/new_zealand/index.html',
'https://www.independent.co.uk/life-style/food-and-drink/coronavirus-restrictions-three-tier-lockdown-boris-johnson-restaurants-bars-pubs-hospitality-b990820.html',
'https://www.standard.co.uk/showbiz/celebrity-news/nicola-adams-lesbian-gay-strictly-come-dancing-a4542561.html',
'https://www.msn.com/pl-pl/styl-zycia/podroze/na-zdjęciach-majestatyczne-góry-z-całego-świata/ss-BBYaukt',
'https://www.futbin.com/squad-building-challenges/ALL/21/Give Me Five',
'https://www.op.gg/champion/statistics',
'https://statisticsglobe.com/change-font-size-of-ggplot2-plot-in-r-axis-text-main-title-legend']
|
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/adbar/trafilatura/issues/25#issuecomment-716710049,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAQHAKX323JPNCFP5XNXOMDSMWXNTANCNFSM4S5WOLJQ.

Hey @adbar to answer your points:

I am using the latest version from pypi 0.5.2
yes using the latest of reareadability-lxml 0.8.1
yes I agree, less can be better! I am only including URLs where the difference was over 6k chars
within the list I provided of URLs where readability returns more, there are many where trafilatura returns None

Some fixes could be needed for a few patterns found in these webpages (partly addressed in 62b1e9ab249edc1ca003afb7b86c949c597a4692 and 25a08edec50b300894d033c07ef71f2436d0260e), that said text extraction remains a balancing act... I added a few pages to the evaluation, things should get slightly better by the next release.
I'd say that for most webpages in your list nothing is really wrong, what to do with pages like https://www.futbin.com/21/player/25374/thomas-partey remains a open for discussion. I'd say trafilatura remains more efficient overall in terms of precision and recall. Thanks again for the input!

Yeah I get that its a non exact process and maybe impossible to handle all cases.
When is the next release due?
Thanks for looking into this!

I plan to ship the next release by the end of next week at the latest.

The new release is out, if you have the occasion could you please confirm it's gotten better? It does in my benchmark.

Yeah will give it a try and come back to you!


[New game]: Flappy Bird

🎮 Game Request
Game logic and basic description
The HTML and CSS are used to structure and style the game interface, while JavaScript handles the game's logic and interactivity. JavaScript provides the necessary functionality to detect player input, control the bird's movement, calculate collisions, and update the game's state in real-time.
Point down the features
Game points
The player uses button  or mouse to tap the bird to make it fly and avoid the obstacles in the path.
Select program in which you are contributing
GSSoC23
Code of Conduct

[X] I follow CONTRIBUTING GUIDELINE of this project.

Hey @Teddy1404 !
Thank you for raising an issue 💗
You can self assign the issue by commenting /assign in comment 😀
Make sure you follow CODE OF CONDUCT and CONTRIBUTING GUIDELINES 🚀
Don’t Forget to ⭐ our GameZone🎮
Make sure you join our Discord🕹

Hey @Teddy1404 ! Thank you so much for your raising the issue💗
It’s all yours, you can come anytime again and make some contributions! 🚀
Alone, we can do little, but together we can do so much! 😇


Loss and acc validation

Hello @layumi
Thank you for your valuable work. I have a question regarding the train.py file. In the draw_curve function, I noticed that the values of y_err['val'] and y_loss['val'] are empty. I have checked the code, but I'm unable to find where the validation values are calculated. Could you please guide me on where I can find the calculation of these values and suggest a place where I can print the values to display them in the log?
Thank you!

Hi @zahraahmadi1996
I do not set validation set in this work and dataset.
Therefore, I disable the validation.


Configure Azure Pipelines

This pipeline configuration builds the application and produces a build for the Release configuration.

done


MODEXPS-130: Upgrade Users interface to 16.0

@khandramai
thank you!  If this Pr is acceptable, can you please merge?  I don't have access to the repo.


Provide a proc macro to register actions

One very cool things about GTK is the usage of GAction. You can either easily add an action to a widget that implements the necessary interfaces to be an GActionGroup  (gio::ActionMap & gio::ActionGroup)  or you have to add a SimpleActionGroup to your widget yourself.  While the creation part of the actions is very verbose already with all the casting from GAction to GSimpleAction, it's currently not easy nor nice to write.
I have written in the past few macros that did nothing but wrap the procedure of creating an action, adding to the action group along with all the casting stuff and it looks currently more or less like this
fn setup_gactions(&self) {
        // Quit
        action!(
            self,
            "quit",
            clone!(@weak self as app => move |_, _| {
                app.quit();
            })
        );

        // About
        action!(
            self,
            "about",
            clone!(@weak self as app => move |_, _| app.display_about_dialog())
        );

        let win = self.get_main_window();
        action!(
            win,
            "fullscreen",
            clone!(@weak self as app => move |_, _| {
                app.get_main_window().toggle_fullscreen();
            })
        );
    }

Note that more complex apps can have a ton of actions, so having to define them "twice" is not great.
I suggest we provide a proc macro based on the future work in https://github.com/gtk-rs/gtk-rs/issues/128 to automatically create an action and use the annotated function as the activate callback.
So that the example above would look more or less like this
#[gaction(name="quit")]
fn on_quit(&self, action: gio::SimpleAction, target: Option<glib::Value>) {
    self.quit();
}
#[gaction]
fn about(&self, action: gio::SimpleAction, target: Option<glib::Value>) {
    let dialog = gtk::AboutDialog::new();
    ...
    dialog.show();
}

#[gaction]
fn fullscreen(&self, action: gio::SimpleAction, target: Option<glib::Value>) {
    self. app.get_main_window().toggle_fullscreen();
}

This will require having some kind of register_actions methods to call later on to register all those actions in a specific action group (or something else? not sure how that could work)

@sdroege @bilelmoussaoui I started to implement such macro for one of my applications (which I use as a playground) here. I this something you wanna have in gtk-rs?

@sdroege @bilelmoussaoui I started to implement such macro for one of my applications (which I use as a playground) here. I this something you wanna have in gtk-rs?

That would be super cool if you can work on it. What I'm still not sure about, is where to register those actions if the current Self doesn't implement gio::ActionMap

That would be super cool if you can work on it. What I'm still not sure about, is where to register those actions if the current Self doesn't implement gio::ActionMap

@bilelmoussaoui In my example Self does not implement ActionMap. It is self.0 who implements it.

@bilelmoussaoui In my example Self does not implement ActionMap. It is self.0 who implements it.

Now that #190 has landed, I think we should have almost everything in place to move this forward

Now that #190 has landed, I think we should have almost everything in place to move this forward


User Errors

I am fairly new at using ansible.  I am really not certain what exactly I am doing wrong.
I created a role using the ansible-galaxy init command - cis_ubuntu_2004 and copied all of the folders into the new role I had created.
I executed this command: sudo ansible-playbook -i hosts playbook_with_defaults.yml and have tried adding tags "level_1", but I continue to get errors.
TASK [cis_ubuntu_2004 : <IP_ADDRESS> | Ensure mounting of cramfs filesystems is disabled (Automated) | custom template] ***************
fatal: [<IP_ADDRESS>]: FAILED! => {"msg": "The conditional check 'ubuntu_2004_cis_section1' failed. The error was: error while evaluating conditional (ubuntu_2004_cis_section1): 'ubuntu_2004_cis_section1' is undefined\n\nThe error appears to be in '/etc/ansible/roles/cis_ubuntu_2004/tasks/section_01.yml': line 7, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n    - name: "<IP_ADDRESS> | Ensure mounting of cramfs filesystems is disabled (Automated) | custom template"\n      ^ here\n"}
Do you have any idea why I am getting these errors?
Thank you for your assistance.

No worries @eljefe-3, happy to help.
ERROR:
The cause of task failure is:
error while evaluating conditional (ubuntu_2004_cis_section1): 'ubuntu_2004_cis_section1' is undefined

EXPLANATION:
Each of the tasks in the role have when conditions as well as tags associated. Purpose of this was to allow any users who utilize this role to be able to customize CIS sections and subsections based on their business needs.
If we look at the error, that means the task is failing because it cannot identify where the dependent when condition requirement is set., which in this case is ubuntu_2004_cis_section1.  Now, ubuntu_2004_cis_section1 is a boolean ansible variable (i.e. true or false).

If variable is defined and declared as true, then the task runs.
If variable is defined and declared as false,  then the task skips.
If variable is not defined or declared, then the task fails.

The code for the task <IP_ADDRESS> itself is in the below snippet:
https://github.com/darkwizard242/cis_ubuntu_2004/blob/aaccda1ce3c10e599a16d2d751741d0481441496/tasks/section_01.yml#L4-L27
The variable by default has been declared in the below snippet:
https://github.com/darkwizard242/cis_ubuntu_2004/blob/aaccda1ce3c10e599a16d2d751741d0481441496/defaults/main/main.yml#L7
SUGGESTION:
It might be highly possible that either the variables files originally located in defaults/main or just the variable itself might be missing in the location that you may have copied to, which from the error you shared looks like: /etc/ansible/roles/cis_ubuntu_2004
You can also do a quick check using the following command (and share output if possible):
sudo grep -irn "ubuntu_2004_cis_section1:" /etc/ansible/roles/cis_ubuntu_2004

First off, thank you for your willingness to help.  As I previously
mentioned I am new to using Ansible, but here are the steps that I have
taken.
Per the instructions, I ran the following command and it downloaded the
repo in /home/ubuntu/.ansible
ansible-galaxy install darkwizard242.cis_ubuntu_2004,master
I ran the /home/ubuntu/.ansible/roles/darkwizard242.cis_ubuntu_2004
ansible-playbook playbook_defauflt.yml --tags: "level_1" -i hosts
[image: image.png]
In order to get rid of the error I created a new role in
/etc/ansible/role/cis_ubuntu_2004.  I proceeded by deleting all the
files and folders and copied the entire directory previously cloned.
I executed the grep commanded listed above for each of the sections.
The output was true.  Although I did change the value from true to
false on section 1 to test, but continue to get errors.  Again, I know
it is probably a step I missed that is why I thought I show all the
steps.
Thanks,
Wilberto
On Fri, Feb 12, 2021 at 10:29 PM Ali Muhammad<EMAIL_ADDRESS>wrote:

No worries @eljefe-3 https://github.com/eljefe-3, happy to help.
ERROR:
The cause of task failure is:
error while evaluating conditional (ubuntu_2004_cis_section1): 'ubuntu_2004_cis_section1' is undefined
EXPLANATION:
Each of the tasks in the role have when conditions as well as tags
associated. Purpose of this was to allow any users who utilize this role to
be able to customize CIS sections and subsections based on their business
needs.
If we look at the error, that means the task is failing because it cannot
identify where the dependent when condition requirement is set., which in
this case is ubuntu_2004_cis_section1. Now, ubuntu_2004_cis_section1 is a
boolean ansible variable (i.e. true or false).

If variable is defined and declared as true, then the task runs.
If variable is defined and declared as false, then the task skips.
If variable is not defined or declared, then the task fails.

The code for the task <IP_ADDRESS> itself is in the below snippet:
https://github.com/darkwizard242/cis_ubuntu_2004/blob/aaccda1ce3c10e599a16d2d751741d0481441496/tasks/section_01.yml#L4-L27
The variable by default has been declared in the below snippet:
https://github.com/darkwizard242/cis_ubuntu_2004/blob/aaccda1ce3c10e599a16d2d751741d0481441496/defaults/main/main.yml#L7
SUGGESTION:
It might be highly possible that either the variables files originally
located in defaults/main
https://github.com/darkwizard242/cis_ubuntu_2004/tree/master/defaults/main
or just the variable itself might be missing in the location that you may
have copied to, which from the error you shared looks like:
/etc/ansible/roles/cis_ubuntu_2004
You can also do a quick check using the following command (and share
output if possible):
sudo grep -irn "ubuntu_2004_cis_section1:" /etc/ansible/roles/cis_ubuntu_2004
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/darkwizard242/cis_ubuntu_2004/issues/3#issuecomment-778555213,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ANKNWURF6IV3QELTI6MFOWDS6XWZFANCNFSM4XROUH4A
.


ACE registry

... is now available at
https://github.com/ACEsuit/ACEregistry

This should now be used instead of the MolSim registry. Can somebody please update the docs?

I will take care of this.


Restored thin polylines if line width <= 0

The alternative to #44337, #44338 which achieve the same goal by using parameter width <= 0.0
Fix #44332

Personally, I would expect a line with width <= 0 to be an error, not a 1px line at any zoom level.  I'd prefer the version that has separate functions for thin lines.

I've lost interest, closed


Feature Request: Slim (Template) support

Slim is a Ruby HTML Template engine alternative to Haml and ERB.
It would be nice to see slim template engine support with this outstanding extension.
In slim, we use tabbed syntax and everything starts with . (dot) is a class. Everything starts with  # (hash) is an ID.
Whenever we hit dot on keyboard, this plugin can start to show us possible class names.

Note: dots in strings ''/"" or followed by | does not count.

Sample:
.my-sass-class-1.another-class
	a#link-1.this-is-a-class class="we-also-have-this-too"
		| Hello.
#my-id.class

Reference: https://github.com/slim-template/slim

I will go with my drop-in replacement for this. See here
Reason I did not send Pull request was;

I need it urgently on few environments. So local deploy is not something I want. I needed this extension install-able through marketplace.
I've manually processed SCSS instead of how did you do in CSS. I thought you might reject and not like that. (again I did in need of urgent solution.)

If you are comply with my reasons, I can un-publish it, send you the pull request and go with your way. Until then I would like to keep my extension as it is.

You can use "html-css-class-completion.includeGlobPattern": "**/*.{css,html,scss}", to add other extensions.

@muuvmuuv, Of course that was the first thing I tried. This won't solve classes in a class, and magic class functions. My implementation just does it very well. Also Slim support is number one priority for me -which I solved it too-.

Ah, I understand, but why did you not simply created a fork and pull request for your feature? That would reduce duplicate plugins with the same functionality in the VSCode Marketplace?
Please ask the author about to merge with you extension :)

Ah sorry did not read your second comment. Anyway, I hope the author will be fine or you both find a way to make this plugin more scalable for future templates.


awswrangler.s3.describe_objects() fails with client error 403 in version 2.9 when a wildcard path contains multiple objects

Describe the bug
Given an S3 path of the form s3://example_bucket/path/to/key/prefix_* that you have access to, if there are multiple objects under this path, then the call awswrangler.s3.describe_objects() will fail with error botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden. This does not occur when there only exists a single object under the path, in which case the outcome matches the expected behavior and the call succeeds. This is unrelated to authentication.
Testing confirms that this issue does not exist on major versions 2.7 and 2.8. It is new in 2.9.
Environment
I can't share all of our dependencies, but here are the relevant AWS packages from our environment.yml file:
- awswrangler=2.9
- boto3~=1.17
- botocore~=1.20

To Reproduce

To show that the issue is awswrangler.s3.describe_objects() and not an e.g. auth issue, attempt both awswrangler.s3.describe_objects() and awswrangler.s3.list_objects() on a path you have access to with the form described in the foregoing bug description, i.e. s3://example_bucket/path/to/key/prefix_* and compare the outputs. Only the describe_objects call results in a 403 client error.

To show that the issue is related to paths including shell-style wildcards, attempt an awswrangler.s3.describe_objects() call on a regular bucket path you have access to, containing multiple objects under it, without a wildcard in the path: e.g. s3://bucket_name/path/to/key/prefix/, where under prefix there is object1, object2, ...

To confirm that this issue is new in major version 2.9, attempt the two previous reproduction steps with major versions 2.7, 2.8.

To confirm that this is an issue in awswrangler, and not e.g. boto3 or botocore, hold those versions constant while changing awswrangler versions and confirming that the issue is not present in earlier major versions of awswrangler.

Hi @fractionalhare, first of all thank you for this issue description, it's clear and thorough.
I am finding it difficult to reproduce it however. In my test setup, I have uploaded 4 files with the same prefix:

s3://my-bucket/my-prefix/dummy_1.txt
s3://my-bucket/my-prefix/dummy_2.txt
s3://my-bucket/my-prefix/dummy_3.txt
s3://my-bucket/my-prefix/dummy_4.txt

Then ran the below commands which match your description:
path = "s3://my-bucket/my-prefix/"
prefix = "dummy"

print("Version: ", wr.__version__)
print("List Objects: ", wr.s3.list_objects(path=f"{path}{prefix}*"))
print("Describe Object: ", wr.s3.describe_objects(path=path).keys())
print("Describe Object: ", wr.s3.describe_objects(path=f"{path}{prefix}_1.txt").keys())
print("Describe Objects: ", wr.s3.describe_objects(path=f"{path}{prefix}*").keys())

Results are as expected, event the last call which you mention is failing for you:
Version:  2.9.0
List Objects:  ['s3://my-bucket/my-prefix/dummy_1.txt', 's3://my-bucket/my-prefix/dummy_2.txt', 's3://my-bucket/my-prefix/dummy_3.txt', 's3://my-bucket/my-prefix/dummy_4.txt']
Describe Object:  dict_keys(['s3://my-bucket/my-prefix/dummy_1.txt', 's3://my-bucket/my-prefix/dummy_2.txt', 's3://my-bucket/my-prefix/dummy_3.txt', 's3://my-bucket/my-prefix/dummy_4.txt'])
Describe Object:  dict_keys(['s3://my-bucket/my-prefix/dummy_1.txt'])
Describe Objects:  dict_keys(['s3://my-bucket/my-prefix/dummy_1.txt', 's3://my-bucket/my-prefix/dummy_2.txt', 's3://my-bucket/my-prefix/dummy_3.txt', 's3://my-bucket/my-prefix/dummy_4.txt'])

I have also created multiple paths with the same prefix:

s3://my-bucket/dummy_1/dummy_1.txt
s3://my-bucket/dummy_1/dummy_2.txt
s3://my-bucket/dummy_2/dummy_1.txt
s3://my-bucket/dummy_2/dummy_2.txt
s3://my-bucket/dummy_3/dummy_1.txt
s3://my-bucket/dummy_3/dummy_2.txt

and ran:
path = "s3://my-bucket/dummy"

print("Version: ", wr.__version__)
print("List Objects: ", wr.s3.list_objects(path=f"{path}*"))
print("Describe Object: ", wr.s3.describe_objects(path=f"{path}_1/").keys())
print("Describe Objects: ", wr.s3.describe_objects(path=f"{path}*").keys())

again without any issue:
Version:  2.9.0
List Objects:  ['s3://my-bucket/dummy_1/dummy_1.txt', 's3://my-bucket/dummy_1/dummy_2.txt', 's3://my-bucket/dummy_2/dummy_1.txt', 's3://my-bucket/dummy_2/dummy_2.txt', 's3://my-bucket/dummy_3/dummy_1.txt', 's3://my-bucket/dummy_3/dummy_2.txt']
Describe Object:  dict_keys(['s3://my-bucket/dummy_1/dummy_1.txt', 's3://my-bucket/dummy_1/dummy_2.txt'])
Describe Objects:  dict_keys(['s3://my-bucket/dummy_1/dummy_1.txt', 's3://my-bucket/dummy_1/dummy_2.txt', 's3://my-bucket/dummy_2/dummy_1.txt', 's3://my-bucket/dummy_2/dummy_2.txt', 's3://my-bucket/dummy_3/dummy_1.txt', 's3://my-bucket/dummy_3/dummy_2.txt'])

I must admit that the fact it's failing for you for a specific Wrangler version is perplexing. Could you please provide more details on how the files are stored in the bucket? Likewise, if you could provide more details on the Wrangler commands you are running and the parameters used. Ideally we would want a snippet of code with actual Wrangler commands that reproduces your issue.
Finally, the IAM role used for my tests is an admin, not that it should matter but you never know...

Closing for now as no response. Feel free to reopen in the future


Fix broken documentations in maximum and minimum.

This PR fixes broken documentations in maximum and minimum functions.

LGTM 👍


No callback when destroying websocket client tracker with consummers > 1

Hi :)
If consummers > 1, the callback is not fired (https://github.com/feross/bittorrent-tracker/blob/master/lib/client/websocket-tracker.js#L142).
I don't have the documentation of the destroy method so I don't know if a "normal comportment".
But it seems to be the cause of an issue with webtorrent: if you have add 2 (or more) torrents of the same tracker and  you want to remove one of them the callback is not fired.

Thanks for reporting this. Looks like it is indeed an issue.

8.0.0

Great, thanks a lot :)


[SPARK-31022][SQL] group by alias should fail if there are name conflicts

What changes were proposed in this pull request?

Make group by alias fail if there are name conflicts like SELECT col + 1 as col FROM t GROUP BY col.
Why are the changes needed?

It's super confusing that SELECT col + 1 as new_col FROM t GROUP BY new_col and SELECT col + 1 as col FROM t GROUP BY col works differently.
Does this PR introduce any user-facing change?

yes, group by alias now fails if there are name conflicts.
How was this patch tested?

new tests

cc @maropu @gatorsmile

Test build #119241 has started for PR 27775 at commit 1317477.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23981/
Test PASSed.

Test build #119241 has finished for PR 27775 at commit 1317477.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119241/
Test FAILed.

It turns out the TPCDS queries also have name conflicts. I think users can only accept it and be careful when writing GROUP BY columns.


Add game mode Shipwrecked

Every commit is more or less atomic and represents a feature which the primary commit is dependent on. I will be doing a merge commit on this PR, as opposed to raising a dozen individual PRs, in the event forks want certain features but not others.
I think only commit 44a6fe5c554ba5caabd472f187890a5a89f49f53 is dependent on e95a51320395037f0090a657fed4324de2f753fe. The rest should be clean isolates.
This game mode has had some testing but not a thorough run through. Consider it in a beta status. I'm hoping to get some feedback from players. It is not in the Secret pool.
:cl:

add: Added a new game mode for small servers and low player counts: Shipwrecked. Vote for it in the lobby.

Huge

Shader variables need precision specified to work in compat mode

Alright, that should fix it.

Shaders work in compat mode now

I had tests pass locally, but I'll give it a moment while I update the main PR description to document my folly with shaders.

Ubuntu debug & release passed, and that is very well for me.


Syntax highlighting malfunctions with raw string literals

When raw string literals that use ##s appear in attributes, syntax highlighting gets messed up. Example:

Literals outside of attributes are not affected.
Editor: VSCode

Hm, can't trivially reproduce this:

Not sure what might be the problem.

@matklad It seems that it only occurs when the literal contains an odd number of quotes.

Mystery solved!
The problem is with the build-in textmate grammar for rust:
https://github.com/microsoft/vscode/blob/master/extensions/rust/syntaxes/rust.tmLanguage.json
I get the following if I disable rust-analyzer plugin:

OTOH, if i enable rust-analyer and minimal rust (empty grammar for rust), I get the following:

So I believe the issue should be reported upstream, against vscode repository.


Can't use e_sqlcipher and e_sqlite with .net 6 ios project

Hello,
I would like use sqlite and sqlcipher with EF Core in my .net 6 IOS project but there are some linking errors. (With .net 6 Mac and Windows, it's ok).
My cproj: (I tested with registrar static and without, same issue)
https://pastebin.com/G3YXj6hW
My output:
https://pastebin.com/4UYHDhMt
I have that with each libraries versions. How can I make this work?
Have a nice day !

    <PackageReference Include="Microsoft.EntityFrameworkCore.Sqlite" Version="6.0.1" />
    <PackageReference Include="SQLitePCLRaw.lib.e_sqlcipher.ios" Version="2.0.7" />

The problem here is that Microsoft.EntityFrameworkCore.Sqlite is bringing in sqlite binaries, and then you're also bringing in the e_sqlcipher binaries as well.
Try using these two packages instead: Microsoft.EntityFrameworkCore.Sqlite.Core and SQLitePCLRaw.bundle_e_sqlcipher.

Yes, it works. Thank you !
For use sqlcipher (and sqlite3), I have that:

2.0.7


Custom Opponents (dad) Notes

Describe your problem here.
Hello! I'm making my mod, and i'm have issue. I'm don't know how to make custom opponents notes in Psych Engine 4.2. Like it be in Bob and Bosip mod. Like:
https://cdn.discordapp.com/attachments/982251158926155796/997779081514979378/dad.png
Please, help me!
Are you modding a build from source or with Lua?
Source
What is your build target?
Windows x64
Did you edit anything in this build? If so, mention or summarize your changes.
I'm edited PlayState.hx, and HealthIcon.hx

I only know how to do with lua , source I have no idea


Optimize nth_value, remove first_value, last_value structs and use idiomatic rust style

Which issue does this PR close?

Closes #448 .
Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

Codecov Report

Merging #452 (80ad5a2) into master (321fda4) will increase coverage by 0.15%.
The diff coverage is 92.24%.

@@            Coverage Diff             @@
##           master     #452      +/-   ##
==========================================
+ Coverage   75.16%   75.31%   +0.15%
==========================================
  Files         150      151       +1
  Lines       25144    25253     +109
==========================================
+ Hits        18899    19020     +121
+ Misses       6245     6233      -12

Impacted Files
Coverage Δ

datafusion/src/physical_plan/expressions/mod.rs
71.42% <ø> (ø)

...afusion/src/physical_plan/expressions/nth_value.rs
78.57% <78.57%> (ø)

datafusion/src/physical_plan/windows.rs
87.12% <100.00%> (+13.34%)
:arrow_up:

datafusion/tests/sql.rs
99.89% <100.00%> (ø)

ballista/rust/core/src/utils.rs
0.00% <0.00%> (ø)

ballista/rust/executor/src/flight_service.rs
0.00% <0.00%> (ø)

ballista/rust/core/src/serde/logical_plan/mod.rs
99.40% <0.00%> (ø)

ballista/rust/core/src/serde/physical_plan/mod.rs
100.00% <0.00%> (ø)

...lista/rust/core/src/serde/logical_plan/to_proto.rs
62.40% <0.00%> (+0.03%)
:arrow_up:

... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 321fda4...80ad5a2. Read the comment docs.

@alamb as promised :-)


Editor displaying export properties for Resource script on Node with Node script (script have the same name)

Godot version
3.3.stable.mono
System information
Windows 10
Issue description
Two C# scripts with the same name, but different namespace and in different folders, one extending Node, the other extending Resource. If you add the script extending Node to a node in a scene, the editor will display the exported properties from the script that extends Resource, and those properties will be saved to the .tscn file. The path in the Script field is the path to the script that extends Node.
Happened in 3.3.stable.mono and also in 3.4.beta2.mono.
Steps to reproduce

Create new project
Create folder Namespace1
Create script Class1.cs - extends Node, surround class with namespace Namespace1 {}
Create folder Namespace2
Create script Class1.cs - extends Resource, surround class with namespace Namespace2 {}, add an Export property
Build .csproj
Create Scene
Add Namespace1.Class1.cs script to node in scene
Export property from Namespace2.Class1.cs shows up in inspector on node in editor

Namespace1.Class1.cs
using Godot;
using System;

namespace Namespace1
{
    public class Class1 : Node
    {

    }
}

Namespace2.Class1.cs
using Godot;
using System;

namespace Namespace2
{
    public class Class1 : Resource
    {
        [Export]
        public int MyProperty { get; set; }
    }
}

Minimal reproduction project
ClassNameConfusion.zip

I tried the provided project on v3.3 and v3.4.beta.mono.custom_build [95162ca39] but I can't reproduce the described behaviour.

I am not able to reproduce this with v3.3 anymore either. Weird...

Closing, as nobody has been able to reproduce this recently. Please comment if you can still reproduce this on Godot 3.3.3 or later.


Add cumulative_mstar_formed function to dsps.utils

This PR brings in a new kernel, dsps.utils.cumulative_mstar_formed, that computes the cumulative stellar mass formed from an input SFH table. A few notes:

Trapezoidal integration is used to integrate an input sfh_table across an input t_table under the assumption that SFR=SFR_MIN during the interval dsps.constants.T_BIRTH_MIN < t < t_table[0].
The implementation requires that t_table[0] >= dsps.constants.T_TABLE_MIN.
Note that T_BIRTH_MIN=0.001 Gyr corresponds to z~600, T_TABLE_MIN=0.01 Gyr corresponds to z~140, and SFR_MIN=1e-14, so these boundary conditions should not impact galaxy mass at any redshift of practical interest.

Cumulative trapezoidal integration is not implemented in jax.numpy, and so this PR brings in a new custom kernel for this computation, dsps.utils.cumtrapz, which is based on lax.scan.

Prior to this PR, stellar mass was integrated by computing an array δt from the input t_table, and then simply taking jnp.cumsum(sfh_table * δt). This gives a reasonable approximation except for dwarf-mass galaxies at high-redshift z>~3. The implementation brought in with this PR should give more accurate results for galaxies in this regime.
Note that this PR only brings in the integration kernel but these changes have not yet been propagated to the rest of the DSPS repository.


remove mcmc_comparison_vis_code

I plan to remove this directory unless any one points out a reason to leave it.  I think it has all been integrated into compareMCMCs and related functions.

Agreed, safe to remove.

Done a while ago.


fix .tabs-component-tab-a class user-select

.tabs-component-tab-a class you should be add css attr user-select:none, because the mouse will be selected when sliding

I'd agree with this change. @willemvb do you have an opinion on user-select?

@shaodahong Decided not to pull this. Seems to make more sense to keep this unopinionated. Tab titles can contain crucial content, in which case you'll want to be able to select them.
This behaviour van easily be added on a case-by-case basis, so we're not going to ship this by default.


Switch to edition 2018

Coverage increased (+4.2%) to 94.096% when pulling aaa72c33724bbbf8867a8625e53837f33fe03738 on switch-2018-edition into 4a532ba1175c175a288e3c303de89769e6cc0e01 on master.


modified image series annotator

As discussed, I changed a lot of things to:

Load images of different sizes.
Easily change brush size using up and down arrows to manually change segmented object.
Addition of a button to manually modify a not-satisfying result.
Currently, RGB images are computed as grayscale, as I didn't manage to make it work. This modification is dedicated to making batch processing smoother.

Hi @koopa31
I realized how to implement the image annotator in a better and more re-usable way, see #56. So I will go ahead and close this.
Regarding these two contributions:

Easily change brush size using up and down arrows to manually change segmented object.
Addition of a button to manually modify a not-satisfying result.

I think this makes sense, and I would be happy to see it as a contribution! However, it would be better if you can first create an issue about it where you describe the intended functionality, then I can give you some pointers on where you need to change code for it. (Because the current implementation would break a lot of things.)


[Docs] add cncf logo and code-of-conduct

Add cncf logo to README
Add code of conduct (coc) file to follow cncf style

@windsonsea Thanks for doing this.
/lgtm


FutureWarning raised when constructing lil_matrix

Description
Constructing a scipy.sparse.lil_matrix in UMAP.inverse_transform raises a FutureWarning
Versions
OS: macOS
Python 3.7.7 (default, Mar 26 2020, 10:32:53)
NumPy 1.17.5
SciPy 1.2.1
umap 0.4

Steps/Code to Reproduce
I put the code to reproduce the issue in this gist
Cause
This issue is caused by a backward incompatibility of scipy < 1.3 with the latest updates of Numpy (numpy>=1.17) APIs for broadcast_arrays.
scipy 1.3.1 fixes this issue: scipy/scipy#10379
Proposed solution
Looking at UMAP requirements.txt this is a perfectly valid combination of packages versions.
Due to the recent API changes in Numpy I would suggest to updates the requirements with most recent and non-conflicting numpy and scipy versions:
numpy>=1.17
scipy>=1.3.1

Thanks -- good catch. I am not great at keeping up with the changing requirements.


[BUG]: drizzle breaks typescript lib.es2015.collection.d.ts

What version of drizzle-orm are you using?
0.28.5
What version of drizzle-kit are you using?
0.19.13
Describe the Bug
Since version 0.28.3 looks like drizzle is somehow conflicting with typescript collection lib.
For instance: document.fonts.add(fontFace) gives the error Property 'add' does not exist on type 'FontFaceSet' or for (const child of element.children) {...} gives the error Type 'HTMLCollection' must have a '[Symbol.iterator]()' method that returns an iterator.
my tsconfig:
{
  "compilerOptions": {
    "target": "esnext",
    "lib": [
      "dom",
      "dom.iterable",
      "esnext"
    ],
    ...
  }
}

Downgrading to 0.28.2 fixed the issue.
Expected behavior
No response
Environment & setup
macOS 13.5
nodejs 20.2.0
next 13.4.20-canary.18

Just tried on latest version of drizzle-kit and it works as expected, can't reproduce
<EMAIL_ADDRESS>Feel free to reopen this issue, if it's still an issue for you


Lack of Clear Installation Instructions for Environment and Repository

Dear Stefanie Molin,
I hope this message finds you well. I recently purchased your book, "Hands-On Data Analysis with Pandas – Second Edition," and I must express my frustration regarding the lack of clear instructions on how to download the associated GitHub repository and set up an interactive environment in Jupyter Notebook.
I've spent the last three hours grappling with this issue and sought assistance from various sources, including ChatGPT, but unfortunately, I've yet to find a satisfactory solution. The material in your book, particularly on this topic, seems unwelcoming to newcomers, making the learning experience needlessly challenging.
I believe that providing a clear and beginner-friendly guide on downloading the repository and establishing an interactive environment in Jupyter Notebook would greatly enhance the overall learning experience for your readers. As a consumer, I find it disheartening when critical information is not adequately explained, hindering the learning process.
I understand that creating comprehensive materials can be a challenging task, and I appreciate the effort you've put into the book. However, I urge you to consider revising and expanding on the explanations for these crucial steps in future editions or supplementary materials.
Thank you for your attention to this matter. I hope my feedback proves constructive and contributes to the improvement of your excellent book.

Hi there,
I appreciate the feedback. However, without specific details of what you are struggling with, I cannot help you or improve the instructions.
Pages 4 and 5 talk about getting the files locally. You have multiple options with the relevant buttons circled on the screenshot on page 5. You can also look at the GitHub documentation's general explanation of the process here. I see you already forked the repository, so you just have click the big green "Code" button and pick your option.
Let me know once have the files locally, and I can help with the virtual environment setup if you are still struggling. Alternatively, you can try Binder (no installation, but subject to availability).

Thank you for your response and recommendations. Currently, I find it beneficial to review the code directly on your GitHub. However, during a more in-depth second reading, I plan to utilize the code on Binder for practice and experimentation.


Document Compaction Throughput Config Option

Add documentation to describe: https://github.com/influxdata/influxdb/pull/9939
URL for relevant page?
http://docs.influxdata.com/influxdb/v1.6/administration/config/#data-settings-data
What products and version are you using?
InfluxDB 1.6.1
Default config will align precisely with the current "hardcoded" setting.  This means the behavior will be the same as before.  However, it will allow for more fine tuning over the compaction process beyond just the number of processors to leverage.

Should I close https://github.com/influxdata/docs.influxdata.com/issues/1654 here? Does this replace that?

Sorry @zachgoldstein -- you did this perfectly!  I duped your original issue.  I'll close my dup.


Updaing readme

Added gateways we have added + updating universal text
@girasquid @ivanfer

Two questions, otherwise :+1: