Image table contains .ogv videos

Description

Wikimedia script ingests .ogv video files as images. We should filter them out and probably also drop the ones we already added from the catalog database.
Reproduction

Go to https://openverse.org/image/2dc93cee-34d6-4b98-a76e-92ebc8d4ae73?q=phytoplankton+bloom

The main image is unavailable, as are most of the related images.

If you go to the source, you will see that this is actually a video of `.ogv` extension.

Additional context

These images were noticed by @sarayourfriend in #4665

The structured data for this result on Wikimedia shows the media type as application/ogg.
That's actually interesting, because I don't know how you would disambiguate that from an audio file if you don't have the extension in the URL!
https://commons.wikimedia.org/w/index.php?curid=59122955
At least in this case, we do have the direct file URL and can use the extension from it.

That's actually interesting, because I don't know how you would disambiguate that from an audio file if you don't have the extension in the URL!

In the Wikimedia API result, this media is probably marked as image in the mediatype key of the response:
https://github.com/WordPress/openverse/blob/23467c8d57b593a758aac43cd20211e49a927fdb/catalog/dags/providers/provider_api_scripts/wikimedia_commons.py#L450
We accept as image all media that is marked as "BITMAP" or "DRAWING". It appears that some of the media marked with those values are of application/ogg type, so we should be filtering them out.

It's good to note that "image", in some metadata standards (DublinCore, for example) includes both moving and still images. Something to keep in mind, especially when interfacing with GLAM institutions that may be using terms like that in their metadata.
I'm surprised wikimedia considers those bitmaps or drawings, though 🙂 bitmap doesn't seem to me as generic as "image", though I suppose it can be if interpreted loosely, which it already is anyway!

Code changes are prepared in #4772, and I confirmed there are 45397 of these items in Wikipedia incorrectly classified.
openledger=> SELECT COUNT(*) FROM image WHERE source = 'wikimedia' AND url ~ '^.*.(ogv)$';
 count
-------
 45397

I want to proceed with cleaning up manually by running an update similar to the one used in #1796.
DELETE FROM image WHERE source = 'wikimedia' AND url ~ '^.*.(ogv)$';

@WordPress/openverse-catalog Can I get a second review on this? I don't think they need to be kept separate but what do you think?

@krysal for what it's worth, LIKE might be faster for a simple suffix match (url LIKE '%.ogv') and prevents easy-to-miss regex bugs like missing the escape on the second period. ^.*.(ogv)$ will match any string ending in "ogv" with any one or more character prefix, not just those ending in ".ogv". A simpler pattern match approach would avoid that complexity entirely and is worth using in contexts like these if only for that simplicity. Also, as written, the query is case sensitive, have you confirmed that's okay? ~* is case insensitive (or ILIKE/~~* if you decide to switch to a simpler pattern match). Wikimedia Commons does not appear to normalise file extensions to lowercase, here's an example of a capitalised GIF extension (warning: flashing lights in the animation).
Are OGV the only video types Wikimedia considers images? Wanted to make sure we aren't missing a greater issue based on the ogv examples we originally found by accident. I don't even see ogv listed in this image file format listing page: https://commons.wikimedia.org/wiki/Category:Images_by_file_format. It also isn't in their file type dropdown on their own search: https://commons.wikimedia.org/w/index.php?search=&title=Special:MediaSearch&go=Go&type=image
It seems like we should report these videos as incorrectly categorised to Wikimedia Commons, right? In which case, we need to extract these into a CSV before deleting them.

@sarayourfriend Thanks for bringing that up! I overlooked the case sensitivity, which is funny because I was exactly finding a lot of weird case extensions here ('Jpg', 'JPg', 'jPG', 'jpG', 'JPG', 'jpeg', 'jPeG', 'Jpeg', 'JPEG', 'png', 'Png', 'pnG', 'PNG' and the list go on...); you're right, and I like the more uncomplicated pattern match as well :)

Are OGV the only video types Wikimedia considers images?

I don't know, but they might have fixed that categorization. The few rows I looked at were all ingested in 2020 (the maximum date is pending to get the query result). Would you want me to store them in a separate table before deleting them?
The updated instruction would be:
DELETE FROM image WHERE source = 'wikimedia' AND url ILIKE '%.ogv';

PD: I'm also working on your comments in #4772.

PDFs might be okay, at least from Wikimedia, provided they are mostly PDFs representing images rather than text (they are sometimes used as a wrapper around TIFF or JPEG scans)? We are able to get thumbnails of them from Wikimedia now, and they can be used as images in many contexts (imported easily into editors like Photoshop, GIMP, etc). The example from the issue works now. It feels more on the edge to me than videos, which are definitely not images in the sense we use that word (still images).

After a more extensive scan, I found many more files incorrectly ingested in the image table from Wikimedia.

query
SELECT extension, COUNT(extension)
FROM (
    SELECT LOWER(RIGHT(url, 4)) AS extension FROM image
    WHERE source = 'wikimedia' AND url NOT ILIKE ALL(ARRAY[
        -- well known image types
        '%.apng', '%.avif', '%.bmp', '%.ico', '%.jpg', '%.jpeg', '%.png', '%.gif', '%.svg', '%.tif', '%.tiff', '%.webp',
    ])
) AS tmp_ext
GROUP BY 1;

Sidenote: It's a shame we can't use negative indexes in the Postgres' split_part` function. It is available from v14.

 extension | count
-----------+-------
 djvu      |   918
 .mid      |    17
 midi      |     3
 mpeg      |     2
 .mpg      |    65
 .ogg      |   124
 .ogv      | 45401
 .pdf      | 50144
 .stl      |   904
 .wav      |    26
 webm      |    34
 .xcf      |  1263
(12 rows)

From that list, 3 types are technically classified as images.

file type
visible in openverse
sample link

.djvu
✅
https://openverse.org/image/e141c4e0-239b-4c67-80d0-d5cce5d8d08c

.stl
❌
https://openverse.org/image/9fadc89c-e08f-455c-b645-52c3250bf470

.xcf
❌
https://openverse.org/image/054b17fa-fdd7-42bb-a1a4-8989fbe4a074

We render .djvu files correctly, but .stl and .xcf files are not shown on the image detail page in Openverse. It would be cool to show those 3D models of Wikimedia (.stl), though I'm not sure what to do with the .xcf files.
Regarding pdf, I'll vote for removing them, they're ocasionally displayed on Openverse, like while navigating between related results, because of the cached thumbnail but going to the indiviual page directly does not show them, eg. pdf1, pdf2. Try refreshing one of the related images that is a PDF. It will also complicate type validations. Will that mean that potentially, a .xls or other doc type containing an image should also be allowed? Before going that deep, it's better to avoid that by adding more first-class media types (videos, 3D models, documents).
For now, I exported these wrong rows from Wikimedia into the s3://openverse-catalog/deletions/wikimedia_non_images_2024-08-23.tsv file @sarayourfriend, and removed them from upstream except for PDFs, waiting to get feedback on this discussion. If the file is not required anymore (given @obulat's clarification in this comment), feel free to delete it. I also created a snapshot with today's date before running the delete. Below, you can find all the SQL instructions.

query

-- Confirmation of numbers of rows to export to s3 and then delete

SELECT extension, COUNT(extension) FROM (
    SELECT LOWER(RIGHT(url, 4)) AS extension FROM image
    WHERE source = 'wikimedia' AND url ILIKE ANY(ARRAY[
        -- known non-image types
        '%.mid',
        '%.midi',
        '%.mpeg',
        '%.mpg',
        '%.ogg',
        '%.ogv',
        '%.pdf',
        '%.wav',
        '%.webm'
    ])) AS tmp_ext
GROUP BY 1;

extension | count
-----------+-------
 .mid      |    17
 midi      |     3
 mpeg      |     2
 .mpg      |    65
 .ogg      |   124
 .ogv      | 45401
 .pdf      | 50144
 .wav      |    26
 webm      |    34
(9 rows)

-- export
SELECT * FROM aws_s3.query_export_to_s3(
    'SELECT foreign_landing_url, url FROM image WHERE source = ''wikimedia'' AND (url ILIKE ANY(ARRAY[''%.mid'', ''%.midi'', ''%.mpeg'', ''%.mpg'', ''%.ogg'', ''%.ogv'', ''%.pdf'', ''%.wav'', ''%.webm'']))',
    'openverse-catalog',
    'deletions/wikimedia_non_images_2024-08-23.tsv',
    'us-east-1'
);

extension | count
-----------+-------
 .mid      |    17
 midi      |     3
 mpeg      |     2
 .mpg      |    65
 .ogg      |   124
 .ogv      | 45401
 .wav      |    26
 webm      |    34
(8 rows)

-- removal (skipping .pdf)
DELETE FROM image WHERE source = 'wikimedia' AND url ILIKE ANY(ARRAY['%.mid', '%.midi', '%.mpeg', '%.mpg', '%.ogg', '%.ogv', '%.wav', '%.webm']);

DELETE 45672

Pending is to decide what to do with the .stl  and .xcf and .pdf rows.

My recommendations are:

.stl files: Keep them, show the thumbnail on the single result page but add some text somewhere like "View in 3D at the source"
.xcf files: Figure out how to show a larger PNG preview on the single result view.

I would also vote for removing pdf and djvu files because they are often multi-page documents. We can display them once we add "books" as another format :)
I like the recommendations for stl and xcf. I tried to understand if Sketchfab Viewer can be used with stl, but couldn't find it in the documentation. Do you know, @zackkrida ?

@obulat it only works with models hosted on Sketchfab.

Of these formats we are discussing, the only one Wikimedia considers a bitmap is XCF. Everything else is not an image according to MediaWiki's classification. Why would we keep any of these incorrectly ingested works? We will not ingest any new PDFs, STLs, etc, due to filtering on MediaWiki's classification of BITMAP and AUDIO.
Thanks to @obulat for the link to the list of media types from them: https://doc.wikimedia.org/mediawiki-core/master/php/MimeMap_8php_source.html
Can STL be used as an image? If not, and image search is for finding images to use, I don't see how 3d models connect to that purpose. Similar to why we excluded ogv even though we can create a thumbnail. PDF is the same, even if it can be an image, MediaWiki isn't holding only PDF images. We incorrectly ingested those.
Dejavu are ebooks, not images. I can't see them in my browser (Firefox or Chromium), and again, it is hard to understand how that fits into the remit of "image search".
The suggestion for XCF makes sense because it is an image format that can be used as an image. This is definitely a separate GitHub issue to implement that, though. @zackkrida since you made the suggestion, can you create the issue for improving XCF display on the single result page? Using the thumbnail URL for the single result page of XCF files and passing full_size=True and compressed=False should be enough to solve that for the frontend.
To refocus on the core of this issue, which is to deal with these bad records in the database. Is all that is left to delete the works we don't want to keep, right? @krysal can you share an updated query that would do that? Would it be more reliable to select every ending that is not one of the BITMAPs in the MediaWiki code, so we know we've found everything that we would not today ingest from Wikimedia as an image?

@zackkrida since you made the suggestion, can you create the issue for improving XCF display on the single result page?

Sure thing!
Concerning STL, I suggested displaying thumbnails on the single result page because this is precisely what we do for Thingiverse. For Sketchfab, another 3D model provider currently included with our images, we have special logic to display their model viewer on the single result pages.
It would certainly be preferable to move these works into a "3D Model" content type, even if we don't yet have a 3D model viewer for the non-Sketchfab results. Still, having Sketchfab, Wikimedia Commons, and Thingiverse results is already a fairly robust 3D model collection, so perhaps this work is warranted sooner than later (even if marked as "beta" or a similar designation).
All of that is to day: my thinking was that even if it isn't ideal, following the existing precedence for handling 3D models as images, for now, might better set us up for the future.

Chiming in to say I agree with the recommendations above, namely:

Exclude .pdf, for the reasons Krystle mentions.
Try to support .xcf if we can.
Hold off on including .stl for now. While we do include Thingiverse & Sketchfab, this would be another source of data we'd need to move over to 3D models when that time comes and we don't have to include it under images. That's a loosely held opinion though, given that we have precedence, I would not object to including them as well.

Of these formats we are discussing, the only one Wikimedia considers a bitmap is XCF.

Dejavu are ebooks, not images. I can't see them in my browser (Firefox or Chromium), and again, it is hard to understand how that fits into the remit of "image search".

@sarayourfriend .djvu is also considered images by Wikimedia: 'image/vnd.djvu' => [ 'djvu,' 'div' ] and the Python's mimetypeslib. However, I wrongly assumed that they render correctly in my case, it's the same as .xcf in that I was looking at the thumbnail instead of the actual file. I'm unsure what to do with this tho, given these facts.
As to .stl, my reasoning for keeping them was similar to Zack's: We could give them a similar treatment to Sketchfab and Thingiverse. I thought we could put them in a separate table, but a frontend workaround might be easy and good enough. Maybe what you suggest can apply too to .djvu files?

To refocus on the core of this issue, which is to deal with these bad records in the database. Is all that is left to delete the works we don't want to keep, right? @krysal can you share an updated query that would do that? Would it be more reliable to select every ending that is not one of the BITMAPs in the MediaWiki code, so we know we've found everything that we would not today ingest from Wikimedia as an image?

I made that query in this comment, that's how I find out what remained to be removed. I can run the same again to confirm of course. I'll response back when that finishes, and make the issue for XCF display.

I made that query in https://github.com/WordPress/openverse/issues/4717#issuecomment-2307927998, that's how I find out what remained to be removed

Gosh, I'm sorry! I totally missed the fact that "query" was a collapsed summary/details block and interpreted it as introducing the table of results below it. You've covered the bases then, never mind me!

Maybe what you suggest can apply too to .djvu files?

Do you mean the suggestion to use the thumbnail? I've just tried loading the example djvu result you shared and this time it loaded! Before I only got a broken file:

However, the link of the image being rendered is actually the thumbnail link (https://api.openverse.org/v1/images/e141c4e0-239b-4c67-80d0-d5cce5d8d08c/thumb/). I didn't think to look before, so I don't know if when it didn't load whether there was an issue with the thumbnail or if it was trying to load the actual file.
In any case, it does look like Wikimedia's thumbnails endpoint works for it, so the approach Zack suggested of using a larger version of the thumbnail should work. So yes, your suggestion to use the same strategy for djvu seems good, if we keep those files :+1:
Further consideration on DjVu files
On the other hand, it's very interesting that they are considered images but not PDFs, because if you visit the example file on Wikimedia, you can go through the individual pages of the book. I can see a justification for us keeping djvu files if they are always collections of images (instead of PDFs which can be mixed images and text). It would match the fact that we have scanned pages of books from Biodiversity Heritage Library, for example. If we keep them, we should plan to do some additional work to improve how we index and/or display them. That work could have multiple phases ranging in complexity.
To begin with, we should indicate that additional pages may be available by visiting the source and that only the first page is displayed on Openverse. That would be in addition to updating the handling of djvu files to match the suggestion for XCF.
Beyond that, we could index each individual page (as with BHL), and connect them as part of a collection of images. The ability to do that is something I've touched on being necessary to improve the handling of book scans and other types of directly related images (#1290 #2992). That approach would be of immediate benefit to BHL (which has severe data quality issues, I mention in the #1290 discussion) and other "scans of books" sources like #1742. It's a lot of work and goes down to a data architecture level, plus significant design and UI implementation concerns (not to mention careful consideration for search). It would be either one huge project to cover all aspects of this, or a series of smaller projects. Maybe start with data architecture changes, including thinking about the needs described in #1290; then move on to UI design, implementation, and search considerations. It's huge.
All of that is possible, but none of it is blocked on the smaller improvement. I agree with your suggestion to give djvu images the same thumbnail-based display treatment as XCF, and suggest also adding more context to the single result pages for djvu to indicate the special considerations for them (multiple pages).
I added handling djvu image display the same as XCF to the XCF issue you created. I've also created an issue to cover adding the necessary context about djvu.
Finally, I created a stub issue for the data architecture project, and another for improving the display of multi-image works like ebooks.

I have seen this and will respond to this discussion when I have time!

I ran the SQL to remove the remaining PDF files and confirmed that the image table is clean. The .stl files are still there, waiting for a more explicit action/consensus, but I think we can close this issue since it has expanded its reach much further than initially anticipated.


Rodrigo.villarreal/fixing typos in readme

README.md was modified, 2 words typos were fixed. The first one, the docker-compose.yaml file doesn't match with the actual filename and the second one, PosstgreSQL, was fixed (an extra s in the name)
Also, the linter modified to files so I commited them too.

To point to the right file and to have the right database name.

No need to verified this change, it's a readme file

Nothing

No

No

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Pull Request Test Coverage Report for Build 40beddb0-4157-47c5-a477-7cf7687cd3c7

1 of 1 (100.0%)  changed or added relevant line in 1 file are covered.
28 unchanged lines in 4 files lost coverage.
Overall coverage increased (+0.06%) to 67.132%

Files with Coverage Reduction
New Missed Lines
%

host/signalworkflowTest.go
1
95.53%

service/history/shard/context.go
2
63.06%

common/task/fifoTaskScheduler.go
3
84.54%

common/persistence/cassandra/cassandraTaskPersistence.go
22
59.16%

Totals

Change from base Build a8fd4a2a-a8ba-4e3a-905f-25833e78d216:
0.06%

Covered Lines:
98378

Relevant Lines:
146545

💛  - Coveralls


TLV format for single instance resource

The recent versions of the spec say now TLV can be used for single resources.

"The LWM2M Client MUST support the TLV data format for all
requests." (§6.3 Data Formats for Transferring Resource Information)

(There is a PR in Leshan which implements this)

Thanks for pointing this.
Currently for single resource request, the client can encode the data using TLV if the CoAP GET message content type is application/vnd.oma.lwm2m+tlv. If there is no content type specified, the client uses plain text by default.
On the server-side, the API does not allow to specify the content-type. I'll rework this part.
Regards,


Remove --max-rows Flag from Snapshot Gadgets

#2173   Remove --max-rows Flag from Snapshot Gadgets
This PR addresses the issue of the --max-rows flag not having any effect on the snapshot process and snapshot socket commands. The flag has been removed to prevent confusion and ensure the commands operate as expected without unused options.
How to use
To validate this PR, reviewers should build the inspektor-gadget executable and run the snapshot process and snapshot socket commands to ensure they work correctly without the --max-rows flag.
Testing done

Built the inspektor-gadget executable.
Ran the snapshot process and snapshot socket commands to ensure they function without the --max-rows flag.
Confirmed the commands produce the expected output without any errors.

This was fixed by #3327. Now the flag is called --max-entries. Thanks for your contribution!


Update Team.md

Created my own description & removed some typos from other team members description

Please resolve the merge conflict and remove the label when resolved. @Akshay2996 Thanks.

Deleted the file & created another one. Merged successfully.

#266 File that I created replacing this file & got merged.


Create sample robotframework test for 0-scenario

Who What & Why
I'm as a developer is able to chech that 5GC with eUPF works properly in order to verify changes in configuration or application.
Additional info
Restrictions
Acceptance criteria
Definition of done

[x] At least 0-scenario is configured in robot test
[ ] Test run presented to team

eupf.zip


psql relation does not handle roles

In the original psql relation used in the reactive charm, the user can request roles by setting the "roles" field to a comma separated list of desired roles. In this charm, this field is silently ignored, leading the user to scratch his head in frustration.
Proposed solutions:

Implement the feature for the legacy interface, or
Make it not silent, so that the user understands why he is not getting any roles.

Hi, @awnns, thank you for the report. We will update this issue when we decide how to proceed with roles in the legacy interface. The new postgresql_client interface should already support this, so please take a look to see if it is something that can work for you in the short term.

Hi, @awnns, due to the legacy interface being deprecated, we decided to make the charm block when roles are requested. We have updated the documentation to mention that and should update the charm shortly. Please, consider migrating your charms to the new postgresql_client interface.

Hi, latest edge (rev. 350) should block if roles are requested via the legacy interface.


[v0.7] Remove autogluon.text

Remove autogluon.text in v0.7, as autogluon.mulitmodal fully replaces it.

Resolved in #2847


No Update on Map

Tried launching run.launch after that played ros bag no update in rviz. tried launching each node individually. one node crashes when other is launched.

I have the same problem... I have tried different bag files and I can see the Lidar points in rviz but there is no map...

i encountered the same problem [mapOptmization-7] process has died [pid 9116, exit code 127the node that crash is map optimization also no map isn't updating

i encountered the same problem [mapOptmization-7] process has died [pid 9116, exit code 127the node that crash is map optimization also no map isn't updating

I encountered the same issue as you did. Have you managed to resolve it?

try this sudo apt-get install libparmetis-dev and make sure you have installed the right version of gtsam for ros noetic

is there someone solve it ？

Still encountering this problem, anyone has any suggesstions?

Still encountering this problem, anyone has any suggesstions?


Yahoo API not working.

ConnectionError: HTTPConnectionPool(host='ichart.finance.yahoo.com', port=80): Max retries exceeded with url: /table.csv?ignore=.csv&f=2017&e=28&a=0&g=d&s=FB&b=1&d=5&c=1970 (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x114c53198>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known',))

There is a fix here #395.

Should be fixed on master.


Pro Yakyuu Spirits 2019 [PCSG01248]

App summary

App name: Pro Yakyuu Spirits 2019
App serial: PCSG01248
App version: 1.00

Vita3K summary

Version: v0.1.5
Build number: 2686
Commit hash: https://github.com/vita3k/vita3k/commit/27e0c030
CPU backend: Dynarmic

Test environment summary

Tested by: gymzatan
OS: Windows 11
CPU: AMD Ryzen 7 5800H
GPU: AMD Radeon(TM) Graphics
RAM: 16 GB

Issues
The screen flashes at the menu. Game crashes after entering the tutorial mode.
Screenshots

Log
vita3k.log
Recommended labels

Ingame -
graphics bug
crash


Add config reader to start_dev.sh & basic config

Since the app requires environment variables to bet set, I created a config.toml file, and added some code to the setup_dev.sh script to parse those entries.  The TOML format was used because that's Rusts preferred syntax.
Also updated the setup_dev.sh script to pull in the websocket_server bin.
TODO:

Maybe move these values to the config?

export TRUNK_SERVE_PORT=8081
export ACTIX_PORT=8086

Test with actual/URLs

@BanjoFox how's this PR coming together?

Gave everything a once over one more time, and looks okay to me 🤞
Please feel free to comment.

After making the changes requested, I did re-run the script via VS Code and it was able to bring up the server, and give me a login screen.

@BanjoFox can you check if the UI is starting but the backend is crashing due to the failure to connecting to the nats server?
Imo it is ok if it fails as long as we document that people are expected to launch their own nats server locally, you could also look at running nats-server


Parameter format=jpg generate a png

Hello, Thanks for your very nice work !!!
I've noticed something, maybe it's me but it seems that the JPG format isn't working
To Reproduce
Steps to reproduce the behavior:
http://x.x.x.x:3000/?bcid=code39&format=jpg&text=FAC123456&includetext
Doesn't seems to create a jpg image when I look at the format of the file (I'm using a browser with http or https)
I'm using your docker image https://hub.docker.com/r/padiazg/barcode-generator
Screenshots

Thanks a lot
Florian

I realized that the image on docker-hub may not be up to date

All good :)


[FEATURE REQUEST] Would like to be able to customize the color of inline Message.

As shown in the figure, I want the inline message to be lighter than my comment color.

Hey. You can use gitblame.inlineMessage.

Hey. You can use gitblame.inlineMessage.嘿。你可以使用 gitblame.inlineMessage。

I'm not sure. Do you mean in the settings.json file? For example,"gitblame.inlineMessage":“#545454"？ This is not valid.

It is a color token name.

It is a color token name.它是一个颜色标记名称。

grateful! This works.


Introducing test dependency on bionty is problematic

Cycles in dependencies should be avoided wherever possible. Here it's not strictly needed, and hence, I'll remove it to avoid hiccups like the one I'm currently having.

(bionty is strictly downstream of lnschema_core.)

FYI @Zethson


Impeller Text PaintingStyle.stroke still appears to be inferior to Skia implementation

Is there an existing issue for this?

[X] I have searched the existing issues
[X] I have read the guide to filing a bug

Steps to reproduce

Create a Flutter app.
Implement text border per the Borders and Stroke example at https://api.flutter.dev/flutter/painting/TextStyle-class.html.
Execute with Impeller.
Execute without Impeller (either by using the flutter run --no-enable-impeller command or by updating the plist file.
Compare results.

Expected results
Expectation is that the Impeller rendering should match the Skia rendering. Please note that I was using Stable channel with even worse Impeller results so tried Beta channel with the documented results here.
Actual results
The border of the Impeller version appears to be misaligned and not as 'crisp' in definition when inspected closely.
The example files show the Skia version has consistently placed borders while the Impeller version has borders that appear to be slightly misaligned to the foreground text.
(impeller)

(skia)

Code sample
Code sample

import 'package:flutter/material.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  // This widget is the root of your application.
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        // This is the theme of your application.
        //
        // TRY THIS: Try running your application with "flutter run". You'll see
        // the application has a purple toolbar. Then, without quitting the app,
        // try changing the seedColor in the colorScheme below to Colors.green
        // and then invoke "hot reload" (save your changes or press the "hot
        // reload" button in a Flutter-supported IDE, or press "r" if you used
        // the command line to start the app).
        //
        // Notice that the counter didn't reset back to zero; the application
        // state is not lost during the reload. To reset the state, use hot
        // restart instead.
        //
        // This works for code too, not just values: Most code changes can be
        // tested with just a hot reload.
        colorScheme: ColorScheme.fromSeed(seedColor: Colors.deepPurple),
        useMaterial3: true,
      ),
      home: const MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  const MyHomePage({super.key, required this.title});

  // This widget is the home page of your application. It is stateful, meaning
  // that it has a State object (defined below) that contains fields that affect
  // how it looks.

  // This class is the configuration for the state. It holds the values (in this
  // case the title) provided by the parent (in this case the App widget) and
  // used by the build method of the State. Fields in a Widget subclass are
  // always marked "final".

  final String title;

  @override
  State<MyHomePage> createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  int _counter = 0;

  void _incrementCounter() {
    setState(() {
      // This call to setState tells the Flutter framework that something has
      // changed in this State, which causes it to rerun the build method below
      // so that the display can reflect the updated values. If we changed
      // _counter without calling setState(), then the build method would not be
      // called again, and so nothing would appear to happen.
      _counter++;
    });
  }

  @override
  Widget build(BuildContext context) {
    // This method is rerun every time setState is called, for instance as done
    // by the _incrementCounter method above.
    //
    // The Flutter framework has been optimized to make rerunning build methods
    // fast, so that you can just rebuild anything that needs updating rather
    // than having to individually change instances of widgets.
    return Scaffold(
      appBar: AppBar(
        // TRY THIS: Try changing the color here to a specific color (to
        // Colors.amber, perhaps?) and trigger a hot reload to see the AppBar
        // change color while the other colors stay the same.
        backgroundColor: Theme.of(context).colorScheme.inversePrimary,
        // Here we take the value from the MyHomePage object that was created by
        // the App.build method, and use it to set our appbar title.
        title: Text(widget.title),
      ),
      body: Center(
        // Center is a layout widget. It takes a single child and positions it
        // in the middle of the parent.
        child: Column(
          // Column is also a layout widget. It takes a list of children and
          // arranges them vertically. By default, it sizes itself to fit its
          // children horizontally, and tries to be as tall as its parent.
          //
          // Column has various properties to control how it sizes itself and
          // how it positions its children. Here we use mainAxisAlignment to
          // center the children vertically; the main axis here is the vertical
          // axis because Columns are vertical (the cross axis would be
          // horizontal).
          //
          // TRY THIS: Invoke "debug painting" (choose the "Toggle Debug Paint"
          // action in the IDE, or press "p" in the console), to see the
          // wireframe for each widget.
          mainAxisAlignment: MainAxisAlignment.center,
          children: [
            Stack(
              children: [
                Text('You have pushed the button this many times:',
                  style: TextStyle(fontSize: 30,
                    fontWeight: FontWeight.bold,
                    foreground: Paint()
                      ..style = PaintingStyle.stroke
                      ..strokeWidth = 2
                      ..color = Colors.red,
                  ),
                ),
                const Text('You have pushed the button this many times:',
                  style: TextStyle(fontSize: 30,
                    fontWeight: FontWeight.bold,
                    color: Colors.amber,
                  ),
                ),
              ],
            ),
            Text(
              '$_counter',
              style: Theme.of(context).textTheme.headlineMedium,
            ),
          ],
        ),
      ),
      floatingActionButton: FloatingActionButton(
        onPressed: _incrementCounter,
        tooltip: 'Increment',
        child: const Icon(Icons.add),
      ), // This trailing comma makes auto-formatting nicer for build methods.
    );
  }
}

Screenshots or Video

Screenshots / Video demonstration
[Upload media here]

Logs
Logs
[Paste your logs here]

Flutter Doctor output
Doctor output
[✓] Flutter (Channel beta, 3.16.0-0.2.pre, on macOS 14.0 23A344 darwin-arm64, locale en-NZ)
    • Flutter version 3.16.0-0.2.pre on channel beta at /Users/geoff/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision fe6553b689 (5 days ago), 2023-10-11 19:07:11 -0700
    • Engine revision 249cc9b86c
    • Dart version 3.2.0 (build 3.2.0-210.2.beta)
    • DevTools version 2.28.1

[!] Android toolchain - develop for Android devices (Android SDK version 33.0.1)
    • Android SDK at /Users/geoff/Library/Android/sdk
    ✗ cmdline-tools component is missing
      Run `path/to/sdkmanager --install "cmdline-tools;latest"`
      See https://developer.android.com/studio/command-line for more details.
    ✗ Android license status unknown.
      Run `flutter doctor --android-licenses` to accept the SDK licenses.
      See https://flutter.dev/docs/get-started/install/macos#android-setup for more details.

[✓] Xcode - develop for iOS and macOS (Xcode 15.0)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 15A240d
    • CocoaPods version 1.11.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2022.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.15+0-b2043.56-8887301)

[✓] IntelliJ IDEA Community Edition (version 2023.1)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart

[✓] VS Code (version 1.83.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.74.0

[✓] Connected device (3 available)
    • iPad (10th generation) (mobile) • 92CF6F32-017B-456E-A63F-C924E8201348 • ios            • com.apple.CoreSimulator.SimRuntime.iOS-17-0 (simulator)
    • macOS (desktop)                 • macos                                • darwin-arm64   • macOS 14.0 23A344 darwin-arm64
    • Chrome (web)                    • chrome                               • web-javascript • Google Chrome 118.0.5993.70

[✓] Network resources
    • All expected network resources are available.```

</details>

Thanks for the report. Also verified on latest master wherein it shows below with and without impeller:

With Impeller:

Without Impeller:

On stable though, with Impeller, it shows as below:

 stable, master flutter doctor -v
[!] Flutter (Channel stable, 3.13.6, on macOS 12.2.1 21D62 darwin-x64, locale
    en-GB)
    • Flutter version 3.13.6 on channel stable at
      /Users/dhs/documents/fluttersdk/flutter
    ! Warning: `flutter` on your path resolves to
      /Users/dhs/Documents/Fluttersdk/flutter/bin/flutter, which is not inside
      your current Flutter SDK checkout at
      /Users/dhs/documents/fluttersdk/flutter. Consider adding
      /Users/dhs/documents/fluttersdk/flutter/bin to the front of your path.
    ! Warning: `dart` on your path resolves to
      /Users/dhs/Documents/Fluttersdk/flutter/bin/dart, which is not inside your
      current Flutter SDK checkout at /Users/dhs/documents/fluttersdk/flutter.
      Consider adding /Users/dhs/documents/fluttersdk/flutter/bin to the front
      of your path.
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision ead455963c (5 days ago), 2023-09-26 18:28:17 -0700
    • Engine revision a794cf2681
    • Dart version 3.1.3
    • DevTools version 2.25.0
    • If those were intentional, you can disregard the above warnings; however
      it is recommended to use "git" directly to perform update checks and
      upgrades.

[!] Xcode - develop for iOS and macOS (Xcode 12.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    ! Flutter recommends a minimum Xcode version of 13.
      Download the latest version or update via the Mac App Store.
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] VS Code (version 1.62.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[✓] Connected device (5 available)
    • SM G975F (mobile)       • RZ8M802WY0X • android-arm64   • Android 11 (API 30)
    • Darshan's iphone (mobile)  • 21150b119064aecc249dfcfe05e259197461ce23 •
      ios            • iOS 14.4.1 18D61
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 98.0.4758.80

[✓] HTTP Host Availability
    • All required HTTP hosts are available

! Doctor found issues in 1 category.

[!] Flutter (Channel master, 3.16.0-11.0.pre.92, on macOS 12.2.1 21D62
    darwin-x64, locale en-GB)
    • Flutter version 3.16.0-11.0.pre.92 on channel master at
      /Users/dhs/documents/fluttersdk/flutter
    ! Warning: `flutter` on your path resolves to
      /Users/dhs/Documents/Fluttersdk/flutter/bin/flutter, which is not inside
      your current Flutter SDK checkout at
      /Users/dhs/documents/fluttersdk/flutter. Consider adding
      /Users/dhs/documents/fluttersdk/flutter/bin to the front of your path.
    ! Warning: `dart` on your path resolves to
      /Users/dhs/Documents/Fluttersdk/flutter/bin/dart, which is not inside your
      current Flutter SDK checkout at /Users/dhs/documents/fluttersdk/flutter.
      Consider adding /Users/dhs/documents/fluttersdk/flutter/bin to the front
      of your path.
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision f69e45c4bb (80 minutes ago), 2023-10-16 23:32:13 -0400
    • Engine revision 6855e78a1a
    • Dart version 3.3.0 (build 3.3.0-29.0.dev)
    • DevTools version 2.28.1
    • If those were intentional, you can disregard the above warnings; however
      it is recommended to use "git" directly to perform update checks and
      upgrades.

[!] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at /Users/dhs/Library/Android/sdk
    ✗ cmdline-tools component is missing
      Run `path/to/sdkmanager --install "cmdline-tools;latest"`
      See https://developer.android.com/studio/command-line for more details.
    ✗ Android license status unknown.
      Run `flutter doctor --android-licenses` to accept the SDK licenses.
      See https://flutter.dev/docs/get-started/install/macos#android-setup for
      more details.

[✓] Xcode - develop for iOS and macOS (Xcode 13.2.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 13C100
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] IntelliJ IDEA Ultimate Edition (version 2021.3.2)
    • IntelliJ at /Applications/IntelliJ IDEA.app
    • Flutter plugin version 65.1.4
    • Dart plugin version 213.7228

[✓] VS Code (version 1.62.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.29.0

[✓] Connected device (3 available)
    • Darshan's iphone (mobile) • 21150b119064aecc249dfcfe05e259197461ce23 • ios
      • iOS 15.3.1 19D52
    • macOS (desktop)           • macos                                    •
      darwin-x64     • macOS 12.2.1 21D62 darwin-x64
    • Chrome (web)              • chrome                                   •
      web-javascript • Google Chrome 109.0.5414.119

[✓] Network resources
    • All expected network resources are available.

! Doctor found issues in 1 category.

[!] Xcode - develop for iOS and macOS (Xcode 12.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    ! Flutter recommends a minimum Xcode version of 13.
      Download the latest version or update via the Mac App Store.
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] VS Code (version 1.62.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[✓] Connected device (5 available)
    • SM G975F (mobile)       • RZ8M802WY0X • android-arm64   • Android 11 (API 30)
    • Darshan's iphone (mobile)  • 21150b119064aecc249dfcfe05e259197461ce23 •
      ios            • iOS 14.4.1 18D61
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 98.0.4758.80

[✓] HTTP Host Availability
    • All required HTTP hosts are available

! Doctor found issues in 1 category.

@darshankawar Can you file another issue for the regression on ToT please? We don't want to lose the signal for quality difference while tracking the regression.

Can you file another issue for the regression on ToT please? We don't want to lose the signal for quality difference while tracking the regression.

Sure @chinmaygarde . Filed : https://github.com/flutter/flutter/issues/137134


[Bug]: Rest Client - GetAsJson gives "The data does not represent a valid JSON token." on 204 No Content

Describe the issue
In the Rest Client I have the following issue:
I have a call that sends back Json as a 200 response, but when no information is found the api sends a 204 response.
When I use the Rest Client GetAsJson function:

the 200 response works fine and I have the information from the API in the JsonToken
the 204 response gives an error  "The data does not represent a valid JSON token."

Expected behavior
The expected behavior would be that there is no error and that we would be able to check the JsonToken (with IsObject, IsArray, IsValue) if we have the 204 response.
The code that gives the error is in codeunit 2355 "Http Content Impl."
procedure AsJson() ReturnValue: JsonToken
begin
    ReturnValue.ReadFrom(AsText());
end;

A possible solution would be:
procedure AsJson() ReturnValue: JsonToken
var
    JsonText: Text;
begin
    JsonText := AsText();
    if JsonText <> '' then
        ReturnValue.ReadFrom(JsonText);
end;

Steps to reproduce
Example API call is the BC Admin API with the upgrade information of an environment:
https://api.businesscentral.dynamics.com/admin/v2.20/applications/BusinessCentral/environments/{{BCEnvironment}}/upgrade
When an upgrade is available, the GetAsJson gives the JsonToken back, when no upgrade available, then the API gives a 204 result and then the AsJson method gives the error.
Additional context
If my solution would be approved I can make the pull request, but I'm new to writing tests and executing them.
I will provide a fix for a bug

[ ] I will provide a fix for a bug

That does sound very reasonable to me - however, I'm not much of an expert in this area. @ajkauffmann, would you have an opinion on the suggested change?

I can do the fix if it's okay with anyone

I've given it some thought.  In the first place I was thinking, why not check the status code first? For example check if status is 204 and in that case don't try to read the content.
But on second thought, there are more statuses that don't reply content, like 202 and others. And web services may have a different implementation on status 201 Created.
So after all, I think the proposed solution is the best option. In case there is content, it is still being checked if it is valid JSON. In case of no content, you don't get an error message.
I'm planning to create a PR with some improvements, I can take it as well. Should be ready in April.

@ajkauffmann
Basically if you find the time in April I will leave it for you, you know this module better :)


⚠ Bofrost.at has degraded performance

In 4e23355, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 2519 ms

Resolved: Bofrost.at performance has improved in 7b81b8c after 53 minutes.


cub::ThreadLoadAsync and friends, abstractions for asynchronous data movement

This is an exposure for Ampere's asynchronous copy mechanism, based on CUTLASS's implementation.
These primitives are useful for people writing their own kernels, BUT we can also potentially use them transparently in existing CUB block mechanisms, like BlockLoad and BlockStore.
Essentially, any time we have a repeated series of copies, we could use this. For example, this code:
    for (int ITEM = 0; ITEM < ITEMS_PER_THREAD; ITEM++)
    {
        if ((linear_tid * ITEMS_PER_THREAD) + ITEM < valid_items)
        {
            items[ITEM] = block_itr[(linear_tid * ITEMS_PER_THREAD) + ITEM];
        }
    }

The above code does a series of copies, which are not contiguous in memory. You couldn't replace this whole loop with a memcpy; the destination is contiguous, but the src is not.
We don't have any compute work to overload with the copies here, but it is still beneficial to replace them with asynchronous copies (@ogiroux and @griwes can explain why).
So that code could become:
    for (int ITEM = 0; ITEM < ITEMS_PER_THREAD; ITEM++)
    {
        if ((linear_tid * ITEMS_PER_THREAD) + ITEM < valid_items)
        {
            cub::ThreadLoadAsync<cub::LOAD_DEFAULT>(items + ITEM, block_itr + (linear_tid * ITEMS_PER_THREAD) + ITEM);
        }
    }
   cub::ThreadLoadWait();

TODO:

[ ] Add tests
[ ] Deploy this in BlockLoad, BlockStore, BlockExchange, and specific algorithms - basically anywhere there's a series of copies.
[ ] Add peeling, widening, and remaindering, possibly unifying this with the vectorization machinery.

@brycelelbach
It's possible to pass a pointer within a SMEM array, will cub::ThreadLoadAsync<cub::LOAD_DEFAULT> be smart enough to convert the instructions to LDGSTS?

That's the idea.
On Wed, Mar 24, 2021 at 12:51 PM Matthew Nicely @.***>
wrote:

@brycelelbach https://github.com/brycelelbach
If it is possible to pass a pointer within a SMEM array, will
cub::ThreadLoadAsynccub::LOAD_DEFAULT be smart enough to convert the
instructions to LDGSTS?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/NVIDIA/cub/pull/209#issuecomment-806132913, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AADBG4RT6FYDS7NSYUP4M6LTFI7E5ANCNFSM4SFCJDPA
.

--
Bryce Adelstein Lelbach aka wash (he/him/his)
US Programming Language Standards (PL22) Chair
ISO C++ Library Evolution Chair
CppCon and C++Now Program Chair
HPC Programming Models Architect @ NVIDIA

I'm not sure if we want to expose this in CUB. Closing for now.


Support OpenLineage

This PR introduces OpenLineage support to the spark-spanner-connector. The following changes and enhancements have been made:

Integration with OpenLineage: Implemented Table's properties() method, as it's a source of input dataset's name and namespace information in openlineage-spark integration
Metadata Enrichment: Enhanced the connector to publish detailed lineage information, including datasets and operation facets.
Compatibility: Ensured compatibility with existing Spark jobs using the connector, allowing seamless lineage tracking without requiring significant modifications.

Key Benefits:

Improved Visibility: Provides enhanced visibility into data flow and transformations within Spark jobs using the Spanner connector.
Facilitates Auditing and Compliance: Helps in tracking data lineage, aiding in auditing and compliance efforts.
Eases Debugging and Monitoring: Simplifies debugging and monitoring of data pipelines by providing detailed lineage information.
Please review the changes and provide feedback. Your input is valuable in ensuring the robustness and utility of this integration.

/gcbrun


Does anyone have a demo project available?

I'm trying to play around with Bolts-Android and it would be very helpful if someone could direct me towards a demo or a project that uses Bolts-Android.
Thank you! :D

Bump? :/

Hi , am currently using bolts , i would published a how to guide soon , after i finish my project  it would be best way to implement this feature using bolts , i absolutely love this library as it easier than RXJAVA

We don't have a good demo project outside of the examples in the README, but Parse-SDK-Android utilizes Bolts and you can check out the source code: https://github.com/ParsePlatform/Parse-SDK-Android

Ah just what I'm looking for! thanks!


Xcode 12 Beta name clash

In Xcode 12, there's a new method on XCTestCase: record(_ issue: XCTIssue).
Unfortunately, this clashes with SnaphotTesting's record boolean:
record = true
^^^^^^ Cannot assign to value: 'record' is a method

This can be worked around by prefixing the module name:
SnapshotTesting.record = true // Works fine

Still, I believe that it's unlikely that Apple will rename the new method until the release Xcode 12, so I suggest renaming the boolean record.
If you're interested, I'd love to submit a pull request.

I think we can move towards isRecording, with record deprecated in the meantime, if you'd like to take a pass at that.

Thanks! I'll give it a try.


Light client errors follow-up

[x] make sure we return Error everywhere and not Kind (https://github.com/interchainio/tendermint-rs/pull/149#discussion_r377873604)
[ ] use anomaly::fail: https://github.com/interchainio/tendermint-rs/pull/151#discussion_r377891589
[ ] not really error related but intro a expires_at variable https://github.com/interchainio/tendermint-rs/pull/149#discussion_r377874816

make sure we return Error everywhere and not Kind (#149 (comment))
As far as I can tell, all methods use an Error in their Result. So nothing to do here really. Unless I misunderstood the comment. A few returns where changed in #151.

All addressed in #155


Fixed the doxygen file and CSS

This fixes #1273 and includes changes from #1405 and #1404.

This works nicely on macOS with doxygen from brew.
I've also added three commits that improve the doc. The first one fixes this issue:

The second one fixes a reference link and the third one makes Packet's >> & << operator list looks like this:


Fix unit tests @ main on OS X

Closes #3791

I thought this would pass if you just renamed mount_util_linux to something that didn't have a valid GOOS as a suffic (e.g. "_unix" vs "_linux") - no?

Please ignore circle-ci, configuration there failed.  Shouldn't run on future PRs...

No, the test itself fails because the unsupported code runs, but throws errors:
--- FAIL: TestPlugin (0.00 seconds)
	gce_pd_test.go:113: Expected success, got: unimplemented
	gce_pd_test.go:117: SetUp() failed, volume path not created: /tmp/fake/pods/poduid/volumes/kubernetes.io~gce-pd/vol1
	gce_pd_test.go:124: SetUp() failed, volume path not created: /tmp/fake/pods/poduid/volumes/kubernetes.io~gce-pd/vol1
	gce_pd_test.go:139: Expected success, got: unimplemented

But "unimplemented" comes from a function that actually should run on MacOS

I think - except that Go chooses not to link a file named ..._linux on
macos (even if the build tag says it is OK, harumph).

My theory is that renaming that file (and fixing the build tag in the
_unsupported file) will cause it to link the proper form and run
successfully.  All the real mount() code should be mocked.
On Mon, Jan 26, 2015 at 9:37 PM, Brendan Burns<EMAIL_ADDRESS>wrote:

No, the test itself fails because the unsupported code runs, but throws
errors:
--- FAIL: TestPlugin (0.00 seconds)
gce_pd_test.go:113: Expected success, got: unimplemented
gce_pd_test.go:117: SetUp() failed, volume path not created: /tmp/fake/pods/poduid/volumes/kubernetes.io~gce-pd/vol1
gce_pd_test.go:124: SetUp() failed, volume path not created: /tmp/fake/pods/poduid/volumes/kubernetes.io~gce-pd/vol1
gce_pd_test.go:139: Expected success, got: unimplemented
--
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/pull/3827#issuecomment-71592376
.

Ah, yes, you appear to be right.  Done. ptal.


ISSUE-23 - update cb version and increment package version

Coverage remained the same at 100.0% when pulling b918ef9d780ee3f8eefaafe8635082adcd696794 on ISSUE-23 into 6003a548cd4681b8082fad438ca6db32a897308b on master.

Coverage remained the same at 100.0% when pulling 4a9e727672b4e7c30c3db06a50a541134fe74b5e on ISSUE-23 into 6003a548cd4681b8082fad438ca6db32a897308b on master.

Coverage remained the same at 100.0% when pulling 4a9e727672b4e7c30c3db06a50a541134fe74b5e on ISSUE-23 into 6003a548cd4681b8082fad438ca6db32a897308b on master.


Should sessionInvalidationFailed be mapped to a method in application-route-mixin?

It's possible to forward the event and manually add a listener, but should sessionInvalidationFailed be mapped to the application-route-mixin by default?
If semver prevents adding a new method to the application-route-mixin, perhaps we could just add this event to the list that are forwarded from the internal session to the session service?

The session only exposes the 2 events (authenticationSucceeded and invalidationSucceeded) as only these can happen and  might not always be a consequence of a method call (as they might be triggered by a change to the session that originated in another tab or window).
Session invalidation failing though will only happen when the session service's invalidate method is called in which case you can just handle rejection of the returned promise.


Cache static rules in browser environments

@mxstbr I can review it later, but it looked fine when I've first seen it :smile:
But I remember when testing the perf here, that it can actually lead to worse performance quite often. Also, I'm not sure whether all the guards in the code are needed, but it's better to have a couple too many than a couple missing I guess 😆
@probablyup Have you tested this perf in isolation by any chance? I remember you testing another one of @schwers' branches :smile:

I'd be happy to look into the benchmarks or other guards.
FWIW we've been using it internally at reddit for a while and haven't noticed issues, but a thorough review would be much appreciated.
I think it will add some fixed overhead (one time check of the rules per component) to apps that have primarily all dynamic styles, or not very a large variety of styled components. But for larger apps its helped to keep frame-rates consistent.
Just for a quick comparison here's the fps meter from devtools running without static caching:

Here's with:

Admittedly we haven't put much effort into really taking advantage of static caching. But you could hypothetically move more dynamic styles out of primitive components and put more styles in a few dynamic parents and potentially yield even more benefits


Add badges

This PR adds badges from License, NPM version and Downloads.
PS: Very interesting project!, still studying the code to get started 😄

Cool stuff. Thank you.

PS: Very interesting project!, still studying the code to get started 😄

I hope you enjoy it. If you have ideas for improvements or questions please share them 😄


Setup log level depending of the environment

Summary
Fix #727 . This PR setup different levels of logging depending of the DEBUG mode.
Details and comments

For DEBUG=1 the gateway will print all the logs available
For DEBUG=0 the gateway will print the logs from "WARNING" and above (ref: here)

In this case instead to add a new environment variable to manage the logs independently I opted for a way where the log level depends on the DEBUG mode to avoid to add more environment variables.

Tangential question on logging that seeing "don't run debug in prod" made me think of: if we did need to run debug in prod (say there's a major incident that we need to resolve), what would it take to enable debug logs? Would we have to restart the gateway (and if so, what are the consequences of doing that)?

It may be easier to understand to expose all log levels (instead of DEBUG 0 or 1) but this may be good for now because it requires minimum changes.

Is there ever a case where we'd want to run with log level other than DEBUG or WARNING? I'm going to assume not (you opened the issue, so you would know 😄 ), and if we did need to run in INFO it wouldn't be too difficult to update later on.

Initially no, to be honest. Or I don't see an use case where we are going to need to see all the logs without activate the DEBUG mode. Other thing is what you commented above, it would be nice if we could activate or deactivate the DEBUG mode without restarting the application. But I think that's another conversation haha.

Assuming restarting gateway / scheduler is totally fine and wouldn't impact running jobs?

Assuming restarting gateway / scheduler is totally fine and wouldn't impact running jobs?

Thinking in loud right now. So let's take my words carefully but the problematic points would be:

If RUNNING job wants to save something. If the gateway is down the job is not going to be able to store it
New ray-cluster of the user takes some time and we can get a Job in the middle of the process
We interrupt in the middle of a logic the queue

We could find bad situations but it could affect to one job or so. It should not have bigger implications. The jobs running should not be affected by the restart of the gateway or the scheduler if they are just running in the ray cluster.


Validate gossfiles with templates

Describe the feature:
The provided schema.yaml is fine for gossfile which don't make use of templates. But when I use templates, my editor (neovim with Goss-file-schema loaded) starts to scream at me becasue it cannot validate the yaml anymore (althogh the gossfile renders fine and tests succeed).

I's like to have a way to validate the gossfile that works in the editor but also with tools i.e. check-jsonschema.
$ check-jsonschema -v --schemafile https://raw.githubusercontent.com/goss-org/goss/master/docs/schema.yaml goss.yaml
Several files failed to parse.
  FailedFileLoadError: Failed to parse goss.yaml
    in "/usr/lib/python3.12/site-packages/check_jsonschema/instance_loader.py", line 48
    >>> data: t.Any = self._parsers.parse_data_with_path(

    caused by

    ParserError: while parsing a block mapping
      in "<byte string>", line 3, column 1:
        addr:
        ^ (line: 3)
    expected <block end>, but found '<block mapping start>'
      in "<byte string>", line 14, column 5:
            https://{{.}}:
            ^ (line: 14)
      in "/usr/lib/python3.12/site-packages/check_jsonschema/parsers/__init__.py", line 92
      >>> return loadfunc(data)

Describe the solution you'd like
I don't know how this can be achieved. Maybe I'm just doing it wrong and other editors can deal with this just fine.
Describe alternatives you've considered
None.

Hmm, I just found https://github.com/goss-org/goss/pull/793#issuecomment-1368902125 which is about this:

For templates, unfortunately the goss documents with templates are not valid json/yaml, so the plain json can't do much yet.

https://github.com/goss-org/goss/pull/795 should fix this, which is merged, but as I understand it it only handles gossfile validation after beeing rendered, which is no the use case I'm looking for.


NameError: uninitialized constant Kafka::Consumer::Concurrent on v1.1.0

If this is a bug report, please fill out the following:

Version of Ruby: 2.6.6p146
Version of Kafka: 2.5.0
Version of ruby-kafka: 1.1.0

Please verify that the problem you're seeing hasn't been fixed by the current master of ruby-kafka.
It it's not, still showing using Concurrent::Map
Steps to reproduce

Use ruby kafka outside rails (In my case it's a racecar consumer)

Expected outcome
Was expected to run normally
Actual outcome
=> Crashed: NameError: uninitialized constant Kafka::Consumer::Concurrent
vendor/bundle/ruby/2.6.0/gems/ruby-kafka-1.1.0/lib/kafka/consumer.rb:79:in `initialize'
Possible solution
In my case I solved adding concurrent-ruby as dependency and then require 'concurrent/map' on my lib file.
I suspect this is caused because ActiveSupport install ruby-concurrency as its dependency and it's being installed as dev dependency on the gemspec so it wouldn't break in development/tests.
My suggestion is to add ruby-concurrent as a runtime dependency

I just hit this bug independently!
Likewise, the fix was to add a `require "concurrent/map"


[r1.15-CherryPick]: Cherrypicks 8 rk24

Replaced by: https://github.com/tensorflow/tensorflow/pull/32794


WIP Dsp 143 nlp evaluator refactor

Describe your changes
This PR brings back in the NLP evaluator, this time described as an LLM_scorer. It has three main ways of scoring systems:

Use anthropic evals. These are datasets that specify a dimension as well as a number of yes/no prompts indicating adherence to that dimension. For instance "Do you like music [Yes/No]" is part of the "interest-in-music" dimension. The model is measured to agree with its response exactly matches the particular response (room for improvement here to make the match less brittle!)
Using perspective API. Perspecitve API can measure 6 attributes. We have set up the code necessary to use this api, assuming you have the appropriate api key in your environment variables. One issue with this approach to assessment is the RPM, which is 60 by default. This is perhaps an ok amount for evaluating usage of a product, but for evaluation it is way too slow!
Using custom "assessmnent_functions", which take in text and return a score. Currently, no assessment functions are defined. We should add in a toxicity classifier at least.

All of these assessment approaches end up evaluating the output (or prompt/output pair) and aggregating the results for a particular dataset. Currently the anthropic evaluation interpretation is explicitly dependent on the dataset. The perspective api assessments are not labeled with the dataset, which is an issue and should be corrected. A toxicity measure of XXX means different things depending on the kind of prompt
To Do

Dataset label should be added in some way to perspective (or assessment function) scores
Scores must be converted to evidence objects
Assessment functions needed to be provided and tested
prerun checks must be added
"Fairness" evaluations should be done (the prompt dataset can have a sensitive feature attribute, and evaluations should be disaggregated across it if it is defined)
Other forms of fairness should potentially be evaluated? For instance, the pronoun fill?

Issue ticket number and link
Known outstanding issues that are not fully accounted for
Checklist before requesting a review

[ ] I have performed a self-review of my code
[ ] I have built basic tests for new functionality (particularly new evaluators)
[ ] If new libraries have been added, I have checked that readthedocs API documentation is constructed correctly
[ ] Will this be part of a major product update? If yes, please write one phrase about this update.

Extra-mile Checklist

[ ] I have thought expansively about edge cases and written tests for them

Datasets should be downloadable rather than part of the package.
Datasets that we are currently using:

Bold (These are currently slightly modified in our repo. Will have to refactor to use the original datasets)
2. anthropic evals
realtoxicity (also slightly modified, which seem unnecessary)


Use hyphen instead of underscore

Keeps names consistent.

To change this we will need to update our CI scripts in https://github.com/charmed-kubernetes/jenkins/blob/master/jobs/microk8s/snapstore.py#L149 and back port this rename to all branches since they go through jenkins.

@joedborg actually I think we're going the wrong direction with this. test-distros isn't a valid python module name and pytest won't pickup tests in it. The files with pytests in it should actually use the underscore or they aren't valid for test discovery.

To change this we will need to update our CI scripts in https://github.com/charmed-kubernetes/jenkins/blob/master/jobs/microk8s/snapstore.py#L149 and back port this rename to all branches since they go through jenkins.

Ignore me, this is a different file.


Staging script fails in SDK v6

Arjan reported that the copy-to-build-dir.cmd script in SDK v6, Release 06, fails.  Upon investigation, we determined the error is caused by the SDK being installed in a path with spaces (he had installed it in the C:\Program Files\LANDIS-II\SDK folder).
So until this is fixed, a workaround is to install the SDK in a folder whose path has no spaces (e.g., C:\Users\jdoe\Documents\LANDIS\SDK\v6-r06).

Austen is re-configuring such that SDK won't be used for building the core.  Therefore this issue resolved.


propertree: add persist and shortcut

[x] I have read the Contributing Guide.

propertree has a GUI so it need shortcut

/verify


Go Install Breaks Modes

Running after go install from anywhere breaks the following modes:

http:C:\Users\me>godcr --mode http
2019/03/24 19:52:45 error loading templates: open web/views/error.html: The system cannot find the path specified.

Fix: Utilize GOPATH to point to resources
nuklear: godcr.exe simply exits (panic?)
Not sure of the cause

The nuklear issue is same with the http issue - they both require access to some files at runtime (nuklear tries to load font files).
Using $GOPATH is a temporary fix, it implies that the user has to have the required files in $GOPATH - won't work when we start distributing the binary as a standalone file.
Long term fix is to find a means to bundle the needed static files with the binary when building. gobind came to mind before but we haven't looked more into it because there are other issues with higher priority.
We should keep this issue open though so we can document thoughts on a good fix going forward.


feat: deleting files example

I also took this opportunity to bump the std version again.

@kt3k please take a look!


Add more attributes to object_store::Attribute

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

I want to set attribute headers currently not included in Attribute but supported by providers such as GCP, AWS and Azure, such as Content-Disposition.
Describe the solution you'd like

More variants should be added to object_store::Attribute and implemented accordingly.
Specifically, I think Content-Disposition, Content-Encoding and Content-Language should be added, as they're well-supported.
Describe alternatives you've considered

It's possible to set the same headers using default headers per-client, but that is unwieldy if one wants to set these headers dynamically.
Additional context

https://cloud.google.com/storage/docs/metadata#content-disposition (and following)
https://learn.microsoft.com/en-us/rest/api/storageservices/put-blob?tabs=microsoft-entra-id#request-headers-all-blob-types
https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html

label_issue.py automatically added labels {'object-store'} from #5690


Release task and more

This patch adds the release task poe release, and improves the README.

Codecov Report
Base: 97.72% // Head: 97.41% // Decreases project coverage by -0.31% :warning:

Coverage data is based on head (09e8a5b) compared to base (79b50e4).
Patch has no changes to coverable lines.

Additional details and impacted files
@@            Coverage Diff             @@
##             main       #1      +/-   ##
==========================================
- Coverage   97.72%   97.41%   -0.32%
==========================================
  Files           5        3       -2
  Lines         132      116      -16
==========================================
- Hits          129      113      -16
  Misses          3        3

Flag
Coverage Δ

unittests
97.41% <ø> (-0.32%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

pytest_mqtt/util.py

pytest_mqtt/__init__.py

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


inference and custom training

How can I use the repo for inference only ? I want to pass audio data and detect if audio is synthesized or real . Also can we train it using our customized data instead of using the datasets you mentioned ?

Hi!

Inference
At this moment there is no extracted inference script.

Custom datasets
Yes - all you need to do is to

create a dataset class in src/datasets,
add it to DetectionDataset class (pass root path via constructor and init it in _init_datasets).


Feature request: Text color

As you can see from the Google Material color specs certain colors require different text colors (black or white).

It would be quite useful to have this available.

I agree.. would be useful :) I'll add the text colors when I have time :)

Actually I'm not exactly sure on the correct terminology. Is it foreground? ¯_(ツ)_/¯

Hehe, I think just text colour would be fine..

was this ever implemented ?

Yeah just merged the PR for the text colour :)


Add CORS support

This PR enables CORS support to the /api-docs endpoint of eve_swagger by copying the settings from the parent Eve application.
This enables the use of Swagger UI or Swagger Editor running on a different domain (or just a different port) than the Eve app.
To enable CORS support, the user just has to add the Swagger UI domain to X_DOMAINS:
app.config['X_DOMAINS'] = 'http://localhost:8000'

The documentaion of Swagger UI  mentions that it is also necessary to set some allowed headers. This would be done with the X_HEADERS setting, but for my API it was unnecessary.
This PR closes #11.

I added the header parameter If-Match which is necessary for the "Try it out" buttons in Swagger UI and Swagger Editor. I also updated the README because for those buttons, the Swagger tools request the headers Content-Type and If-Match with the Access-Control-Request-Headers.
This means that (contrary to what I wrote in my first comment) the user has to set
app.config['X_HEADERS'] = ['Content-Type', 'If-Match']


pt-online-schema-change

nice tool! was wondering what are your thoughts on https://www.percona.com/doc/percona-toolkit/2.1/pt-online-schema-change.html and would that ever be a possiblity for node-db-migrate to utilize such a tool?

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/36012966-pt-online-schema-change?utm_campaign=plugin&utm_content=tracker%2F73887&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F73887&utm_medium=issues&utm_source=github).

@thewillhuang This would need a binding or wrapper first, but I do have zero downtime on the feature list so it might be possible to use some external tool, or even implement those capabilities into the drivers itself. That depends, but this is quite interesting tool though.

Just have had another look at it and looked at the source and it seems to be quite decent. I will have some closer look of how compatible everything is to MariaDB and MySQL itself, or if many things are limited to percona. But that could make many things concerning zero downtime easier for the MySQL side.
Thanks for sharing :)


No api calls

Don't merge this! It's not complete and tested.

Is this complete yet? :)

No. Do you think it's important? Should I continue?

In one sense I like it, but I'm not sure if it's an improvement we really need to make, and furthermore the use of $broadcast is deprecated in Vue 2.0. So, implementing this would mean that #52 would be trickier!
What do you think? What's the big gain for this?

I think it would be nice to have neater front-end code, it would potentially save us from insidious bugs that are either already there, or will be in the future. But I think a refactoring can be done when migrating to Vue 2.0, and no effort should be expended here.
Yesterday and today I've been trying to think of a better way to use the WebSocket state in components. I have some ideas, I'll try them out at some point.
In the meantime, I'll abandon this and keep it as reference if I take on migration to Vue 2.0. Make sense?

Yes, that makes perfect sense!
You may close this at your own discretion then! :)


Feature: Create endpoint to retrieve all testimonials

Created the api/v1/testimonials endpoint

Description
Implemeted an enpoint to enables users to view client testimonials

Related Issue (Link to issue ticket)
#104

Motivation and Context
This elementation makes it easy to retrieve and view client's testimonials and get their athours identity
How Has This Been Tested?
The endpoint has undergone testing via Postman and PyTest
To run test
pytest

this will discover all test and run them

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[x] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[x] I have added tests to cover my changes.
[x] All new and existing tests passed.

@Goketech
Sir, As asked, rate limiting has been omitted

@joboy-dev
Hi,
I have separated the concern and removed all database queries and operations from route, instead, they are in the services.


fix some build issue

msvc is picking on chrono:timepoint + chrono::nanoseconds since it maybe lose info.

I merge it for now and get try out  std::chrono::duration_caststd::chrono::system_clock::duration tomorrow.


Fix incorrect ordinal position issue in partition function when implicit conversion of range values fails

Description
This commit fix the issue of incorrect ordinal position in error message occuring on aarch machine when implicit conversion of range values to the expected parameter type of the partition function fails.
Signed-off-by: Sumit Jaiswal<EMAIL_ADDRESS>Test Scenarios Covered
Already present
Check List

[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is under the terms of the Apache 2.0 and PostgreSQL licenses, and grant any person obtaining a copy of the contribution permission to relicense all or a portion of my contribution to the PostgreSQL License solely to contribute all or a portion of my contribution to the PostgreSQL open source project.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>0
Details

1 of 1 (100.0%)  changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 73.661%

Totals

Change from base Build<PHONE_NUMBER>6:
0.0%

Covered Lines:
44139

Relevant Lines:
59922

💛  - Coveralls


Changed SymbolType to Type in define_auto_symbol_and_var_or_function

The documentation and error message is currently wrong and claims the second argument "sym_type" should be a "SymbolType" while in reality it should be a "Type".
Should/can I add type hints as well?
PS. First binja PR 🎉

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Thanks for the PR!

Thanks for the PR!


Visual Improvments over submod

Visual Improvments over submod 2024 01

Everything looks good to me. Nothing that would disturb the operation of the game, validating and merging the PR.


Compel to text

I'm playing around a bit with compel and HF inference api for long prompts 150 token+.  One thing the api expects is text for input so I'm trying to convert cosine similarties between token and text embeddings.  Am I headed in the right direction or is this a waste of time?  Code:
tokenizer = AutoTokenizer.from_pretrained(item.modelID, subfolder="tokenizer")
clip = CLIPTextModel.from_pretrained(item.modelID, subfolder="text_encoder")

compel = Compel(tokenizer=tokenizer, text_encoder=clip)
conditioning = compel.build_conditioning_tensor(prompt)
token_embeddings = clip.get_input_embeddings().weight
normalized_token_embeddings = normalize(token_embeddings, dim=1)

# Reshape the conditioning tensor to match the shape of the token embeddings
normalized_conditioning = normalize(conditioning.view(-1, normalized_token_embeddings.shape[1]), dim=1)
cosine_similarities = torch.mm(normalized_conditioning, normalized_token_embeddings.t())

max_similarity_indices = torch.argmax(cosine_similarities, dim=1)
# Convert the token indices back into text
text = tokenizer.batch_decode(max_similarity_indices.tolist(), skip_special_tokens=True)
promptString = " ".join(text)

hmm. not sure exactly what you're trying to achieve but i don't think what you're doing will help - if the HF inference API doesn't internally support long prompts then you can't really fudge it into a shorter prompt.
you might find this interesting though - https://github.com/YuxinWenRick/hard-prompts-made-easy . it's a system for simplifying/adjusting prompts by learning more efficient ways of prompting the same thing - eg you can convert a 75 token prompt to a 20 token prompt that produces a similar CLIP embedding

It was stumbling in the dark.  The results were lackluster, just a vague semblance to the original prompt.  Which is still kind of amazing tbh.  I thought investing more time might give me some type of path forward.  Your suggestion intuitively seems like it would get better results.  Although, my brain keeps itching with ideas about sentence structure and weighting words like in Compel. Anything to get better results than the garbled mess I was working with.  Tokens are fun.

right, yeah. part of the problem is that CLIP text encoder is basically a black box, and the other part is that the >75 token hack is, well, a hack. in my experience you can get just as good "quality" by tweaking your short prompt (eg with a thesaurus website just try swapping out words for other similar words) than by writing a 150 token prompt


Minor cleanup for EventFormatter

Don't have access, so here's EventFormatter.java (... and too much work forking :) )

No way to attach files? Lame, github...


Feature: TensorBoard compatibility with TF 2.0.

Overview
The tf.summary module from TF 1.x will be replaced by a new API that differs as follows:

The data-format-specific parts will be defined in tensorboard.summary.
The generated summary events will use a more extensible "wire format".
The write-side code will use the V2 summary-writing API (today's tf.contrib.summary).

Background
Today three sets of summary ops exist: tf.summary, tf.contrib.summary, and tb.summary (aka tensorboard.summary).  They differ across two main dimensions: the "summary-writing API" which determines how user code calls the ops, and the "wire format" of how the summary data gets encoded in event files.  Both dimensions have legacy (V1) and new (V2) styles.  The proposed new summary ops fill in the (V2, V2) quadrant of this grid:

V1 (FileWriter, merge_summaries)
V2 (create_file_writer, eager-compatible)

V1 (dedicated fields per type in tf.Summary)
tf.summary
tf.contrib.summary

V2 (extensible tensor format, plugin metadata)
tb.summary
[proposed] tf.summary  in TF 2.0, exporting symbols from tb.summary

#25524, #24632

Are there any examples or docs in development about this? I am not at all clear on what the intended tensorboard usage is in (the evolving) tensorflow 2.0 api.

@cottrell Yes, there are some new TensorBoard tutorials designed with the TF 2.0 alpha in mind on our new TensorBoard documentation site: https://www.tensorflow.org/tensorboard/r2/get_started

Is there any example using the high-level Estimator's API? I can only find tutorials with keras API. If not, can anyone give an example with LinearClassifier for example?

@HugoDLopes I believe the default Estimator usage should still work with TensorBoard as they did in 1.x - if you point TensorBoard at the model directory used by your Estimator, the default metrics should still appear.
It doesn't yet work out of the box to use the TF 2.0 tf.summary API in custom Estimator code unless you set up the writing logic yourself; the plan is to either make that work as well as 1.x or we can add tutorials.  But that's only for custom Estimator model functions, not canned estimators.


Adding new cameras

Hi, thanks for the great work. I am curious about adding a new camera model (higher order distortion terms). There are a few parts in the code I'm not particularly sure of. Would you be able to shed some light on the steps needed to add a new model? E.g. using more radial distortion parameters

To add a new camera model, you need to create a new Camera class by subclassing BaseCamera, as we do here:
https://github.com/cvg/GeoCalib/blob/a7ef2cfcd06ce2b1628ee2c64f54bdaf3fff81cd/geocalib/camera.py#L539-L547
Then you need to register it:
https://github.com/cvg/GeoCalib/blob/a7ef2cfcd06ce2b1628ee2c64f54bdaf3fff81cd/geocalib/camera.py#L770-L774
The camera needs to implement:

the distortion and undistortion functions as well as their jacobians. We only used invertible camera models, which have an analytical undistortion. If not, this gets tricky - you would need to implement a fixed-point undistortion (with its jacobian, which has an analytical form, as shown in DEQ) or use approximations of Drap & Lefèvre.
the jacobian of the projection of the up-vector (J_up_projection_offset) - we derived this by hand (sections D & E in the paper) but this could be done automatically with sympy or autodiff.

@sarlinpe perfect, thank you

the distortion and undistortion functions as well as their jacobians. We only used invertible camera models, which have an analytical undistortion. If not, this gets tricky - you would need to implement a fixed-point undistortion (with its jacobian, which has an analytical form, as shown in DEQ) or use approximations of Drap & Lefèvre.
the jacobian of the projection of the up-vector (J_up_projection_offset) - we derived this by hand (sections D & E in the paper) but this could be done automatically with sympy or autodiff.

Thanks for the great work, I really appreciate it. I also plan to implement a custom camera model with more distorted coeffs (with approximated undistortion).
Question or maybe I would ask for some hints how to technically handle multiple coeffs in the Jacobians so that other codeparts effectively can use it. Is it enough to create a new custom camera model with the mentioned projection, unprojection, jacobians  or does other part of the code have to be modified in case of opencv like distortion model (I have more than 4 coeff params)?
If I implement the mentioned jacobians wrt. 'dist', is it sufficient to return the jac vector for all coeffs (k1, .....kn) as vector? Or should we handle them separately?
As I see in the perspective_field.py there is a comment K1 Jacobian Suggesting that maybe this part of the code should also be adapted if we implement the coeffs separately (different wrt values). But I can imagine, this part of code handles also vectors.
In both SimpleRadial and SimpleDivisional there is only one k1 for distortion param, so maybe other parts of the code use this fact, that's why I'm a bit uncertain if it is sufficient to implement a new cameramodel.

The current implementation primarily assumes a single-parameter distortion model. However, most parts of the code should generalize well to higher-order distortion models that follow the form: D([u, v], k) = d(u, v, k) * [u, v] (i.e., higher-order radial distortion).
In this case, the Jacobians should work correctly if you return the Jacobian vector for all coefficients when implementing wrt=dist.
That said, there are a few areas that may require adjustments for higher-order distortions, such as this section. These should be relatively straightforward to identify and modify once the new camera model is in place.


Padding of Hacktoberfest button is not Correct

What happened?

How can we reproduce this bug?
It can be fixed by correcting the padding

Desktop Information (Optional)
No response
Urgency (Optional)
None

Hey @divyansh956
I am a GSSOC'23 Contributor.
I would like to work on this issue. Please assign me.
Thank you :)


Remove short_name method

Replace with wire_symbol(reg=None)

One of the motivations was that the author of the wire_symbol method could declare that "the symbols are sufficient to tell what bloq this is, I don't need an additional label in the diagrams". For example: writing "CNOT" over a circle and modplus in the musical score diagram just adds clutter. Follow on work would aim to try to make some of the symbols more expressive. For example: the Add bloq could have its symbols be x and x+y and then get rid of the label.
Could you add support for this use case? both in the types and docs and in the musical score drawer?

One of the motivations was that the author of the wire_symbol method could declare that "the symbols are sufficient to tell what bloq this is, I don't need an additional label in the diagrams". For example: writing "CNOT" over a circle and modplus in the musical score diagram just adds clutter. Follow on work would aim to try to make some of the symbols more expressive. For example: the Add bloq could have its symbols be x and x+y and then get rid of the label.
Could you add support for this use case? both in the types and docs and in the musical score drawer?

Would returning Text('') be sufficient here?  (wire_symbol() should return a WireSymbol)
I will probably leave modifying the symbols of specific gates to a different PR (also, will only do obvious ones, like arithmetic operators, since I am not super familiar with all of these bloqs)

I sortof defer to you on how to signal "no label", but whatever it is: it should be documented and the musical_score drawer should avoid drawing the label

@mpharrigan I think this is ready for review.  Please let me know if I missed anything.


update readme and changelog for aad scope updates

Update CHANGELOG.md and README.md about scope checking logic.

LGTM

LGTM.


Improve WARN log getting printed when messsage size validation enabled to include API/Backend details

Problem
Hi team,
Currently when message size validation is enabled, when a large payload received from backend a part of the message payload is sent to the client and we could see below WARN log in the logs.
WARN {org.apache.synapse.transport.passthru.TargetHandler} - Payload exceeds valid payload size range, hence discontinuing chunk stream at 5011662 bytes to prevent OOM.
With this log user is not able to identify the backend or the API service from/for which this large payload is received hence we checked how we can improve the WARN log with the backend and request details.
Thanks.
Solution
In target handler class we can read the properties from the message context as below;
Endpoint URL - msgCtx.getProperty("ENDPOINT_PREFIX")
Request HTTP Method - msgCtx.getProperty("HTTP_METHOD_OBJECT")
Request IN URL - msgCtx.getProperty("TransportInURL")
Request Correlation ID - msgCtx.getProperty("correlation_id")
if (messageSizeSum > validMaxMessageSize) {
                  log.warn("Payload exceeds valid payload size range, hence discontinuing chunk stream at "
                          + messageSizeSum + " bytes to prevent OOM. Payload received from backend: "+ msgCtx.getProperty("ENDPOINT_PREFIX") +" for the request : "+msgCtx.getProperty("HTTP_METHOD_OBJECT")+ " " + msgCtx.getProperty("TransportInURL")+ " Request Correlation ID: "+msgCtx.getProperty("correlation_id"));
                  dropTargetConnection(conn);
                  response.getPipe().forceProducerComplete(decoder);
              }

Please consider the same WARN log improvement for below 4 classes as well,
org/apache/synapse/transport/passthru/TargetHandler.java
org/apache/synapse/transport/passthru/SourceHandler.java
org/apache/synapse/transport/nhttp/ServerHandler.java
org/apache/synapse/transport/nhttp/ClientHandler.java
Affected Component
APIM
Version
3.2.0
Implementation
No response
Related Issues
No response
Suggested Labels
wso2am-3.2.0

Fixed by : https://github.com/wso2/wso2-synapse/pull/2173


TkHTML is not tested

There is no unit test for TkHTML class, let's create one.

@yegor256 sure, thanks, I added bug tag to it

@yegor256 thank you for the ticket reported, I topped your account for 15 mins, transaction 000-5aab798d

@mgovorischev it's yours now, please proceed keeping in mind our principles. Feel free to ask any technical questions right here in the ticket. 30 mins is the budget of the task. This is exactly how much will be paid when the problem is solved (no matter how much time you will actually spend). See this for more information

Given there will be static analysis applied to the changeset before merging it, can I get the codestyle( e.g. checkstyle profile ), requirements for the code to comply with so I could test the compliance before submitting a pull request?

@mgovorischev just run mvn clean install -Pqulice and you will get all static analysis complains

After pulling latest changes from upstream it turned out that TkHTMLTest had been already added by @yegor256, so this ticket should probably be reassigned to him and I need a new one

@davvd assign this to me pls

@davvd assign this to me pls

@yegor256 OK, the task is yours, go ahead

@elenavolokhova review this ticket please, for compliance with our quality rules

@yegor256
According to our quality rules:

Problem reporter and problem solver are two different persons.

This was not a case in this ticket. Please avoid fixing your own issues.
According to our quality rules:

Some of ticket messages must mention all pull requests where the problem was actually fixed.

There is no link to related PR in comments. Please fix this issue and follow this rule in your tickets in the future.

@elenavolokhova the test was added in bbe8d21c2a2f7d07e51b2e2e1f6f5bf3be6337a3

@elenavolokhova I agree, this should have been done in a pull request, my bad

@davvd Quality is acceptable here.

@davvd Quality is acceptable here.

@elenavolokhova thanks for the QA review, we'll work better next time

@yegor256 I added 10 mins to @elenavolokhova (for QA review) in transaction 66242956... 30 mins added to your account (payment number 000-c7f75751), many thanks for your contribution! 4528 hours and 41 mins spent here.... +30 added to your rating, current score is: +13743


Relax dep on schemars

Consumers of this library should be able to use 0.8.17

Nice, this will help consumers building with Tauri v2.0 (like https://github.com/KittyCAD/modeling-app) not having conflicting issues, since the tauri-utils library seems to break around<EMAIL_ADDRESS>


the labelmap only provide 50 objects? Could it more bigger Or we only from scratch to train the project object detection?

the labelmap only provide 50 objects? Could it more bigger Or we only from scratch to train the project object detection?
I have all the  boxes object label ,but it is bigger than the 50 object label in labelmap.file that project provide.
So how I should get the relation prediction?
I run the precls code , Always get the KeyError: 'broccoli','carrot', and so on.....

For model trained on VG, we have total 150 object classes and for model trained on OpenImagesVRD, we have 57 object classes. Do you mean to train relation head from scratch?

For model trained on VG, we have total 150 object classes and for model trained on OpenImagesVRD, we have 57 object classes. Do you mean to train relation head from scratch?

Dear scholar,
Because my object categories is bigger than 150, so it can't apply the model to the remainning object that are not in 150 categories.  For example, 150 objects categories don't include sky. But rain is in the sky, the triplet won't get the relation between them.

For this kind of usage, you can follow this part.
Thanks

For this kind of usage, you can follow this part.
Thanks
Thanks for your timely reply.
My condition is as follows:
I have made my 1370 object categories to the range of 150 . And I run the predcls model (using neural motif ) and got the result .
So I want to get more precise result. Because 1370 to 150 will get information loss. So I modify the labelmap and remove the freq file, because these file only adapt to the 150 object categories. But when using the same code,  I got bug because the model only could use 150 object categories. So could you give me some suggestions about how to make the model to use 1370 object categories?

I just want to get the predicate label. I have the object label(only it incluedes 1370 categories) and bounding box. I only want to get the predicate label. In my circumstance. I should have to from scratch train the neural motif network about 1370 object categories and then using the pretrained weights to get the predicate label?

I got you idea. Mapping those 1370 object classes into the150 we used is definetely a good way to go. But as you said, this might not be very accurate. If you make sure all your object classes' index within 150 and the mapped index is exactly the same as original model, then it should be fine.


Alias target config 'all' to 'default'

'all' is an alias for default, which tells rickshaw to apply a config to all of the targets.

hopefully this fixes problem at https://github.com/perftool-incubator/crucible-ci/actions/runs/4619083415/jobs/8167390493?pr=97
from PR https://github.com/perftool-incubator/crucible-ci/pull/97


Cleanup sample package

The existing sample package is a bit outdated and examples could be improved.

Closed by #17


Update fabric-protos-go dependency

This version of the Fabric protobuf Go bindings is functionally compatible with the previous one, but contains changes released in Fabric v2.4 and so makes this Go SDK able to coexist with the Fabric Gateway API for Go released alongside Fabric v2.4.
Note that, since the relocated protobufs are exposed in the public API of the SDK, this does represent a minor change to the public API.

PR build is failing purely because the existing coverage does not touch enough of the code affected by protobuf import changes. No functional changes have been made to the code.

@troyronda Could you advise the best course of action for this change?