99 datasets failed in latest run

Description
On the latest run to create the intake catalogue, using the datasets listed at https://github.com/cp4cds/c3s_34g_qc_results/blob/release3/QC_Results/QC_passed_dataset_ids_latest.txt , 99 datasets failed to run successfully. The error output is included:
errors.txt
There were 4 different kinds of error:

KeyError: "Receive multiple variables for key 'longitude': ['longitude', 'lon']. Expected only one. Please pass a list ['longitude'] instead to get all variables matching 'longitude'."

This is the issue I mentioned: https://github.com/roocs/roocs-utils/issues/61. So it looks like lon shouldn't have units=degrees_east and standard_name=longitude.

a)

Exception: Latitude is not within expected bounds. The minimum and maximum are -79.22052001953125, 9.969209968386869e+36

This is also the same for longitude in files with this error. There doesn't seem to be a fill value set for latitude/longitude but the fill value is set as 1e+20 for the main variable. These all look to be NCAR datasets.

b)

Exception: Latitude is not within expected bounds. The minimum and maximum are -79.22052001953125, 1.0000000150474662e+30

First mentioned here: https://github.com/cp4cds/c3s_34g_manifests/issues/10. Also the same for longitude. Fill value for latitude and longitude is set as 1e+20. These are SNU.SAM0-UNICON datasets and one CAS.FGOALS-g3 dataset.
There is no mention of these issues on the errata service. When I tested opening these datasets with netCDF4, for 2(a) the fill values were set as 9e+26 and the maximum of the latitude/longitude was normal, so this is only an issue with xarray.
For 2(b) when opening with netCDF4, the fill values are set as 1e+20 and the max latitude/longitude was 1e+30 as well, so this is an issue with the data.

For the HDF errors: I've checked the checksums and they all match up with whats on https://esgf-index1.ceda.ac.uk/search/cmip6-ceda/.

Investigating opening these with netCDF4:

/badc/cmip6/data/CMIP6/CMIP/AWI/AWI-CM-1-1-MR/historical/r1i1p1f1/Amon/va/gn/v20181218/va_Amon_AWI-CM-1-1-MR_historical_r1i1p1f1_gn_186601-186612.nc : Opening error with xarray and netcdf

/badc/cmip6/data/CMIP6/ScenarioMIP/MOHC/HadGEM3-GC31-LL/ssp126/r1i1p1f3/Amon/ts/gn/v20200114/ts_Amon_HadGEM3-GC31-LL_ssp126_r1i1p1f3_gn_205001-210012.nc: Just error opening with xarray - opens fine with netCDF

/badc/cmip6/data/CMIP6/ScenarioMIP/MOHC/HadGEM3-GC31-MM/ssp126/r1i1p1f3/Omon/tos/gn/v20200515/tos_Omon_HadGEM3-GC31-MM_ssp126_r1i1p1f3_gn_203001-204912.nc: Error getting values of lat/lon with xarray and netcdf

/badc/cmip6/data/CMIP6/CMIP/MOHC/UKESM1-0-LL/historical/r1i1p1f2/Amon/tas/gn/v20190406/tas_Amon_UKESM1-0-LL_historical_r1i1p1f2_gn_185001-194912.nc: Opening error with xarray and netcdf

Data with time range 4029-01-16 12:00:00 to 4114-12-16 12:00:00, mentioned here: https://github.com/cp4cds/c3s_34g_manifests/issues/9. Only 3 datasets have this issue in those that I scanned.

I came across this issue in xarray https://github.com/pydata/xarray/issues/2742 which might be related to 2 (a)

Suggested issues we can fix:
(1) we can fix.
(2) If these are dimensions - then they cannot be fixed; if they are auxiliary coordinates: we can fix.
(3) @ellesmith88 double-check the results of the HDF errors
(4) we cannot fix, exclude this
In all cases:

[ ] we should check the error reports that DKRZ generated in 34g
[ ] we should report to the Data Providers.
[ ] synchronise all these results with the QC results repo (see: https://github.com/cp4cds/c3s_34g_qc_results/blob/master/QC_Results/QC_template_v2.json) - we need to talk to Fran about how to report these. Some are "minor", some will "fail"

Checking the HDF errors:
/badc/cmip6/data/CMIP6/CMIP/AWI/AWI-CM-1-1-MR/historical/r1i1p1f1/Amon/va/gn/v20181218/va_Amon_AWI-CM-1-1-MR_historical_r1i1p1f1_gn_186601-186612.nc : Opens fine with xarray and netCDF4
/badc/cmip6/data/CMIP6/ScenarioMIP/MOHC/HadGEM3-GC31-LL/ssp126/r1i1p1f3/Amon/ts/gn/v20200114/ts_Amon_HadGEM3-GC31-LL_ssp126_r1i1p1f3_gn_205001-210012.nc: Opens fine with xarray and netCDF4
/badc/cmip6/data/CMIP6/ScenarioMIP/MOHC/HadGEM3-GC31-MM/ssp126/r1i1p1f3/Omon/tos/gn/v20200515/tos_Omon_HadGEM3-GC31-MM_ssp126_r1i1p1f3_gn_203001-204912.nc: still error getting values of lat/lon with xarray (ds.latitude.values) and netcdf (ds['latitude'][:])
/badc/cmip6/data/CMIP6/CMIP/MOHC/UKESM1-0-LL/historical/r1i1p1f2/Amon/tas/gn/v20190406/tas_Amon_UKESM1-0-LL_historical_r1i1p1f2_gn_185001-194912.nc: Opening error with xarray and netcdf

@ellesmith88: Please re-check both files that had errors. I have copied them off quobyte and re-ingested the same files - which I think fixes them !!!!! I'm confused.

@agstephens they both work now!

@ellesmith88: actions as agreed:

[ ] 1. Duplicate lat and/or lon definitions in file:

[ ] Contact data provider to confirm which coord variables should be tagged with the cf-attributes of standard_name and units.
[ ] Write a fix to remove from the relevant coord variables.

[ ] 2. Missing values in coordinate variables: If these are dimensions - then they cannot be fixed (see CF rule at: https://cfconventions.org/cf-conventions/cf-conventions.html#missing-data); if they are auxiliary coordinates: we can fix.

[ ] Write a fix for this.

[x] 4. No action needed on item 4

data providers not responded, but have been contacted.

All auxiliary coordinates. But can include check for this in fix anyway.

Fixes below:
def replace_lat_and_lon_fill_values(ds, **operands):

    # get the value to mask
    value = operands.get('value')

    # gets latitude and longitude
    lat = xu.get_coord_by_type(ds, 'latitude', ignore_aux_coords=False).name
    lon = xu.get_coord_by_type(ds, 'longitude', ignore_aux_coords=False).name

    # if they are coordinate variables - don't fix
    for coord_id in lat, lon:
        if ds.coords[coord_id].dims == (coord_id,):
            return ds

    # make nans
    # ds[lat] = ds[lat].where(ds[lat] != value)
    # ds[lon] = ds[lon].where(ds[lon] != value)

    # or change fill value and sest encoding for fill value
    ds[lat] = ds[lat].where(ds[lat] != value, 1e+20)
    ds[lon] = ds[lon].where(ds[lon] != value, 1e+20)
    ds[lat].encoding['_FillValue'] = 1e+20
    ds[lon].encoding['_FillValue'] = 1e+20

    return ds

def remove_var_attrs(ds, **operands):
    """
    :param ds: Xarray DataSet
    :param operands: sequence of arguments
    :return: Xarray Dataset

    Change the attributes of a variable.
    """
    var_id = operands.get("var_id")

    attrs = operands.get("attrs")
    for attr in operands.get("attrs"):
        ds[var_id].attrs.pop(attr)

    return ds

The data providers haven't replied concerning the duplicate lat and/or lon definitions in file

Fixes have been put into branches: https://github.com/roocs/dachar/compare/failed_dataset_fixes and https://github.com/roocs/daops/compare/failed_dataset_fixes

These fixes are to fix datasets that have failed when they are being scanned into the catalog, so they would somehow need to be fixed before this scan.
The fixes would need to be turned on


[BazelCI] Add support for environment variables

Description of the problem / feature request:
This is specific to the benchmarking pipeline on BazelCI.
Right now we only specify the platform to run benchmarking on. We should be able to replicate CI's tasks' environments, which include not only the platform but also things like ENV vars, build flags, ...
Example:
https://github.com/bazelbuild/bazel/blob/master/.bazelci/postsubmit.yml#L116

Closing this as it's a duplicate of #46.


Update .gitignore

IMPORTANT: Please do not create a Pull Request without creating an issue first.
Any change needs to be discussed before proceeding. Failure to do so may result in the rejection of the pull request.
Please provide enough information so that others can review your pull request:

Explain the details for making this change. What existing problem does the pull request solve?

Test plan (required)
Demonstrate the code is solid. If not, please add WIP: in its title.

Closing issues
Put closes #XXXX in your comment to auto-close the issue that your PR fixes (if such).
Self Check before Merge
Please check all items below before review.

[ ] Corresponding issues are created/updated or not needed
[ ] Docs are updated/provided or not needed
[ ] Examples are updated/provided or not needed
[ ] TypeScript definitions are updated/provided or not needed
[ ] Tests are updated/provided or not needed
[ ] Changesets are provided or not needed

Please add hacktoberfest-accepted label

Hey @Divyarajsinh1312, sorry but nothing has changed since https://github.com/pankod/refine/pull/2796 this PR has been rejected. If you want to contribute to this project, please try to put a little bit more effort on your work. We're trying to be as much welcoming as possible to any kind of contribution but unfortunately this is not one of them.
Closing this for now, please check open issues if you want to contribute or you can look through the project to find something new. 🙏


Porting Lib failed to apply mixin

Context
Startup crash; only happens when adding Create Fabric to the modpack
Crash Report
https://mclo.gs/cD9qgWr

was a problem with Combat Enchantments


bunch of cleanup and more cruft removal

Looks good to me except for my comment on https://github.com/quantopian/zipline/commit/a89e299f31659e70adfa8c62803c642e32e88c1c


Payload Mediatype and certificate thumbprint annotation constants are not exported for API consumers

What is not working as expected?
The constants defined in internal/envelope/envelope.go
MediaTypePayloadV1            = "application/vnd.cncf.notary.payload.v1+json"
AnnotationX509ChainThumbprint = "io.cncf.notary.x509chain.thumbprint#S256"

are not exported, and therefore not available for consumers of the sign and verify API.   Since these are required values per the specification, programs that wish to work with notation signatures must duplicate these constants.
What did you expect to happen?
These constants should be available through a public notation-go or notation-core-go export
How can we reproduce it?
Attempt to access these required constant values from external code.
Describe your environment
Linux/Go 1.22
What is the version of your notation-go Library?
notation-go v1.1.0
notation-core-go v1.0.2

That's a good idea.
@c-grund may I ask what you are trying to do that requires the export of these two variables?

I'm working on set of tools that can sign and verify signed custom oci objects (among other things).  We're using notation as a library for one of the signature formats.  Despite performing sign/verify through the API, we want to maintain compatibility with the Notation CLI for performing those functions, and thus wish to stick close to the spec.  The constants referred to here are marked as MUST items, so we'd rather not redefine them in our code.

This issue requires a revisit since general developers should not re-invent the wheel for signing and verification.
@c-grund I can understand that you need AnnotationX509ChainThumbprint for OCI signature filtering. May I ask why you require MediaTypePayloadV1 to be exposed as it should be completely hidden from the end users?

@c-grund May I know if this issue is still critical for your scenario?

Sorry for the delay in response.
I've moved on to a new project, but when I last worked with this scenario, I believe we got away from needing to use MediaTypePayloadV1.   I think at one point we were directly constructing the payload, and in order to maintain compatibility with the spec we wanted to use the library defined constant for the media type data.  Since then, we dropped the requirement that led us to building the payload ourselves, and therefore no longer needed to access the library constant.


BUGFIX for Builtin Resources Target Failing on Windows

Description
Builtin resources cannot be generated automatically with default windows API, due to an OS limit, as it cannot accept a command line instruction with parameters and length longer than a certain limit. Therefore, the instruction is invalid.
One solution is to transform the command into a bash command which can take long arguments. However, it still needs to be manipulated manually as the required cmake paths point to a Windows location than a mounted location on WSL/bash.
The best solution is to write all resource address into a file and make the build to read from the file to generate the builtin resources files which this PR represents.
References

https://docs.microsoft.com/en-us/troubleshoot/windows-client/shell-experience/command-line-string-limitation
https://stackoverflow.com/questions/3205027/maximum-length-of-command-line-string
https://stackoverflow.com/questions/19354870/bash-command-line-and-input-limit

Testing

TODO list:

Can one of the admins verify this patch?

I'll merge as soon as DCO is signed


Adds tags for asserts

This is useful for skipping the asserts in specific environments.

ci check this

@mattymo The pipeline seems to have succeeded but the check on github still says "failed"

ci check this


adding pgvectorscale

does pgvectorscale require specific columns in my PG table, the same as in your example?
e.g. I have 'id' , 'vector', and 'metadata' columns, no dates, with pgvector (hnsw)  index over 'vector'
can I add pgvectorscale? if I can, will I need to / can I  delete the previous pgvector/hnsw index?
thank you

No specific columns are required except for the vector column. Yes, you can just add pgvectorscale on the vector column and drop the hnsw index.


Headless Test Button Command not executed

Describe the bug
In a Headless test it is not possible to get the command executed when we trigger a button click.
I would expect that the command is trigger and not only the click event fired when we trigger a button click with button.RaiseEvent(new RoutedEventArgs(Button.ClickEvent, null)); .
To Reproduce
ViewModel
public partial class ButtonTextViewPageViewModel : ObservableObject
{

	[ObservableProperty]
	private string _text = "Not Set";

	[RelayCommand]
	private void SetTestText()
	{
		Text = "Test";
	}
}

Usercontrol
<UserControl
    x:Class="ButtonTextViewPage"
	...
    xmlns:viewModels="clr-namespace:.ViewModels"
    d:DesignHeight="450"
    d:DesignWidth="800"
    x:DataType="viewModels:ButtonTextViewPageViewModel">

    <StackPanel Orientation="Vertical">
        <TextBox Name="TestTextBox" Text="{Binding Text}" />
        <Button
            Name="TestButton"
            Command="{Binding SetTestTextCommand}"
            Content="Set Test Text" />
    </StackPanel>
</UserControl>

Test which shows that the command binding in general works but button.RaiseEvent(new RoutedEventArgs(Button.ClickEvent, null)); does not trigger command
[AvaloniaTest]
public void TestButtonAndText()
{
	ButtonTextViewPageViewModel viewModel = new ButtonTextViewPageViewModel();

	ButtonTextViewPage page = new ButtonTextViewPage
	{
		DataContext = viewModel
	};

	Window window = new Window
	{
		Content = page
	};

	window.Show();

	Dispatcher.UIThread.RunJobs();

	TextBox textBox = page.Get<TextBox>("TestTextBox");

	const string expectedText = "SetOverInput";
	textBox.Text = expectedText;

	Assert.That(textBox.Text == viewModel.Text);

	Button button = page.Get<Button>("TestButton");

	const string expectedTextOverButton = "Test";

	button.Focus();

	// Fails if we use this
	button.RaiseEvent(new RoutedEventArgs(Button.ClickEvent, null));

	// works if I use this
	// button.Command.Execute(null);

	Dispatcher.UIThread.RunJobs();

	Assert.That(expectedTextOverButton == viewModel.Text);
}

Expected behavior
That the command is trigger on simulated button click.
Environment

OS: All
Avalonia-Version: 11.0.6

The reason why is pretty clear - Button doesn't listen for Click event, it raises right before executing a command - https://github.com/AvaloniaUI/Avalonia/blob/master/src/Avalonia.Controls/Button.cs#L343-L344. In normal world it would still be a normal behavior, but is confusing with unit testing.
Alternatively, you can raise key or pointer events. Also possible to use extension methods on Window class - window.KeyRelease(Space) (Space key triggers click on the button). In this case button needs to be focused.

@maxkatz6 thanks It works like that:
button.Focus();
window.KeyReleaseQwerty(PhysicalKey.Space, RawInputModifiers.None);

I made a PR as this was not clear for me after reading the docs:
https://github.com/AvaloniaUI/avalonia-docs/pull/328


Built in and external usb keyboards stop working when Mac goes to sleep

Hi,
I am suspecting this is already reported by many users (https://github.com/tekezo/Karabiner-Elements/issues/769), however I did not find reference to 'sleep'.
I can reproduce it anytime. MacOS 10.12.5, latest karabiner, MBP 2015

everything works
pmset sleepnow
wake up from sleep - keyboards do not work anymore. disabling Karabiner enables them.

Thanks in advance!

I have this issue as well. MacOS 10.12.6, KE 11.3.0, MBP 2015. (worth noting that it's happened ever since the Sierra debacle and with multiple OS and KE versions). Keyboard is a KUL ES-87.


cloudfront URL of the static web app

http://d3dztcudgkj2nd.cloudfront.net/

Please don't click this link.


RazorViewEngineOptions FileProviders .net core 3.0

services.Configure(options =>
{
options.FileProviders.Add(
new EmbeddedFileProvider(typeof(MySharedController).GetTypeInfo().Assembly));
});
.net core 3.0  RazorViewEngineOptions changed.  without FileProviders to locate view
@Rick-Anderson
tks!

Moved to #14509
@zhou881117 Using the feedback form at the bottom of the topic tags the issue with metadata and pings the author. No worries ... I moved it. Track it on the new issue.


xMySqlProvision Requires New Params

It appears that xMySqlProvision now requires:
MySQLVersion = string
PermissionType = string()
UserName = string
And no longer ServiceName. Could you please provide an example of what needs passed? It's breaking my DSC scripts. Thanks!

I see the example, but not an example of the actual string, and string() values being passed. It does not like what I'm passing for MySQLVersion. "<IP_ADDRESS>"

@psantry This function seems to suggest that it should be "5.6.17". Can you try that three-part version instead?


reset fixes

follow up for #92

the travise-ci/push build is a false positive. I tries to load wip-fixes branch which no longer exists. And coverage remains at 100%.
So please @hoodiehq/maintainers could you review?

LGTM :+1:

I tested this extensively and full stack with https://github.com/hoodiehq/hoodie-app-tracker/pull/54 so going ahead and merging it


[enhancement]: Generate initial latents by running diffusion in reverse from an initial image (if metadata exists)

Is there an existing issue for this?

[X] I have searched the existing issues

Contact Details
No response
What should this feature add?
When doing img2img, add an option to generate initial latents by running diffusion in reverse from the initial image if metadata is present (or from a user-defined description).
See code in https://github.com/bloc97/CrossAttentionControl/blob/main/InverseCrossAttention_Release.ipynb:
gen_latents = inversestablediffusion(input_image, "a photo of a woman with blonde hair", refine_iterations=5, guidance_scale=4.0)
The inversestablediffusion function has the magic sauce.
Alternatives
No response
Aditional Content
No response

this would make img2img considerably more usable for making refinements to an image - in intuitive terms the initial latent noise would be more inclined to cooperate with the user's intentions

How does one apply this in a workflow? Does it work like img2img except that there is an initial step of telling the inverse function what features of the initial image you're particular interested in keeping?
@damian0815 are you assigning yourself to work on this? Sounds like a handy feature.

My take: I imagine this would be done just before the img2img step to create latents representing what the source image is of so they can be altered to something else via img2img. This is where we could lean on the metadata if the image were initially generated via InvokeAI or let the user type in a Rastafarian cat (for example) as the initial image text. I see all kinds of creative possibilities. Note that it doesn't always make sense to do this step, which is why it should be optional in the img2img workflow.

this is going to need UI support, as it functionally disables the seed and variations. perhaps that's where the prompt input would go? ping @blessedcoolant @psychedelicious

I can't say I understand what this actually does, but it sounds cool.


Local files and Anidb mylist not synched.

Specifically, Mirai Fukuin & Extra Chorus, and Mojokko Shimai no Yoyo to Nene (with its special).
These 4 were the few of the latest ones being downloaded, and they are not shown in my anidb mylist. I just noticed this today.
All checked for crc, nothing wrong with them. As per my torrent history, all 4 of them were the last ones being downloaded, but according to my folder details, the last ones being downloaded were another 2 more series. I don't know which one finished first...
I already tried deleting the series and then run import again, nothing.
JMMD server tab > sync anidb mylist > nope.
JMMS > sync mylist > still nope. Funny thing about this step though, no matter how many times i click 'sync mylist', it will always show that 177 items in queue. After a few minutes, it reduced to 0 immediately. This was the last thing i did to jmms. Here's a pic:

I didn't change any settings or anything btw. I download tons of series at a time, and they all showed up nicely in mylist.
Logs here:
https://www.dropbox.com/sh/h4inyqufkoq71i8/AAAx4xNDu2IUw_grNw040_uTa?dl=0

Can you post a screenshot of how your mylist settings look.
This is mine

Like this?

Now confirming that all my recently downloaded anime is not synching with anidb mylist. Just downloaded 3 new series and it's not showing green in anidb.
Speaking of synching, MAL is not being updated too and pressing 'Download Watched States from MAL' in server tab of JMMD does nothing. "Process is Queued" dialog came but that's it, queue command reads 0 all the time.

Ok for the AniDB stuff, can you do the usual thing of clearing the JMM server logs
And then try marking an episode as watched
And then send me those logs

Marking an episode of the new animes right? Doing it now.

Here's the log folder. I've uploaded the fresh log after clicking watch on the new anime. The backup is also there. You wanted JMMS only right?
https://www.dropbox.com/sh/v184wuxwecnk3rm/AABJkQXfpxCNRDj6feD9drEPa?dl=0

Whoa i just read the log file and it seems that it started adding mylist entry after i click watch. And it only add that episode. I've confirmed it in anidb, only that episode was added after clicking watch.

yeah, I think you may have had some temporary issues (shown by these log entries)
411 NO SUCH MYLIST ENTRY
When you get that type of error it menas that it expected the file to be on AniDB, but it wasn't
So it then added it, and mark it as watched

First time this happened was on the 14th Jan, that day around 7 series/movies was being downloaded and 5 managed to get through before Mirai Fukuin & the other one got ignored. Timing from the folder detail was around 02:30am, and the 1st one to be having this problem is Majokko Shimai. I tried looking at the log on that day, and my limited knowledge of reading what was there failed me.

Decided to reinstall the stable version of both server & desktop just now, and all's right with the world. Files are being added nicely and MAL is getting updated now too. Must've been some kind of hidden bug or something.


Update for Angular 4+

Hi, I've updated Metabubbles for Angular 4 using ng-cli and TypeScript if you're interested.

Awesome, thank you!


There was a problem while training.

Why is this happening?
CRY = 0.00 , ANGRY = 0.00 ...
Problem with Batch Size?
Also, I wonder why it is early stop.
Help, please!

Hi Emma, check the parameters in your early stop callback. Your training is being interrupted because the loss is too low and not showing any improvement according to what is defined in the callback.
early_stop = EarlyStopping(monitor='val_loss',                             min_delta=0.001,                             patience=3,                             mode='min',                             verbose=1)


When Storage Explorer is in dark mode the text color inside the Text Box to query in Azure Cosmos DB becomes white when mouse is over it

Storage Explorer Version: 1.14.0
Build Number: 20200624.3
Platform/OS:Windows 10
Bug Description
When Storage Explorer is in dark mode, the text color inside the Text Box to query in Azure Cosmos DB becomes white when mouse is over it.
When I query for a particular document in a collection there is a Text Box that starts with: "SELECT * FROM c" and then I write down the query. When I put the mouse over this text box in order to edit my query the text becomes white and unreadable. Here below is some pictures:

My query:

When the mouse is over it:

When I select to edit the query:

Thank you!

Azure Storage Explorer support for Azure Cosmos DB has been discontinued as of v1.24.0. We recommend users to use Azure Cosmos DB’s Data Explorer, which can be found in Azure Portal or as a stand-alone version in http://cosmos.azure.com/ for the best Azure Cosmos DB experience. You can also find additional Azure Cosmos DB tooling here: Azure Cosmos DB Tools


skpm builder resourceLoader tries to bundle md files

Hi! I have a small bug/issue:
I'm working on implementing the mdx React markdown loader into a react-sketchapp project and added a custom Webpack rule, but was left with Sketch crashing.
After some debugging, I traced the issue to resourceLoader trying to bundle the .md. I was wondering whether md and mdx could be added to the ignore regex, or if you'd have any idea on a way to allow for manual overrides?
https://github.com/skpm/skpm/blob/master/packages/builder/src/utils/resourceLoader.js#L5
Thanks!

You can update the webpack config that skpm uses: https://github.com/skpm/skpm/blob/master/docs/build-configuration.md#webpack


Master to phase2 merge

Need to catch phase2 up to the changes made on master

Coverage increased (+0.02%) to 94.35% when pulling 728be7a3852cccf4d33719e976eb3ea1b4850802 on master-to-phase2 into a5785112087ab796060b14700fc3a5c876c8bdc5 on phase2.

Coverage increased (+0.03%) to 94.36% when pulling fea04d50d03210eef7f4b5350b4097db25eb8e6e on master-to-phase2 into a5785112087ab796060b14700fc3a5c876c8bdc5 on phase2.

:+1:

:+1:


Trying to Stake HNT Issue

Describe the bug
A clear and concise description of what the bug is.
I have several Black wallets, 2 with one for 2 different continents, I share these with my two sons and they’re working fine.   I have a third Black Wallet not connected to any hotspots, here this is my personal HNT and I’m trying stake a substantial amount of HNT.  With this one wallet when I follow the process through the globe and want to connect the wallet there is no Helium option in the drop-down menu, it’s on Phantom & I can’t change to Helium as I did with my other two wallets?
im using the latest update as of 5 minutes ago for the Black Wallet and IPhone 12

To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
When pressing on connect, I should see an option for connecting Helium
Screenshots
If applicable, add screenshots to help explain your problem.

Smartphone (please complete the following information):

Device: iPhone 11
OS: iOS 14.6
App Version I don’t know?

Additional context
Add any other context about the problem here.
I also can’t link to a SOL wallet to send SOL by exchanging some HNT as I have done on my other two wallets.

This is a known issue with the wallet standard. There is a race condition in populating the wallet options. Closing and reopening the browser is a short term solution.


关系图：当进行缩小操作时线的label会跑的很远

Version
4.8.0
Steps to reproduce
鼠标滚轮缩小图
What is expected?
线的label一起缩小或隐藏，label离线的距离不要变
What is actually happening?
线的label距离线会很远（缩小程度越大偏离距离越远）

@skayi It seems you are not using English, I've helped translate the content automatically. To make your issue understood by more people, we'd like to suggest using English next time. 🤗
TRANSLATED
TITLE
Diagram: The cable of the line runs very far when the downsizing operation is performed
BODY
Version
4.8.0
Steps to reproduce
Mouse wheel zoomed out
What is expected?
The label of the line is reduced or hidden together, and the distance of the label offline should not change
What is actually happening?
The label of the line will be far away from the line (the greater the reduction, the farther the deviation will be)

thanks

@skayi 可以设置 labelLayout.hideOverlap 隐藏


Deleting or changing pairs complains "Line value outside range"

Checklist

[X] Have you tried updating the plugin to the latest version?
[X] Have you checked the Breaking Changes issue?
[ ] Have you read through :h nvim-surround to see if there might be any relevant information there?

Neovim Version
NVIM v0.9.0
Plugin Version
Tagged (Stable)
Minimal Configuration
local lazypath = vim.fn.stdpath('data') .. '/lazy/lazy.nvim'

if not vim.loop.fs_stat(lazypath) then
    vim.fn.system {
        'git',
	'clone',
	'--filter=blob:none',
	'https://github.com/folke/lazy.nvim.git',
	'--branch=stable',
	lazypath,
    }
end

vim.opt.rtp:prepend(lazypath)

require('lazy').setup {
    { 'kylechui/nvim-surround', config = true },
}

Sample Buffer
(test)

Keystroke Sequence

Insert something below (test):(test)
------

Move to the new line and enter visual mode.
Delete the second line.
Move inside the parens and ds) or cs)}.

https://github.com/kylechui/nvim-surround/assets/53635735/0bc78098-4382-4d6b-aac3-de03eb3425f9
Expected behavior
Parens deleted or changed.
Actual behavior
E5108: Error executing lua ...are/nvim/lazy/nvim-surround/lua/nvim-surround/buffer.lua
:58: Line value outside range
stack traceback:
        [C]: in function 'nvim_buf_set_mark'
        ...are/nvim/lazy/nvim-surround/lua/nvim-surround/buffer.lua:58: in function 's
et_mark'
        ...are/nvim/lazy/nvim-surround/lua/nvim-surround/buffer.lua:116: in function '
set_operator_marks'
        ...re/nvim/lazy/nvim-surround/lua/nvim-surround/motions.lua:22: in function 'g
et_selection'
        ...are/nvim/lazy/nvim-surround/lua/nvim-surround/config.lua:290: in function <
...are/nvim/lazy/nvim-surround/lua/nvim-surround/config.lua:289>
        ...hare/nvim/lazy/nvim-surround/lua/nvim-surround/utils.lua:33: in function 'g
et_nearest_selections'
        ...share/nvim/lazy/nvim-surround/lua/nvim-surround/init.lua:159: in function '
delete_surround'
        ...share/nvim/lazy/nvim-surround/lua/nvim-surround/init.lua:298: in function <
...share/nvim/lazy/nvim-surround/lua/nvim-surround/init.lua:289>

Additional context
This happens when mark < or > is invalid.

Thanks for the detailed reproduction steps and information! It really helped me diagnose this quickly. Please try the latest commit on main and see if that fixes the issue.

Thanks! Now it works a treat.


RNN unrolled, cannot be converted using from_keras_model

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 18.04
TensorFlow installed from (source or binary): tf-nightly
TensorFlow version (or github SHA if from source):
version is 2.2.0.dev20200508, git version is v1.12.1-31489-g6047d50555

Describe the current behavior
Conversion fails with the following:
ValueError: Cannot unroll a RNN if the time dimension is undefined.
- If using a Sequential model, specify the time dimension by passing an `input_shape` or `batch_input_shape` argument to your first layer. If your first layer is an Embedding, you can also use the `input_length` argument.
- If using the functional API, specify the time dimension by passing a `shape` or `batch_shape` argument to your Input layer.

This seems to be due to the following addition to the from_keras_model conversion:
self._keras_model.save(temp_dir, save_format="tf")

Describe the expected behavior
Conversion should succeed as it did before in previous versions.
Standalone code to reproduce the issue
import numpy as np
import tensorflow as tf

from tensorflow.keras.models import Model

class SimpleModel(Model):
    def __init__(self):
        super().__init__()
        self.model_name = "mnist"
        self.train_data = None
        self.test_data = None
        self.calib_data = None
        self.num_calib = 1000
        # (data preprocessing) Normalize the input image so that
        # each pixel value is between 0 to 1.
        self.pre_process = lambda x: x / 255.0

        self._load_data()

    def _load_data(self):
        # Load MNIST dataset
        mnist = tf.keras.datasets.mnist

        # _data: (images, labels)
        self.train_data, self.test_data = mnist.load_data()
        self.calib_data = self.pre_process(
            self.train_data[0][0 : self.num_calib].astype(np.float32)
        )

    def train(self):
        cell = tf.keras.layers.GRUCell(3)

        model = tf.keras.models.Sequential([
            tf.keras.layers.Input(shape=(28, 28), name='input'),
            #tf.keras.layers.LSTM(32),
            tf.keras.layers.RNN(cell, unroll=True),
            tf.keras.layers.Flatten(),
            tf.keras.layers.Dense(10, activation=tf.nn.softmax, name='output')
        ])
        model.summary()

        train_images = self.pre_process(self.train_data[0])
        train_labels = self.train_data[1]
        test_images = self.pre_process(self.test_data[0])
        test_labels = self.test_data[1]
        # Train the digit classification model
        model.compile(
            optimizer="adam",
            loss="sparse_categorical_crossentropy",
            metrics=["accuracy"],
        )
        model.fit(
            train_images,
            train_labels,
            epochs=1,
            validation_data=(test_images, test_labels),
        )
        # dump SavedModel - ANOTHER BUG HERE!
        #model.save(str(self.savedModel_dir))

        return model

    def eval(self, tflite_model_path: str):
        interpreter = tf.lite.Interpreter(model_path=str(tflite_model_path))
        interpreter.allocate_tensors()

        input_index = interpreter.get_input_details()[0]["index"]
        output_index = interpreter.get_output_details()[0]["index"]

        # (data preprocessing) Normalize the input image so that
        # each pixel value is between 0 to 1.
        test_images = self.pre_process(self.test_data[0])
        test_labels = self.test_data[1]
        # Run predictions on every image in the "test" dataset.
        prediction_digits = []
        for test_image in test_images:
            # Pre-processing: add batch dimension and convert to float32 to match with
            # the model's input data format.
            test_image = np.expand_dims(test_image, axis=0).astype(np.float32)
            interpreter.set_tensor(input_index, test_image)

            # Run inference.
            interpreter.invoke()

            # Post-processing: remove batch dimension and find the digit with highest
            # probability.
            output = interpreter.tensor(output_index)
            digit = np.argmax(output()[0])
            prediction_digits.append(digit)

        # Compare prediction results with ground truth labels to calculate accuracy.
        accurate_count = 0
        for index, _ in enumerate(prediction_digits):
            if prediction_digits[index] == test_labels[index]:
                accurate_count += 1
        accuracy = accurate_count * 1.0 / len(prediction_digits)

        return accuracy

    def _get_calib_data_func(self):
        def representative_data_gen():
            for input_value in self.calib_data:
                input_value = np.expand_dims(input_value, axis=0).astype(np.float32)
                yield [input_value]

        return representative_data_gen

if __name__ == "__main__":
    temp = SimpleModel()
    model = temp.train()

    converter = tf.lite.TFLiteConverter.from_keras_model(model)
    converter.optimizations = [tf.lite.Optimize.DEFAULT]
    converter.representative_dataset = temp._get_calib_data_func()

    # save INT8 tflite
    converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8,
        tf.lite.OpsSet.SELECT_TF_OPS]
    converter.experimental_new_converter = True
    tflite_model_INT8 = converter.convert()
    open("lstm_unrolled_int8.tflite", "wb").write(tflite_model_INT8)

@SaoirseARM
I ran the code shared and face invalid syntax, please share complete indented code such that we can replicate the issue.
Please fin the gist here

Apologies @Saduf2019 , I have updated the comment with syntax error free code.

I am able to replicate this issue, please find the gist here Thanks!

There may be a misunderstanding here, this issue can be seen building from source or from tf-nightly and not in the released tensorflow v2.2.
I have updated the description above. Apologies for the confusion.

@renjie-liu can you take a look?

Seems the issue is indeed for keras_model reloading, can you try convert from saved_model instead? see https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/examples/experimental_new_converter/Keras_LSTM_fusion_Codelab.ipynb
meanwhile, Hi Jaesung, can you help take a look about the keras_model import? I supscious it's because keras model serialize/deserialze caused the problem.
thanks!

Thank you, I am actually already using the workaround you suggested above and it works absolutely fine. Thanks for looking into this!

Great!

Hi @renjie-liu, with the example you pointed. I tried some quantization experiments, seems:
for dynamic range quantization, weights within RNN still FP32;
for fully integer quantization, currently failed with a runtime error:
RuntimeError: tensorflow/lite/kernels/unidirectional_sequence_lstm.cc:349 input_to_output_weights->dims->data[1] != n_input (28 != 0)Node number 0 (UNIDIRECTIONAL_SEQUENCE_LSTM) failed to prepare.
Colab example

fully integer quantization is not done yet.
but the dynamic range quantization should work, can you share any code snippet?
thanks!

This is no longer an issue in tensorflow-2.4.1. Thanks! Closing issue.

Hi @renjie-liu, with the example you pointed. I tried some quantization experiments, seems:
for dynamic range quantization, weights within RNN still FP32;
for fully integer quantization, currently failed with a runtime error:
RuntimeError: tensorflow/lite/kernels/unidirectional_sequence_lstm.cc:349 input_to_output_weights->dims->data[1] != n_input (28 != 0)Node number 0 (UNIDIRECTIONAL_SEQUENCE_LSTM) failed to prepare.
Colab example

I have the same error with Tensorflow-2.5.0 (MNIST_LSTM -> fully integer quantization)
RuntimeError: tensorflow/lite/kernels/unidirectional_sequence_lstm.cc:895 input_to_output_weights->dims->data[1] != n_input (28 != 528)Node number 0 (UNIDIRECTIONAL_SEQUENCE_LSTM) failed to prepare


EL-993: CCQ Service Unavailable Page

Jira ticket
What changed and why
If the CCQ service is unavailable (e.g. down for maintenance) or if CFE is not available the cloud platform error page is shown. We want control of the error message shown.
Guidance to review
My only comment on this, is why can't we class this 'service unavailable' page with a 503 error code, instead of replacing the 500 page?
We could make the 'service unavailable' page more bespoke in the future, if needed and keep the 500 page as a generic page?
Checklist
Before you ask people to review this PR:

Tests and rubocop should be passing
Branch is generally up to date with main Github - definitely no conflicts
No unnecessary whitespace changes. These make diffs harder to read and conflicts more likely.
PR description says what changed and why, with a link to the JIRA story.
Diff has been checked for unexpected changes being included.
Commit messages say why the change was made.

Looks good. For UAT we can use our feature flag override functionality to turn on maintenance mode (although thinking about it, once it's switched on, we won't have access to the feature flag override screens to turn it off again! That's not a problem, though, I don't think - we can us the command line to remove the override if we need to).


⚠ Bofrost.at has degraded performance

In d337635, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1565 ms

Resolved: Bofrost.at performance has improved in 100ea67 after 6 minutes.


Simplify and fix Projection's getter functions

Fixes #90427
Fixes #93683
This PR simplifies and fixes a few functions of struct Projection :

Fix get_aspect() and get_lod_multiplier() with camera in PROJECTION_FRUSTUM mode.
Fix and document get_pixels_per_meter(). Note : this function has a clumsy definition and not used anywhere. It's a good candidate for deprecation.
Optimize get_z_far(), get_z_near(), get_z_far(), get_viewport_half_extents(), get_far_plane_half_extents() and get_fov()
Add (more) unit test to the above functions

Apologies, I had naming issues with the branch.
Just re-opened another identical PR : #100209


[BUG] Can't index Geospatial Data

Using the provided SimpleFieldAttribute for a GeographyPoint, it's creating a complex field rather than Edm.GeographyPoint. Also, if I try to push data to a manually created Edm.GeographyPoint field in my index:
GeographyPoint.Create(item.Latitude, item.Longitude)
I'm getting the following error:
The request is invalid. Details: parameters : Invalid GeoJSON. The 'type' member is required, but was not found. Indeed there's no type when GeographyPoint is serialized:
{"Latitude":80.206673,"Longitude":13.018703,"IsEmpty":false,"Z":null,"M":null,"CoordinateSystem":{"EpsgId":4326,"Id":"4326","Name":"WGS84"}}
So in summary, how to add GeographyPoint data to an index? Is there any updated sample application on Github or somewhere else? Should I add data using REST rather than the official Nuget package?

Hi @thdotnet.   Thank you for your feedback and we regret that you're experiencing difficulties.   Unfortunately, I'm not able to determine what library you're working with from the context in your description.   Would you please help me understand more about your environment and scenario so that I can route you to the right folks to assist?

Hi @jsquire , I am also seeing this behavior, using Azure.Search.Documents 11.2.0-beta.2.  If it would be helpful, I can probably produce a minimum repro app.

Hi friends,
using the beta package is working fine. Here's a sample: https://stackoverflow.com/a/65079044/1384539


Anonymous Consents TM - minimize localStorage data

In the TM for anonymous consents it was suggested to try and minimize the data being stored in the localStorage related to anonymous consents.

The idea is to drop the templates from the store - how will this affect the update detection mechanism?
At the same time, we shouldn't store all the consents in the localStorage. We should store only the consents that the end-user actually interacted with.

After some investigation, this are conclusions:

Dropping the consent templates from the store would prevent the application from detecting the updated templates on the back-end and displaying the banner again. The reason for this is simple - when the page refreshes, the in-memory store is deleted and we don't have the previous consent templates to compare with the new ones. In this case we still want to drop the templates from the store, we'd have to display the banner after each page refresh.

Synchronizing only consents that the user interacted with is also not possible, as it uses the generic store-synchronization mechanism that synchronizes the given state slice. It's only possible in case we knew the consent template IDs upfront (before the build time) which is far from ideal. Even if we proceed with this, it would require a lot of refactoring.


Improve error handling in hipsolver-test

This is a quick-and-dirty addition of more information in the hipsolver-test output. I've added version printing and some error checking to improve the output.
Example output with the new version and device info:
cgmb@docker:~/hipSOLVER$ build/release/clients/staging/hipsolver-test
hipSOLVER version 1.3.0.a4b0787-dirty
Query device success: there are 1 devices
Device ID 0 : AMD Radeon VII ------------------------------------------------------
with 17.2 GB memory, clock rate 1801MHz @ computing capability 9.0
maxGridDimX<PHONE_NUMBER>, sharedMemPerBlock 65.5 KB, maxThreadsPerBlock 1024, warpSize 64
-------------------------------------------------------------------------
[==========] Running 5880 tests from 54 test suites.
[----------] Global test environment set-up.
[----------] 112 tests from checkin_lapack/GETRS
[ RUN      ] checkin_lapack/GETRS.__float/0
[       OK ] checkin_lapack/GETRS.__float/0 (272 ms)
[ RUN      ] checkin_lapack/GETRS.__float/1
...

Example output on error:
cgmb@docker:~/hipSOLVER$ ROCR_VISIBLE_DEVICES='' build/release/clients/staging/hipsolver-test
hipSOLVER version 1.3.0.a4b0787-dirty
Query device error: cannot get device count
Error: No devices found

For comparison, this is what that error looked like before:
cgmb@docker:~/hipSOLVER$ ROCR_VISIBLE_DEVICES='' build/release/clients/staging/hipsolver-test
[==========] Running 5880 tests from 54 test suites.
[----------] Global test environment set-up.
[----------] 112 tests from checkin_lapack/GETRS
[ RUN      ] checkin_lapack/GETRS.__float/0
Segmentation fault (core dumped)

Looks good! In rocSOLVER, however, we also print the version number of the rocBLAS library it's using. I wonder if we should do the same here with rocSOLVER/rocBLAS and cuSOLVER.

Absolutely. That would be a great improvement, though I'd like to leave that for a future PR. I figure we can get the smallest, simplest version in first and improve upon it later.


Make 'Term' a trait

Proposed first in #35 I suggest to introduce a trait Term.
Objective
sophia's aim is to become a central API for RDF in Rust (#23). As such it would be nice if third party terms could smoothly interoperate with sophia's ecosystem, e.g. rio's terms could be stored in sophia's HashGraph, N3 terms from metis could use the Serializer interface or another crate could provide compatible, ASCII-based terms that are based on &[u8] (maybe even going towards no_std-RDF :blush:).
Required changes
With the latest effort to turn each kind of term (IRI, literal, blank node and variable) into an own type with an aligned API the step to create a trait is no longer that much work. A problem could be that this could turn advanced traits, such as Graph, in a mess of trait bounds.
Suggestion
Here a suggestion how a trait may look like:
trait Term: Clone + Hash + Debug + Display
    + PartialEq + PartialEq<Iri> + PartialEq<Literal> + PartialEq<BlankNode> + Eq
where
    Iri: TryFrom<Self>,
    Literal: TryFrom<Self>,
    BlankNode: TryFrom<Self>,
{
    type TermData: TermData;

    fn value(&self) -> String;
}

The focus should be to keep the Term trait as small as possible while other behaviour is kept separate in other traits, e.g. Resolve.

I'm waiting to see your POC, but I'm not sure exactly what benefit you would gain from such a trait, especially if Iri, Literal and BlankNode are still constrained to be concrete types...
Note that nothing forces implementations of Sophia's traits (Graph, Dataset, Parser, Serializer...) to use Term internally. For example, Rio parsers do not, but still interoperate with Sophia. All that is required is to provide easy ways to convert internal "terms" to and from Sophia's Term.
For that matter, thanks to the TermData parameter, Term is already rather flexible:

Term<&str> aka RefTerm can be cheaply constructed whenever str data is already available;
self-owning variants, such as Term<Box<tr>> or Term<Rc<str>> (aka BoxTerm and RcTerm respectively) can be used in other cases;
maybe we should also give more visibility to Term<Cow<str>>, giving it a type alias of its own (CowTerm), as it provides a nice trade-off between the two options above.

That being said, as I wrote, I'm curious to see your POC, and I remain open to discussion on this issue.

A part of this comment from @MattesWhite on #62 is more relavant to this topic:

I'm currently updating metis to the current state of sophia. This will also be the PoC of how a Term-trait could benefit sophia (see #55). I planned to use Namespace for the prefixes. However, after parsing a prefix's IRI it must be resolved against the current base IRI. Meaning (I) parse the IRI, (II) resolve the Iri and finally (III) convert Iri to Namespace. Hence, the PR. Maybe a impl Resolve<Namespace> for IriParsed could also help thinking.
Ideally, I could reuse the prefix-parser from Turtle for N3. Which means that I would like to return an Iri as Turtle and N3 use different terms which both implement From<Iri>. In fact, my N3Term looks something like:
enum N3Term<TD> {
    Iri(Iri<TD>),
    Literal(Literal<TD>),
    Existential(BlankNode<TD>),
    Universal(Variable<TD>),
    Formula(Formula<TD>), // some kind of `Graph<N3Term>`
    List(Vec<N3Term<TD>>),
}

As you can see I would like to reuse a Graph implementation of sophia which I can't do because it's fixed to sophia_term::Term.

Thanks, this helps me understand the benefit you expect from a Term trait.
I had already given some thoughts (although not as much as you) to N3 and Sophia, and my idea was that N3 could be encoded in the generalized model provided by Sophia. Namely:

lists are represented by a chain of blank nodes,
formulae are represented by named graphs (with blank nodes as names).

So rather than forcing some N3Term type into a plain Sophia Graph, I would have gone for using a Sophia Dataset as a back-end, and wrapping it in a N3Graph providing a higher level API. I don't know if that makes sense.
May be you have already explored this lead and found it unfeasible or suboptimal... It is very possible that I am overlooking some problems of this approach.
But what you propose seems to extend Sophia's scope much beyond RDF. Granted, the current generalized model already extends it... but this is even much more. For example, what you suggest is that not all implementations of the Term trait would be entirely interoperable. If I get a triple out of an N3Graph, and that triple involved a formula, what happens when I try to insert it in another graph?

Granted your suggestion for a N3Graph could work. But you are right. Such a hacked implementation is far from what I imagine. Given the complexity we already have in returning triples, further abstraction don't make things better. All in all, I think I should be satisfied with the possibility to re-use sophia's Term and its subtypes, implementing N3 on top is still a huge amount of work. However, I'm convinced that an N3-engine in Rust could out-perform existing N3-engines by orders of magnitudes if we look at cwm (Python), ldfu (Java) and EYE (Prolog).
In summary I'll close this issue as this (as you mentioned) exceeds the intended implementation of RDF to far.

Problems with Dataset as N3Graph
A short list of problems of your suggestion for the record:

Lists are a first class datatype in N3. The rdf:first/rdf:rest implementation in RDF is rather a reification of N3-lists. Therefore, it makes sense to handle lists as some kind of N3-Term.
Lists are frequently used and manipulated in built-in functional predicates each time traversing the formula would probably be very expensive.
Each time resolving if a blank node is a formula-expression or simply a blank node should be expensive as well.

As mentioned above, I imagine a performance optimized N3-engine. Accordingly, an optimized implementation of a Formula is more practical than re-using sophia's Graphs or Datasets.

Again, I am thrilled by your project of an N3 implementation in Rust. And I sympathize entirely with your concerns about performances. Actually, thinking a little more about that, I beleive this should be the opposite: N3Graph should have its own implementation, but comply with the Graph traits (and/or possibly the Dataset traits, although that requires some thinking...).

While thinking on a graph implementation and the map() stuff I stumbled upon another use-case for a Term trait:
In a bunch of cases terms are passed as read-only references, e.g. the filter methods of Graph. Currently, reference passing is either done by monomorphization or &RefTerm. Both have disadvantages. Monomorphization blows up code-size and compilation times while &RefTerm requires often the creation of a new Term instance. I think this is one of the rare cases where passing a trait object per &dyn TermTrait would be more sufficient.
The described use-case is similar to TermMatcher and indeed we could re-use or improve it.
I haven't flashed out this thought entirely but I think we could use this to simplify some of sophia's APIs. In addition, it may make sense to move Graph's filter-methods to TripleSource (as well as for Dataset and QuadSource) (#52).

If I understand trait objects correctly, &dyn TermTrait is not a direct reference to your, say, RcTerm, but a reference to an object, automatically created by the compiler, that contains a pointer to your RcTerm as well as a table of pointers to its methods (the equivalent of C++'s virtual table). My guess is that creating this trait object is at least as expensive as creating a RefTerm...
But yes, we could consider simplifying the API, and requiring RefTerms everywhere the current API accepts Term<T> for any T. Note however, that with the current approach, developers have a choice: reduce compile time and image size by only calling Graph methods with RefTerms, or reduce execution time by passing whatever term they have at hand, and relying on monorphization. In any case, thorough testing of different options is the only way to know which is the best.

In addition, it may make sense to move Graph's filter-methods to TripleSource

This is already the case. Any iterator of Result<Triple, _> is a TripleSource. And as a bonus, itis also an iterator :-)
https://github.com/pchampin/sophia_rs/blob/f8f030448efa70c2a0576319ec796c6fe1935ec4/sophia/src/triple/stream/_iterator.rs#L8-L13

Okay I agree with you the use of RefTerm mitigates the usage of a trait-object &dyn TermTrait. In fact, I tried a little bit around to implement a TermTrait that hasn't a <TD> type parameter which is necessary to avoid monomorphization. If it should be usable it requires to be comparable to at leas some kind of &Term<TD> and I didn't find a way of doing it with returning a RefTerm :roll_eyes: ...

If I understand trait objects correctly, &dyn TermTrait is not a direct reference ...

Well, your understanding is mostly true. &dyn Trait is a so-called fat pointer that, as you stated correctly, contains two pointers: The first points to the data passed in and the second points to the datatype's vtable. All in all, the creation of a trait object only requires taking a pointer to the data and the vtable-pointer is probably already defined at compile time which means creating a trait object has nearly the same cost as taking a single normal reference where a RefTerm allocates a whole new Term (see size-table below). The real cost of a trait object is that it's methods can no longer be inlined by the compiler, probably resulting in less efficient/optimized code.
Maybe it is better to show the impact of an trait object when we look at the size of different Terms:
// Results of std::mem::size_of<${type}>():
Size of RefTerm:       72
Size of Term<String>:  96
Size of BoxTerm:       72
Size of MownTerm:      96
Size of RcTerm:        72
Size of ArcTerm:       72
Size of &dyn Error:    16
-------------------------
Size of Term<&str>:    72
Size of Literal<&str>: 64
Size of Iri<&str>:     40
Size of &str:          16

As you can see on my 64bit machine each RefTerm requires the allocation of 72 additional bytes of the stack (9 words!) while a trait object always only has a size of 16 bytes, i.e. 2 words.

All in all, the creation of a trait object only requires taking a pointer to the data and the vtable-pointer is probably already defined at compile

Makes sense. I stand corrected, then.
The point your raise about the size of Term in general is valid. One question that I still have is: is it worth it two have IRIs split in ns+suffix? How much time and space does it save, compared to the burden of carrying fat Terms around? Unfortunately, investigating this would be very time consuming... This would be easier if Term was a trait, though :-D
Regarding monoporphization of Graph/Dataset methods, it looks as a different (although related) issue to me. Would you mind posting a new issue on that particular subject?

Thought: a good point about a Term trait would be that types such as Iri or Literal could implement that trait too. This would spare us conversions in many situations.
Other thought: if we want to be able to use &dyn Term, then that puts limitations on the methods of Term. In particular, it can have no "constructor" (method returning Self). This limitation might come back to bite us sometime...

Yet another thought: currently the Term<T> type implements PartialEq<Term<U>>, meaning that I can write t1 == t2 with any two terms. I don't think we could make it with a trait (especially given the constraints on trait objects).
Of course, we could define a function fn term_eq<T1: Term, T2: Term>(t1: &T1, t2: &T2) -> bool but that would not be as nice. Is this really bad, though...

@MattesWhite I created a branch term_trait to experiment with this idea.
Have a look at 9e878ad. If that's a kind of things you had in mind, we can try to propagate it into the rest of the API and see how it fits...

I think 9e878ad is a good point to start I added some comments to the commit.
TL;DR:
The idea to downcast TermTrait seems promising. But instead of adding a TermKind I would introduce "KindTraits" that give a &str-only view on the underlying data.

At last, this is implemented by 91bbb99.
This is a big change (including a few unrelated refactoring).
@MattesWhite (or anyone else): if you struggle updating your code to this new version, contact me and I'll to help.

First of all big respect for this major refactoring of the whole crate :clap: . I see the huge amount of work that has gone into it :+1: and the trait really integrates well.
But with such a large change there are some points I'd like to address which could further improve sophia.
str_absolute()
This function currently returns i8 I'd like to suggest to use a dedicated enum. It shouldn't add overhead and makes the code more readable.
term_*() functions
You introduced several term_*() functions to provide generic default implementations for several standard traits, e.g. Display, PartialCmp and so on.
To make better use of Rust's type system I propose to introduce a dedicated type for these implementations:
struct GenericTermImpl<T: TTerm + ?Sized>(T);

// with
impl<T> Deref for GenericTermImpl<T> {
    type Target = T;

    ...
}

// and
impl<T> From<T> for GenericTermImpl<T> {...}

This new-type clearly shows when a generic implementation is used and that implementors have the opportunity to write optimized implementations for their own types.  I think there could be better documentation and more readable code by introducing a dedicated type:
pub struct RawValue<'a>(pub &'a str, pub Option<&'a str>);

I'd gladly submit PRs for my proposals if you like. However, with ESWC approaching I have only a limited capacity currently.

Thanks for the appreciation ;-)

re. an enum for str_absolute: that would indeed be cleaner, but that would add some boiler plate for the return value of a single private function... I'm not convinced it is worth the trouble, but I'm not agains it.

re. a dedicated type for raw values: you are right, it makes sense now that we have a number of "methods" for this type...

re. GenericTermImpl: I'm not entirely following were you are going with this, but feel free to flesh it out in a PR, whenever you have time :)

With the latest commit 9464afa the new trait TTerm is now part of the public API and well integrated into the crate. Therefore, I consider this issue successfully done :+1:


SAK-44360 Pinned npm version numbers in webcomponents

https://jira.sakaiproject.org/browse/SAK-44360

It is a generated file but your suggestion to add it to .gitignore is not a good one. More here:
https://medium.com/coinmonks/everything-you-wanted-to-know-about-package-lock-json-b81911aa8ab8
It needs to be committed because we need to have consistent npm builds and not transient upgrades on every build


Indent Using spaces

When I select indent using spaces option to 4 it successfully updates the visuals but switching to another file and come back to the same file do not show the correct spaces which I have selected. It automatically converts the spaces to 2

@shubhwebkul Please use the settings:
"editor.tabSize": 4,
"editor.insertSpaces": true,
"editor.detectIndentation": false


Frontend run

Make the typescript code run

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.04%) to 48.888%

Totals

Change from base Build<PHONE_NUMBER>:
0.04%

Covered Lines:
2748

Relevant Lines:
5621

💛  - Coveralls


Rendering a ProgressBar in a TreeView

I was expecting to create a CellRendererProgress to assign to a TreeViewColumn to create a progress bar, but there doesn't seem to be a CellRendererProgress binding.
https://developer.gnome.org/gtk3/stable/GtkCellRendererProgress.html
Is there another way to render a ProgressBar?

The best way would be to just create the binding. It's probably relatively easy and I doubt you are the first one who needs this.

I'll give it a shot, thanks!

Just take a look at how the other GtkCellRenderer... bindings were been implemented.

Added the bindings and it worked like a charm.
Here's the repo I forked and added onto if interested in merging.
https://github.com/Diceshake/gotk3

Sure, feel free to make a pull request.

Implemented with #381


Updates / tweaks (see desc)

Changes

Switched from python to python3 to be explicit in which python version should be installed as everything else in the setup is python3
Moved CMD up higher to improve caching
Added default to ENV vars so builds can be run locally without having to resort to finding the github actions matrix for values (this is open ended to keep)
Removed calls to cmake and node for finding version ; this is unnecessary
Added some additional spaces so multi-line commands (apt install / pip install) are more obvious and to improve readability
Move some ENV vars up in the build order to help with caching
Added --purge to apt-get remove calls to cleanup installed packages further (removes configs and more, full cleanup vs non purge which will leave some basic config/cruft behind.

Suggestions

Given the github action is set to run weekly it is probably good to change to debian-slim:stable instead of debian-[date]-slim:stable to get the most recent base container with appropriate updates and fixes (esp security fixes) that come through
Consider adding apt-get upgrade -y at the start of the docker file / first step to ensure all packages are up to date for security/bug fixes

Raspberry Pi Notes

Tested with Raspberry Pi OS Lite ; may be smart to re-test with full version to see how RAM use may be influenced when a full desktop UI is present. Build steps, even the ones heavy on cpu/ram, left at least 128Mb of RAM free, an average of 384Mb or more based on reviewing htop output during tests. (Ram/CPU use is a rough estimate based on human eyeballs and attention)
This container won't build on a raspberry pi, the following hiccups have been found during tests
The standard zephyr sdk appears to NOT support arm, there are work arounds but it greatly alters the approach to building a container dev compared to x86-64 based machines. Given there are enough differences to be considered mostly-significant, is this something to look into further?
gcc-multilib is not a valid package for install within the container
g++-multilib is not a valid package for install within the container
libxml2-dev needed to be installed due to a zephyr dep needing to be built from sources and required this package
libxslt1-dev needed to be installed due to a zephyr dep needing to be built from sources and required this package
cmsis-pack-manager cant be installed via pip3 without rust+cargo being installed. Below is the necessary additional steps needed for raspberry pi support

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs > /tmp/rustup.sh && chmod a+x /tmp/rustup.sh &&/tmp/rustup.sh -y && source ~/.cargo/env && rm /tmp/rustup.sh
rust up self uninstall

Considerations for others reviewing

Docker can stop building a container at a specific stage (eg $ docker build --target builder -t alexellis2/href-counter:latest .) so multiple docker files are not necessary for this setup / situation

Thanks for the input @mcrosson!  I've integrated some of your improvements and will give the rest ongoing consideration.
I've also done a small overhaul to optimize caching and another pass of the tagging system.
Given the issues with Raspbian, I'm not going to look into that for now.  It's not a ZMK priority either as most (if not all?) the developers are using more powerful hardware, and we have GitHub for the container builds.


Attempt to delete object did not result in a single row modification

https://rollbar.com/vizzuality/CartoDB/items/4159/

Couldn't reproduce, postponed.


Route Reload

Hi,
On app route reload or direct link to an app route I get:
Whitelabel Error Page
There was an unexpected error (type=Not Found, status=404).
No message available

Is there a way to make it work?
With Apache and AngularJS we used .htaccess RewriteRule to overcome this issue.
Is there a way for Spring Boot to do similar?

It is now fixed... check the commit, it worths it
https://github.com/jaxio/celerio-angular-quickstart/commit/fdaeba15b8c76bd8997cfecf2f3e9d0c7a67297e
To my surprise reload not only loads the index but also the current view.

Hi Nicolas, thank you for the quick turn around. Works great. For my point of view this was the only missing thing, other than that great projects. GZ