Support AutoMapper's ProjectTo in DataSourceLoader

As described in https://www.devexpress.com/Support/Center/Question/Details/T758528/modify-the-datasourceloader-to-support-projection-as-part-of-the-original-query-operation and referenced threads - repeated below for simplicity
In a nutshell, the issue is this;
It is best practice in EF to return a DTO rather than the original object. Regardless of best practice, efficiency demands in so in my application as I have tables with large text fields that are not necessary for populating lists and would increase the size of the payload over 100x. I get the data for my lists using DataSourceLoader GET controllers, and I use filtering, sorting and grouping in the DataSourceLoader extensively. I ProjectTo to ensure that my payload from SQL to API, and my payload from API to client are efficient and contain no more data that is necessary.
At the moment, it is impossible to perform operations on the full set of object properties, but return only a subset using ProjectTo. Any property specified in the options e.g. a filter occurs after the ProjectTo, so the property is not available for filtering at that point in the SQL. As per the ticket, you cannot simply operate on the data after it is returned, as it breaks other elements of the returned set for more complex operations like grouping.
Also, a Select is not the answer as this requires far too much hard coding to move between types - this is what automapper and ProjectTo are for.
At the moment I have created a workaround that;

Returns the DataSourceLoader result if a Select is specified (obviously no projection is required in this case)
Programatically identifies the key of the original entity (e.g. User => UserId)
Runs the DataSourceLoader without Projecting, but returning only the Id of the entity - at this point I have all of the IDs matching the original query - IDset
Performs a simple where(x=>IDset.Contains(x=>[IDProperty])).ProjectTo()

This works, but it would be far better if the datasourceloader could be modified to append my projection so it occurs after the datasourceloader filtering / sorting / grouping. I can't see that this would require much modification.
My code below for anyone else with this issue.
var src = _context.Approval.Where(x =>
  x.ProjectId == _userService.Project_ID &&
  x.PublishDate != null &&
  x.NewApprovalId == null).Include(x => x.ApprovalTo);
var result = _context.FilterAsDto<Approval, ApprovalListDto>(src, loadOptions);

public LoadResult FilterAsDto<T, TDto>(Func<T, bool> preFilter, DataSourceLoadOptions loadOptions) where T : class
{
	var qryResult = DataSourceLoader.Load(Set<T>().Where(preFilter), loadOptions);
	if (loadOptions.Select == null || loadOptions.Select.Count()==0) return FilterAsDto<T, TDto>(qryResult, loadOptions);
	else return qryResult;

}

public LoadResult FilterAsDto<T, TDto>(IQueryable<T> sourceQuery, DataSourceLoadOptions loadOptions) where T : class
{

	var qryResult = DataSourceLoader.Load(sourceQuery, loadOptions);
	if (loadOptions.Select == null || loadOptions.Select.Count() == 0) return FilterAsDto<T, TDto>(qryResult, loadOptions);
	else return qryResult;
}

private LoadResult FilterAsDto<T, TDto>(LoadResult loadedData, DataSourceLoadOptions loadOptions) where T : class
{
	var pkey = Model.FindEntityType(typeof(T)).FindPrimaryKey().Properties.Select(n => n.Name).Single();
	var pKeyExp = Expression.Parameter(typeof(T));
	var pKeyProperty = Expression.PropertyOrField(pKeyExp, pkey);
	var keySelector = Expression.Lambda<Func<T, int>>(pKeyProperty, pKeyExp).Compile();

	if (loadedData.data is IEnumerable<Group>) return loadedData;
	else
	{
		var OriginalSummary = loadedData.summary;
		List<int> idList = loadedData.data.Cast<T>().Select(keySelector).ToList();

		var pKeyExpDto = Expression.Parameter(typeof(TDto));
		var pKeyPropertyDto = Expression.PropertyOrField(pKeyExpDto, pkey);
		var method = idList.GetType().GetMethod("Contains");
		var call = Expression.Call(Expression.Constant(idList), method, pKeyPropertyDto);
		var lambda = Expression.Lambda<Func<TDto, bool>>(call, pKeyExpDto);
		var defOptions = new DataSourceLoadOptionsBase();
		defOptions.Sort = loadOptions.Sort;
		defOptions.RequireTotalCount = loadOptions.RequireTotalCount;
		var returnData= DataSourceLoader.Load(Set<T>().ProjectTo<TDto>(_mapper.ConfigurationProvider).Where(lambda), defOptions);

		returnData.summary = OriginalSummary;
		returnData.totalCount = loadedData.totalCount;

		return returnData;
	}
}

Thank you for sharing your implementation.
Your points regarding DTOs in general and AutoMapper in particular are totally valid. The topic has been raised several times (#335, #338).
Do you mind if I change the subject to 'Support AutoMapper's ProjectTo in DataSourceLoader'?

No problem at all. Your repo :)

Hi Aleksey
Can we assume that your implementation will use Automapper, or at least read its mappings from the Automapper config? I would really recommend that just slots in automapper as this will provide simple and complex mapping that would form the basis for your projections, and the great majority of anyone projecting to Dtos will already have automapper definitions as there is no other sane way to do this. I expect that if you were rolling your own, you would only be able to do simple property name replacement in the Dto, and that would defeat the purpose because Dtos regularly use complex mappings e.g. here is one of mine
        CreateMap<LotQuantity, LotQuantityDto>()
            .ForMember(dest => dest.ApprovedByName, opt => opt.MapFrom(src => src.ApprovedByUser == null ? null : src.ApprovedByUser.FirstName + " " + src.ApprovedByUser.LastName))
            .ForMember(dest => dest.LotAdjDate, opt => opt.MapFrom(src => src.Lot == null ? src.ModifiedOn :
                src.Lot.DateRejected != null ? src.Lot.DateRejected :
                src.Lot.DateConf != null ? src.Lot.DateConf :
                src.Lot.DateGuar != null ? src.Lot.DateGuar :
                src.Lot.DateOpen));

Whether it is automapper or another system, you will need a config that deals with cases where the property names change. The automapper config would be ideal.
The question becomes, does the query specify property names using the Order or the OrderDto. While the consumer might realistically expect that all of their query would be in terms of the OrderDto, this is not the solution. The projection is happening after the filter and sort because in many cases the filter needs to operate on properties not available in the OrderDto, so properties in the query may contain at least some properties relative to Order, and not necessarily the OrderDto.
The problem then becomes that this negates the ability to sort on properties only contained in the OrderDto that are only available after the projection, including calculated properties such as OrderDto.SumValues.
One option might be (though it would be a bit of a design change) to introduce a new set of operators which can assume the ProductDto properties, something like
Change
{
    Filter:"",
    Sort:"",
    Group:""
}

To
{
    Filter:"",
    Sort:"",
    Group:""
    PostProjection : {
        Filter:"",
        Sort:"",
        Group:""
}

This would have stacks of advantages, and increase the power of the library considerably. It would also deal with the question of when and where to use Order vs OrderDto properties. You could even make the library smart enough that if the property in the base Filter (for example) is not present in Order, but does exist in OrderDto, then it is applied in PostProjection.Filter. This would ensure compatibility with the widgets (even backwards compatibility), and make the projection issues invisible on the client side.
E.g if a datagrid is showing a property OrderDto.SumValues, and the user filters on that column, the widget will send {"Filter",[["SumValues",">=", 10]]. If the library tests and determines that there is no property on Order called OrderValue, but there is a property on OrderDto, then it moves the property to PostProjection.Filter. Doing this with the ordering would fix issue https://github.com/DevExpress/DevExtreme.AspNet.Data/issues/388
This would eliminate the need for any property substitutions, and everything could be done with automapper. No changes would be necessary for the existing widgets, as everything is done in the library.
This would also allow clientside stores to compile complex queries and sorts either pre or post projection (or both)

Thanks @statler for your detailed reply. Now it's clear that member renaming is not sufficient.

Can we assume that your implementation will use Automapper, or at least read its mappings from the Automapper config?

I'm inclined to think that it would be better to integrate with Automapper in a separate library or a plugin. Your code snippets are a good illustration of how this can be done. However, I see that there's a need for built-in projection support, so that developers don't need to manually access/cast/iterate LoadResult.data.
I don't think that we want to make the library smart enough to automatically handle various mapping options and edge cases.

I figured the automatic handling of mapping would be out, but it seems to me that the inclusion of a post projection option would be really quite easy and would not break anything in the existing architecture. All that would be required would be;
On the javascript side allow for the specification of additional options e.g.
{
    PostProjection : {
        Filter:"",
        Sort:"",
        Group:""
}

In the C# library, simply running through the expression tree code a second time, just with the LoadOptions changed to only include the PostProjection Filter, Sort and Group. The easiest way to do this would probably be to create a generic overload of Load (Load).
I would do a PR for it, but I can't get my head arount the grouping and where I can insert into the code so that I am always applying the projection to the IQueryable rather than the Group - Also, some of the Expression work is a bit different to how I work trees :(

it seems to me that the inclusion of a post projection option would be really quite easy

It seems so. However, as you noticed, grouping is a tough subject. Also, interoperability with existing options (Select, PreSelect, etc) is important.

I would do a PR for it

PRs are welcome. If you do, please include unit tests. I think it's essential to test how well Automapper works with SQL translation. Refer to the recent ticket on this topic.

On the javascript side allow for the specification of additional options

Do you need to control projections from the client side? Isn't it sufficient to control mappings on the server?

As per the example, the additional options are for specifying Post-projection filter, sort and group.
{
PostProjection : {
    Filter:"",
    Sort:"",
    Group:""
}

This solves several issues;

When property names change during the projection. This makes it clear where the sort/filter/group should be configured - after the projection or before it. The original filter/sort/group properties are unambiguously applying to the original, unprojected object. Postprojection filter/sort/group occur after.
When the projection introduces new calculated fields. The filter / sort / group can be applied to the calculated field, not just the original object fields
Both applications of the filter/sort/group can occur prior to calculations for attributes on the LoadResult

What are the data types of the Filter, Sort, Group properties in the suggested PostProjection object? Empty strings don't describe them well enough.

Same as Filter, Sort and Group in the base level of the options. When it gets to the server, you would invoke EXACTLY the same expressioncompilers you do for a normal filter, sort , group - you just do it after the projection e.g.
dataSource.postProjection.filter([
    [ "value", ">", 3 ],
    "and",
    [ "value", "<", 7 ]
]);

 dataSource.postProjection.sort([
        { getter: "firstName", desc: true },
        "lastName"
     ])

 dataSource.postProjection.group(["birthYear", "gender"])

I see the following arguments against such an API:

DevExtreme DataSource and load options are part of the DevExtreme data layer interface. Unlike sorting or paging, the concept of server-side projection is too specific to the particular implementation. For example, it cannot be applied to ArrayStore of ODataStore.

I think that DTO mappings should not leak to the browser. They are server-side implementation detail. If they are coded via Automapper configuration, it makes no sense to pass them with load parameters. Of course, a specific app may use this, but at the library level, it would be a poor design.

Consider a data grid sorted by two columns. The first column belongs to the original model, and the second column is a projection or a computed property. If I understand correctly, this will imply:
{
    sort: [ { selector: "PropOfModel" } ]
    postProjection: {
        sort: [ { selector: "PropOfDTO" } ]
    }
}

In case of two-pass Load as suggested, the 'post-projected' Sort will reset the first-pass Sort. Unlike Filter, Sort and Group cannot be easily divided and ruled.

Actually, filter can also be problematic. Example - grid's search panel generates or groups that cannot be separated:
[
  ["PropOfModel","contains","abc"],
  "or",
  ["PropOfDTO","contains","abc"]
]

I also have need for this implementation. While we can use ProjectTo to with the DataSourceLoader.Load() method to list data for the grid we can't filter nor order by many of the columns.

Hello @Arafel-BR

I also have need for this implementation.

Do you mean the specific 'post-projection' idea discussed above or support for ProjectTo in general?

we can't filter nor order by many of the columns.

Do you have a code sample or a project that illustrates the issue? You can share it here or via Support Center.

Hi Aleksey
I have come around to the conclusion that you are correct on the post-projection filtering. Not only;

Actually, filter can also be problematic. Example - grid's search panel generates or groups that cannot be separated:

but also, it will cause paging problems - because the paging would happen before the post-projection and then the subsequent operation would only work on a subset of the data.
That said, it would still be handy to be able to specify an automap projection for the original reasons. All this would need to do would be to apply the ProjectTo method to the end of the expression
E.g.
var qryResult = DataSourceLoader.LoadAndProject<MyDto>(sourceQuery, options, _mapper.ConfigurationProvider);

It would be relatively easy to implement I think. While it introduces an additional dependency, the benefit is immense.

Hi Aleksey
Me again. Further to your comment in #378 I have looked into the CustomAccessorCompiler as an option.
The CustomAccessorCompiler is awesome! It will actually resolve the trivial cases like we discuss in #378, but I am still at an impasse with more complex sort scenarios - and with the exact same problem for grouping.
My specific example at the moment is - I have a field called Status in the Dto, that is calculated like this:
        public bool IsApproved         {             get             {                 return (ApprovalDate != null || ApprovalsCount > 0);             }         }         public bool IsClosedOut         {             get             {                 return CloseOutDate != null;             }         }         public string Status => IsClosedOut ? "Closed Out" : (IsApproved ? "Approved" : "Open");
It is not even a direct result of the projection. With the filtering, this is simple to resolve with the CustomFilterCompilers.RegisterBinaryExpressionCompiler like this;
	CustomFilterCompilers.RegisterBinaryExpressionCompiler(info => 	{ 		if (String.IsNullOrEmpty(info.AccessorText)) 			return Expression.Constant(true); 		if (info.DataItemExpression.Type == typeof(Ncr)) 		{ 			if (info.AccessorText == "Status" && info.Operation == "=") 			{ 				ExpressionStarter<Ncr> predicate=null; 				if (info.Value.ToString() == "Closed Out") predicate = CompileWhereExpression<Ncr>(info, t => t.CloseOutDate != null); 				if (info.Value.ToString()=="Approved") predicate = CompileWhereExpression<Ncr>(info, t=>(t.CloseOutDate == null) && t.ApprovalDate != null); 				if (info.Value.ToString() == "Open") predicate = CompileWhereExpression<Ncr>(info, t => (t.CloseOutDate == null) && t.ApprovalDate == null); 				if (predicate!=null) return predicate.Body; 				return null; 			} 		} 	}
However, because the grouping is done before the projection, I can see no way that I can get the grouping to work. Nor can I see a way to use the CustomAccessorCompiler for the sort.
Would it be at all possible just to either:

Make an equivalent for CustomFilterCompilers.RegisterBinaryExpressionCompiler for sorting and grouping? E.g. CustomSortCompilers.RegisterBinaryExpressionCompiler and CustomGroupCompilers.RegisterBinaryExpressionCompiler
Replace the CustomFilterCompilers.RegisterBinaryExpressionCompiler with a method that allows customizing the expression for any operation?

I see option 2 as being quite powerful. You could add an additional parameter that is an enum describing what operation the expression is being used in.
Right now, any column that is displayed in a grid that is calculated through projection cannot be grouped, and not sorted - unless relatively trivial.

Nor can I see a way to use the CustomAccessorCompiler for the sort.

CustomAccessorCompilers.Register((target, accessorText) => {
    if(target.Type == typeof(Ncr) && accessorText == "Status") {
        return Expression.Condition(
            Expression.Equal(Expression.PropertyOrField(target, "CloseOutDate"), Expression.Constant(null)),
            Expression.Constant("Closed Out"),
            Expression.Condition(
                Expression.Or(
                    Expression.NotEqual(Expression.PropertyOrField(target, "ApprovalDate"), Expression.Constant(null)),
                    Expression.GreaterThan(Expression.PropertyOrField(target, "ApprovalsCount"), Expression.Constant(0))
                ),
                Expression.Constant("Approved"),
                Expression.Constant("Open")
            )
        );
    }

    return null;
});

Resulting expressions:
.OrderBy(obj => IIF((obj.CloseOutDate == null), "Closed Out", IIF(((obj.ApprovalDate != null) Or (obj.ApprovalsCount > 0)), "Approved", "Open")))

.GroupBy(obj => new AnonType`1(I0 = IIF((obj.CloseOutDate == null), "Closed Out", IIF(((obj.ApprovalDate != null) Or (obj.ApprovalsCount > 0)), "Approved", "Open"))))
.OrderBy(g => g.Key.I0)
.Select(g => new AnonType`2(I0 = g.Count(), I1 = g.Key.I0))

However, I'm not sure whether LINQ providers will be able to translate these into SQL.

OK, so that is officially a working answer. It does in fact successfully transpose the LINQ to SQL. Fantastic work - very much appreciated. The SQL is
SELECT [n].[NCRID], [n].[Description], [n].[Disposition], [n].[HRid], [n].[Location], [n].[ModifiedBy], [n].[ModifiedOn], [n].[NCRCost], [n].[NCRNo], [n].[Notes], [n].[OptimisticLockField], [n].[PreventativeAction], [n].[ProjectID], [n].[RaisedBy], [n].[RelatedParties], [n].[RootCauseCategory], [n].[RootCauseDetail], [n].[Severity], [n].[ThirdPartyAppReqd], [n].[UniqueID]
FROM [NCR] AS [n]
WHERE [n].[ProjectID] = 1
ORDER BY CASE
	WHEN [n].[CloseOutDate] IS NOT NULL
	THEN N'Closed Out' ELSE CASE
		WHEN [n].[ApprovalDate] IS NOT NULL
		THEN N'Approved' ELSE N'Open'
	END
END DESC

So, a little snag - this works perfectly for the sort operation, but the grouping returns the original dataset, not the projection - somehow this breaks ProjectTo.
Any way come to mind where we can make this work? As it stands, you would have to run this through the projection clientside whenever a grouping is used, which isnt ideal

I have ended up writing extension methods that are used as follows
await Devex.DataSourceLoader.LoadDtoAsync<T, TDto>(_context, _mapper, ProjectEntities, loadOptions)

When using this code you need to explicitly deal with any properties that may be present in the Dto but not the base object using CustomAccessorCompiler, but only if they may be filtered, grouped or sorted. If you do implement CustomAccessorCompiler  on these properties, all issues with grouping, filtering and sorting just disappear and the AspNet.data projection Just Works.
The code for these is below (includes some sync methods too, and overloads for expressions);
using AutoMapper;
using AutoMapper.QueryableExtensions;
using cpDataASP.DevExtreme.AspNet.Data;
using DevExtreme.AspNet.Data;
using DevExtreme.AspNet.Data.ResponseModel;
using Microsoft.EntityFrameworkCore;
using Newtonsoft.Json;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Linq.Expressions;
using System.Threading.Tasks;
using AspNet= DevExtreme.AspNet.Data;

namespace cpDataASP.Devex
{
	public class DataSourceLoader
	{
		public static LoadResult LoadDto<T, TDto>(DbContext context, IMapper _mapper, IQueryable<T> source, DataSourceLoadOptions options)
			where T : class
		{
			var qryResult = AspNet.DataSourceLoader.Load(source, GetModelOptions<T, TDto>(options));
			if (options.Select == null || options.Select.Count() == 0)
			{
				options.Filter = null;
				return FilterAsDto<T, TDto>(context, _mapper, qryResult, options);
			}
			else return qryResult;
		}

		public static LoadResult LoadDto<T, TDto>(DbContext context, IMapper _mapper, Expression<Func<T, bool>> preFilter, DataSourceLoadOptions options)
	where T : class
		{
			var qryResult = AspNet.DataSourceLoader.Load(context.Set<T>().Where(preFilter), GetModelOptions<T, TDto>(options));
			if (options.Select == null || options.Select.Count() == 0)
			{
				options.Filter = null;
				return FilterAsDto<T, TDto>(context, _mapper, qryResult, options);
			}
			else return qryResult;
		}

		static DataSourceLoadOptions GetModelOptions<T, TDto>(DataSourceLoadOptions originalOptions)
		{
			var modelOptions = Clone(originalOptions);
			//if (modelOptions.Group != null) modelOptions.Sort = modelOptions.Group;
			//modelOptions.Group = null;
			return modelOptions;
		}

		static T Clone<T>(T source)
		{
			var serialized = JsonConvert.SerializeObject(source, Formatting.Indented,
				new JsonSerializerSettings
				{
					ReferenceLoopHandling = ReferenceLoopHandling.Ignore
				});
			return JsonConvert.DeserializeObject<T>(serialized);
		}

		private static LoadResult FilterAsDto<T, TDto>(DbContext context, IMapper _mapper, LoadResult loadedData, DataSourceLoadOptions loadOptions) where T : class
		{
			Expression<Func<TDto, bool>> lambda = GetDtoByIdPredicate<T, TDto>(context, loadedData);
			var returnData = AspNet.DataSourceLoader.Load(context.Set<T>().ProjectTo<TDto>(_mapper.ConfigurationProvider).Where(lambda), loadOptions);

			return returnData;
		}

		public async static Task<LoadResult> LoadDtoAsync<T, TDto>(DbContext context, IMapper _mapper, IQueryable<T> source, DataSourceLoadOptions options)
			where T : class
		{
			//var qryResult = AspNet.DataSourceLoader.Load(source, GetModelOptions<T, TDto>(options));
			var qryResult = await AspNet.DataSourceLoader.LoadAsync(source, GetModelOptions<T, TDto>(options));
			if (options.Select == null || options.Select.Count() == 0)
			{
				options.Filter = null;
				return await FilterAsDtoAsync<T, TDto>(context, _mapper, qryResult, options);
			}
			else return qryResult;
		}

		public async static Task<LoadResult> LoadDtoAsync<T, TDto>(DbContext context, IMapper _mapper, Expression<Func<T, bool>> preFilter, DataSourceLoadOptions options)
	where T : class
		{
			var qryResult = await AspNet.DataSourceLoader.LoadAsync(context.Set<T>().Where(preFilter), GetModelOptions<T, TDto>(options));
			//var qryResult = AspNet.DataSourceLoader.Load(context.Set<T>().Where(preFilter), GetModelOptions<T, TDto>(options));
			if (options.Select == null || options.Select.Count() == 0)
			{
				options.Filter = null;
				return await FilterAsDtoAsync<T, TDto>(context, _mapper, qryResult, options);
			}
			else return qryResult;
		}

		private async static Task<LoadResult> FilterAsDtoAsync<T, TDto>(DbContext context, IMapper _mapper, LoadResult loadedData, DataSourceLoadOptions loadOptions) where T : class
		{
			Expression<Func<TDto, bool>> lambda = GetDtoByIdPredicate<T, TDto>(context, loadedData);
			var returnData = await AspNet.DataSourceLoader.LoadAsync(context.Set<T>().ProjectTo<TDto>(_mapper.ConfigurationProvider).Where(lambda), loadOptions);

			return returnData;
		}

		private static Expression<Func<TDto, bool>> GetDtoByIdPredicate<T, TDto>(DbContext context, LoadResult loadedData) where T : class
		{
			var pkey = context.Model.FindEntityType(typeof(T).FullName).FindPrimaryKey().Properties.Select(n => n.Name).Single();
			var pKeyExp = Expression.Parameter(typeof(T));
			var pKeyProperty = Expression.PropertyOrField(pKeyExp, pkey);
			var keySelector = Expression.Lambda<Func<T, int>>(pKeyProperty, pKeyExp).Compile();

			List<int> idList = loadedData.data.Cast<T>().Select(keySelector).ToList();

			var pKeyExpDto = Expression.Parameter(typeof(TDto));
			var pKeyPropertyDto = Expression.PropertyOrField(pKeyExpDto, pkey);
			var method = idList.GetType().GetMethod("Contains");
			var call = Expression.Call(Expression.Constant(idList), method, pKeyPropertyDto);
			var lambda = Expression.Lambda<Func<TDto, bool>>(call, pKeyExpDto);
			return lambda;
		}
	}
}

As @Arafel-BR even I also need this kind of feature. Maybe as a plugin library if DevExpress doesn't want to make this beautiful library smarter.
I am going to explain my problem:
I'm trying to create a project following the "Clean Architecture" so I've four layers: Domain, Infrastructure, Application, User interface (currently Blazor) where:
Infrastructure has a dependency on Domain and Application.
Application has a dependency on Domain.
Domain has no dependency.
User interface has a dependency on Application
This architecture gives me the possibility to work inside the domain layer using domain entities, domain language and so on. The application layer has the goal of receiving command/query with parameters, translate the request to the domain languages and finally translate the result in a specific DTO for the request's result.
This means that the User interface layer works with the Application layer and I can modify the Domain Layer (for example changing a domain entity) without break the User interface.
Currently, if I use DevExtreme.AspNet.Data I have to take a decision on which roads I want to walk:
The easiest road:
break the Clean Architecture and give the User Interface a dependency on the Domain layer so I can use all the functionalities of DatasourceLoader like applying filtering/grouping/so on directly on the SQL query.
Side effect: I can remove the Application layer because now I don't have anymore an abstraction between the User interface and the Domain layer. This is a huge side effect because what if I have to add another user interface, like a mobile (android/ios) and let it communicate with my system through controllers? I will have to replicate a lot of code. But the worst thing is that I can't translate the result in a specific DTO why I can't pass a DTO as loadOptions. Yeah, I could map the DTO loadOptions but the reason for choosing this road is avoiding this step.
The road that I would like:
maintaining the Clean Architecture, so maintaining an abstract layer between the User interface and the domain layer.
Side effect: DataSourceLoader now works on a data source that I've previously prepared and translated using a DTO model. Using this way I lost all the benefit of applying filtering/grouping/so on directly on the query. This will be a huge problem when my database will grow.
@AlekseyMartynov, @statler I would like to ask you if there is already a plugin library.
Note: even I am using AutoMapper.

As @Arafel-BR even I also need this kind of feature. Maybe as a plugin library if DevExpress doesn't want to make this beautiful library smarter.
I am going to explain my problem:
I'm trying to create a project following the "Clean Architecture" so I've four layers: Domain, Infrastructure, Application, User interface (currently Blazor) where:
Infrastructure has a dependency on Domain and Application.

Application has a dependency on Domain.
Domain has no dependency.
User interface has a dependency on Application

This architecture gives me the possibility to work inside the domain layer using domain entities, domain language and so on. The application layer has the goal of receiving command/query with parameters, translate the request to the domain languages and finally translate the result in a specific DTO for the request's result.
This means that the User interface layer works with the Application layer and I can modify the Domain Layer (for example changing a domain entity) without break the User interface.
Currently, if I use DevExtreme.AspNet.Data I have to take a decision on which roads I want to walk:
The easiest road:
break the Clean Architecture and give the User Interface a dependency on the Domain layer so I can use all the functionalities of DatasourceLoader like applying filtering/grouping/so on directly on the SQL query.
Side effect: I can remove the Application layer because now I don't have anymore an abstraction between the User interface and the Domain layer. This is a huge side effect because what if I have to add another user interface, like a mobile (android/ios) and let it communicate with my system through controllers? I will have to replicate a lot of code. But the worst thing is that I can't translate the result in a specific DTO why I can't pass a DTO as loadOptions. Yeah, I could map the DTO loadOptions but the reason for choosing this road is avoiding this step.
The road that I would like:
maintaining the Clean Architecture, so maintaining an abstract layer between the User interface and the domain layer.
Side effect: DataSourceLoader now works on a data source that I've previously prepared and translated using a DTO model. Using this way I lost all the benefit of applying filtering/grouping/so on directly on the query. This will be a huge problem when my database will grow.
@AlekseyMartynov, @statler I would like to ask you if there is already a plugin library.
Note: even I am using AutoMapper.

@Blackleones

Using this way I lost all the benefit of applying filtering/grouping/so on directly on the query. This will be a huge problem when my database will grow.

Automapper allows you to keep these benefits:

Map via AutoMapper.QueryableExtensions.ProjectTo
In the App layer, use IQueryable<DTO>, not IEnumerable<DTO> or IList<DTO>
Keep the mappings simple, declare them via Expressions

class OrderDTO {
    public int ID { get; set; }
    public DateTime? Date { get; set; }
}

class AppImpl {
    NorthwindContext _nwind;

    public AppImpl(NorthwindContext nwind) {
        _nwind = nwind;
    }

    IMapper _mapper = new MapperConfiguration(cfg => cfg
        .CreateMap<Order, OrderDTO>()
        .ForMember(vm => vm.ID, m => m.MapFrom(o => o.OrderId))
        .ForMember(vm => vm.Date, m => m.MapFrom(o => o.OrderDate))
    ).CreateMapper();

    public IQueryable<OrderDTO> GetDataForView123() {
        return _nwind.Orders.ProjectTo<OrderDTO>(_mapper.ConfigurationProvider);
    }
}

var loadResult = DataSourceLoader.Load(app.GetDataForView123(), new DataSourceLoadOptions {
    Filter = new[] { "Date", ">", "2011-11-11" },
    Sort = new[] { new SortingInfo { Selector = "Date" } },
    Take = 10
});

SQL:
exec sp_executesql N'SELECT TOP(@__p_0) [dtoOrder].[OrderDate] AS [Date], [dtoOrder].[OrderID] AS [ID]
FROM [Orders] AS [dtoOrder]
WHERE [dtoOrder].[OrderDate] > ''2011-11-11T00:00:00.000''
ORDER BY [Date], [ID]',N'@__p_0 int',@__p_0=10

@AlekseyMartynov this is a smart solution. I've tried it and it works!
So the idea behind this code is:

map the SQL fields with the DTO
apply filters over DTO fields
execute the query

right?

Correct. For a more detailed description, check the relevant Automapper docs.

I was so happy to have read this issue. I can't express how happy I am to see a work-around. Great work to @statler for the brains and grunt work, and thank you @AlekseyMartynov for providing direction where it was necessary and remaining active.
I would like to ask if it's possible to have a complete DevExpress example created, illustrating this workaround in the same fashion as the other examples?

This is now available out of the box with the fork at https://github.com/statler/DevExtreme.AspNet.Data


Bug: broken image link in the quick-start guide

Your issue may already be reported! Please search on the issue tracker before creating a new one.
Environment

Client OS (e.g., Linux, macOS, Windows): macOS

Affected Component

[ ] Approval
[ ] Bridge
[ ] CLI
[ ] Datastore
[X] Docs
[ ] Distributor / Event Broker
[ ] Helm
[ ] Istio
[ ] Jmeter
[ ] Kubernetes Integration
[ ] Openshift Integration
[ ] REST API
[ ] Webhook

Describe the bug

The image link in the Outcome section of the quickstart guide is broken
To Reproduce

Steps to reproduce the behavior:

Go to https://keptn.sh/docs/quickstart/
Scroll down to Outcome section
A broken image with alt text keptn hollo world will be visible
See error

Screenshots

Additional context

outdated


Manifest merger failed

Execution failed for task ':app:processReleaseMainManifest'.

Manifest merger failed : Attribute meta-data#android.support.FILE_PROVIDER_PATHS@resource value=(@xml/filepaths) from [:image_picker] AndroidManifest.xml:15:17-72
is also present at [com.github.ishaquehassan:AndroidDocumentScanLibrary:1.8.2] AndroidManifest.xml:36:17-55 value=(@xml/provider_paths).
Suggestion: add 'tools:replace="android:resource"' to  element at AndroidManifest.xml to override.

@Amrokh, you need to add the suggestion to <provider> tag.
Here's mine:
<provider
            android:name="androidx.core.content.FileProvider"
            android:authorities="${applicationId}.flutter_inappwebview.fileprovider"
            tools:replace="android:authorities"
            android:exported="false"
            android:grantUriPermissions="true">
            <meta-data
                android:name="android.support.FILE_PROVIDER_PATHS"
                android:resource="@xml/provider_paths" />
        </provider>

Don't forget to add xmlns:tools="http://schemas.android.com/tools" as well to the  tag.

@Amrokh, you need to add the suggestion to <provider> tag.
Here's mine:
<provider
            android:name="androidx.core.content.FileProvider"
            android:authorities="${applicationId}.flutter_inappwebview.fileprovider"
            tools:replace="android:authorities"
            android:exported="false"
            android:grantUriPermissions="true">
            <meta-data
                android:name="android.support.FILE_PROVIDER_PATHS"
                android:resource="@xml/provider_paths" />
        </provider>

Don't forget to add xmlns:tools="http://schemas.android.com/tools" as well to the <manifest> tag.

Should I add this in my android>app>src>main>AndroidManifest.xml ? as I've done that and still doesnt work.

as I've done that and still doesnt work.

@miguelflores1993 please refer this https://github.com/ishaquehassan/document_scanner_flutter/issues/19#issuecomment-1579374629


ref doc format fix on backup page

missing a + for property.<propertyName> (V1), extraProperties (V2)
what it looks like now https://solr.apache.org/guide/solr/latest/deployment-guide/collection-management.html#backup-parameters

Checklist
Please review the following and check all that apply:

[ ] I have reviewed the guidelines for How to Contribute and my code conforms to the standards described there to the best of my ability.
[ ] I have created a Jira issue and added the issue ID to my pull request title.
[ ] I have given Solr maintainers access to contribute to my PR branch. (optional but recommended)
[ ] I have developed this patch against the main branch.
[ ] I have run ./gradlew check.
[ ] I have added tests for my changes.
[ ] I have added documentation for the Reference Guide

Thanks for this nice fix!


Manually fuse broadcast expressions in diag edmf

This PR fuses a few more broadcast expressions in diagnostic edmf.

The jobs that are failing are with precip_model: nothing. So maybe there is some issue with the empty field?

Yeah, I was afraid that might not work. I think there should be an easy enough work-around.


Version 1.5.0 in GitHub differs from version from Arduino IDE library manager 1.5.0.

Hello,
The version 1.5.0 in GitHub differs from the version from Arduino IDE library manager 1.5.0.
This is probably because there are some updates in GitHub without an updated version in the library.properties file.
I assume that GitHub is correct (latest) (the difference is in the initialisation of the stop bits in the files DMXSerial_avr.h and DMXSerial_megaavr.h).
Question: Why are bytes with one stop bit now accepted (DXM standard is two stop bits)?
Great library by the way!
Kind regards,
Johan.

sometimes I do changes without doing a final test so Github is more advanced than the version from the library manager - not to confuse those people just using the release version.
When you like to support - as you do - you should use the github version (git clone into the library folder).
The DMX sender will always have to send 2 stop bytes. The receiver here already accepts the data after the first stop byte - not waiting for the second one - and just taking a small pause to the next byte. That's compatible on the receiving party.

Hi,
Just wondering: The IDE version uses SERIAL_8N1 (and not SERIAL_8N2) in the below cases. Is this then correct/compatible with the DMX specification?
/// Initialize the Hardware UART serial port registers to the required mode.
void _DMX_setMode(DMXUARTMode mode)
{
...
} else if (mode == DMXUARTMode::TDATA) {
...
UCSRnC = SERIAL_8N2; // send with 2 stop bits for compatibility
} else if (mode == DMXUARTMode::TDONE) {
...
UCSRnC = SERIAL_8N2; // send with 2 stop bits for compatibility
} // if
} // _DMX_setMode()
Kind regards.


Connecting to Gerrit behind Apache server as proxy?

Hello,
is it possible that the Gerrit plugin (installed in PyCharm) accesses Gerrit server that is proxied by a front-end HTTP server (the obtained credentials are passed through to the Gerrit server)?
The reason I ask is the following:
I access my Gerrit server (port 8080) through Apache proxy server (port 8081). I succeeded (finally!) to get Gerrit Web UI appear in browser after analyzing HTTP traffic between the browser and Apache server and configuring the latter appropriately (I mean: it works now at least with the browser).
After that I configured Gerrit plugin in PyCharm with the same data as for the browser (http://host:proxy_port, username, password), but I did not succeed to connect properly to Gerrit (using "Test" button).
If this helps, I attach the HTTP message exchange between the Gerrit plugin and Apache server during testing the connection:
`

GET /login/ HTTP/1.1
Host: raspberrypi-4b.local:8081
Connection: Keep-Alive
User-Agent: gerrit-intellij-plugin/1.2.6-203 using gerrit-rest-java-client/0.9.4 using Apache-HttpClient/4.5.13 (Java/11.0.15)
Accept-Encoding: gzip,deflate
Authorization: Basic xxxxxxxxxxxxxxxx

<<< HTTP/1.1 302 Found
Date: Sat, 04 Mar 2023 00:33:19 GMT
Server: Apache/2.4.38 (Raspbian)
X-Frame-Options: DENY
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Location: http://raspberrypi-4b.local/
Content-Length: 0
Set-Cookie: GerritAccount=yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy; Path=/; Expires=Sat, 04-Mar-2023 12:33:19 GMT; Max-Age=43200
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive

GET / HTTP/1.1
Host: raspberrypi-4b.local
Connection: Keep-Alive
User-Agent: gerrit-intellij-plugin/1.2.6-203 using gerrit-rest-java-client/0.9.4 using Apache-HttpClient/4.5.13 (Java/11.0.15)
Cookie: GerritAccount=yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
Accept-Encoding: gzip,deflate

<<< HTTP/1.1 200 OK
Date: Sat, 04 Mar 2023 00:33:19 GMT
Server: Apache/2.4.38 (Raspbian)
Last-Modified: Fri, 22 Jul 2022 20:20:33 GMT
ETag: "29cd-5e46a90f6a844-gzip"
Accept-Ranges: bytes
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 3041
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html

POST /login/ HTTP/1.1
Content-Length: 30
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Host: raspberrypi-4b.local:8081
Connection: Keep-Alive
User-Agent: gerrit-intellij-plugin/1.2.6-203 using gerrit-rest-java-client/0.9.4 using Apache-HttpClient/4.5.13 (Java/11.0.15)
Cookie: GerritAccount=yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
Accept-Encoding: gzip,deflate
Authorization: Basic xxxxxxxxxxxxxxxx

<<< HTTP/1.1 405 Method Not Allowed
Date: Sat, 04 Mar 2023 00:33:19 GMT
Server: Apache/2.4.38 (Raspbian)
X-Frame-Options: DENY
Content-Type: text/plain;charset=iso-8859-1
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Content-Length: 18
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive

GET /a/changes/?q=reviewer:self&n=1 HTTP/1.1
Accept: application/json
Host: raspberrypi-4b.local:8081
Connection: Keep-Alive
User-Agent: gerrit-intellij-plugin/1.2.6-203 using gerrit-rest-java-client/0.9.4 using Apache-HttpClient/4.5.13 (Java/11.0.15)
Cookie: GerritAccount=yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
Accept-Encoding: gzip,deflate
Authorization: Basic xxxxxxxxxxxxxxxx

<<< HTTP/1.1 401 Unauthorized
Date: Sat, 04 Mar 2023 00:33:19 GMT
Server: Apache/2.4.38 (Raspbian)
WWW-Authenticate: Basic realm="Gerrit Code Review"
Content-Type: text/plain;charset=iso-8859-1
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Content-Length: 12
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
`
This is the end of the communication, the connection test failed.

It is quite hard to say what fails here. Just wondering: what are you reasons for not using a standard Gerrit setup?

Resolved.
Initially, I could not access Gerrit server at its native 8080 port for then-to-me-unknown reason, so I switched to experiment with the proxy port and specifics of the related HTTP authorization.
But looking into gerrit.config file again I saw I set httpd.listenUrl property as proxy-http://<IP_ADDRESS>:8080/, so Gerrit server was not accepting any requests from other hosts (which was the exact intention at the time I succeeded to hide Gerrit Web UI behind Apache HTTP server - they are installed at the same host).  When I updated the property to proxy-http://*:8080/ and restarted the server, the connection test passed successfully.
My bad, sorry for bothering you. In the company we introduced Gerrit, so I decided to install all the relevant services at home in my local network, so I can err carefree while learning. But the installation and the usage are two different things. And I am quite a newbie in this sysadmin stuff...
I have now another question regarding Gerrit plugin, but I will ask it in a separate issue for the sake of not mixing topics.


Adjust update of log driver configuration

The syno_docker_update.sh script updates the file /var/packages/Docker/etc/dockerd.json to fix an issue with Synology's db log driver. However, the script replaces the entire configuration file, instead of fixing the offending log driver only. This might introduce unwanted side effects.

Getting Error response from daemon: failed to initialize logging driver: failed to get logging factory: logger: no log driver named 'db' is registered: error looking up logging plugin db: plugin "db" not found error after docker update and trying to start container

Can you share the full log of the syno_docker_update.sh command you ran? And what is the current content of your /var/packages/Docker/etc/dockerd.json file?

Update Docker Engine and Docker Compose on Synology to target version

Current DSM version: 6.2.3
Current Docker version: Unknown
Current Docker Compose version: Unknown
ERROR: Could not detect current Docker version, use --force to override
vitalii@DS220plus:~/github/synology-docker$ sudo ./syno_docker_update.sh update
Update Docker Engine and Docker Compose on Synology to target version

Current DSM version: 6.2.3
Current Docker version: 18.09.8
Current Docker Compose version: 1.24.0
Target Docker version: 19.03.13
Target Docker Compose version: 1.27.4

WARNING! This will replace:
  - Docker Engine
  - Docker Compose
  - Docker daemon log driver

Are you sure you want to continue? [y/N] y
Step 1 from 10: Downloading target Docker binary (https://download.docker.com/linux/static/stable/x86_64/docker-19.03.13.tgz)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 59.3M  100 59.3M    0     0  10.3M      0  0:00:05  0:00:05 --:--:-- 10.9M
Step 2 from 10: Downloading target Docker Compose binary (https://github.com/docker/compose/releases/download/1.27.4/docker-compose-Linux-x86_64)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   651  100   651    0     0   4691      0 --:--:-- --:--:-- --:--:--  4717
100 11.6M  100 11.6M    0     0  2430k      0  0:00:04  0:00:04 --:--:-- 2615k
Step 3 from 10: Stopping Docker service
pkgctl-Docker stoped.
Step 4 from 10: Backing up current Docker binaries (/volume1/homes/vitalii/github/synology-docker/docker_backup_20201130_140242.tgz)
bin/
bin/docker
bin/docker-proxy
bin/containerd
bin/docker-compose
bin/auplink
bin/docker-init
bin/dockerd
bin/containerd-shim
bin/ctr
bin/runc
dockerd.json
start-stop-status
Step 5 from 10: Extracting target Docker binary (/tmp/docker_update/docker-19.03.13.tgz)
docker/
docker/ctr
docker/runc
docker/containerd-shim
docker/docker-init
docker/docker-proxy
docker/docker
docker/dockerd
docker/containerd
Step 6 from 10: Installing binaries
Step 7 from 10: Configuring log driver
Step 8 from 10: Enabling IP forwarding
Step 9 from 10: Starting Docker service
pkgctl-Docker started.
Step 10 from 10: Cleaning the temp folder
Done.

{
    "data-root" : "/var/packages/Docker/target/docker",
    "log-driver" : "json-file",
    "registry-mirrors" : [],
    "group": "administrators"
}

Both the script and daemon configuration appear to be fine. Can you try stopping and restarting the container(s)? It seems to refer to the now obsolete db log driver.

yep, remove old container and add it again helps
thank you

Great! I'll add an instruction to the README too ;-)

Start container portainer failed: {"message":"failed to initialize logging driver: failed to get logging factory: logger: no log driver named 'db' is registered: error looking up logging plugin db: plugin "db" not found"}.
No any variants of  /var/packages/Docker/etc/dockerd.json from Updated binaries are incompatible with btrfs #22 not helped.
Recreating containers have the same error, i cant run pull any container, can`t delete images.
Current DSM version: 6.2.3
Current Docker version: 19.03.14
Current Docker Compose version: 1.27.4

restore is ok.

hi @exotsk, to help me identify potential causes of your issue, would you mind submitting a new issue using the bug report template?


What-Zit-Tooya-Ad-Block

CC  @What-Zit-Tooya
@StevenBlack please review and merge if you like.

I ping maintainer, maybe he response 😉  @dnmTX

He is using VPN to block IP's.That's why bunch of them are present in his hosts list:

And also this(front page):

....so i'd say......HARD NOPE 👎

He is using VPN on his phone to block IP's.That's why bunch of them are present in his hosts list:

And also this(front page):

That IPs from CNAME adservers address from Adguard log.

....so i'd say......HARD NOPE 👎

Im using hosts that i need, analyze it and block it, if it wrong, i remove it.
Only using what i need.
Nah. im still n00b in this filter, still learning.

Looking at this broadly,
$ ghosts -c https://raw.githubusercontent.com/What-Zit-Tooya/Ad-Block/main/Main-Blocklist/Ad-Block-HOSTS.txt
----------------------------------------
Base hosts file summary:
----------------------------------------
Location: https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
Domains: 81,745
Bytes: 2.5 MB
----------------------------------------
----------------------------------------
Compared hosts file summary:
----------------------------------------
Location: https://raw.githubusercontent.com/What-Zit-Tooya/Ad-Block/main/Main-Blocklist/Ad-Block-HOSTS.txt
Domains: 1,671
Bytes: 54 kB
----------------------------------------
Intersection: 183 domains

@dnmTX @StevenBlack  https://github.com/What-Zit-Tooya/Ad-Block/commit/ab0155d9b9e8783c9af781dc1bebbba48a164773

Up to Steve @StevenBlack  ¯_(ツ)_/¯
That kid's attitude...not really lines up to be close to a responsible lists curator. In my opinion.

I pushed latest changes in hosts from @What-Zit-Tooya in this PR. No more IPs directly in hosts. Only domains.


Improve preview and screenshot display in the asset library

Images are now resized with Lanczos interpolation for higher quality. Video thumbnails now display a "pointing hand" cursor when hovered as they will open in an external browser.
Unlike #25761, this only applies to previews and screenshots, not icons. (We could still re-attempt this for icons now that we have Lanczos filtering, which might look better compared to trilinear filtering.)
Preview
Before

After

Thanks!


Add validation for threat intel source config

Description
Add source config validation so source config cannot be created without valid name, type, format, source, and ioc type. Additionally allows source configs with null values to still be read and deleted.
Related Issues
Resolves #1366
Check List

[ ] New functionality includes testing.
[ ] New functionality has been documented.
[ ] API changes companion pull request created.
[ ] Commits are signed per the DCO using --signoff.
[ ] Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

can we add test to verify #1366 is solved

Added parsing tests to make sure null fields can be parsed


Some updates for uberenv TPL setup on macos

Resolves #46
Resolves #47

Thanks @MarkoSterbentz for working through this!


Tonic Relection needs a readme/howto

Feature Request
Crates
tonic_reflection
Motivation
There are no usage examples and I had to reverse engineer the crate to use it.
Proposal
There should be a readme and/or crate documentation illustrating how to use tonic_reflection
Alternatives
Scour the web/roll my own, which made it hard to use the crate.

+1 on this.  Reflection is a little more spread-out than other examples.  It's the build.rs, tonic-reflection in Cargo.toml, and finally the example file.
There are many more undocumented examples though, not just this one.

+1 on this. Reflection is a little more spread-out than other examples. It's the build.rs, tonic-reflection in Cargo.toml, and finally the example file.
There are many more undocumented examples though, not just this one.

I'd be down to write up an example if the maintainers would be open to a PR.

+1 on this. Reflection is a little more spread-out than other examples. It's the build.rs, tonic-reflection in Cargo.toml, and finally the example file.
There are many more undocumented examples though, not just this one.

I'd be down to write up an example if the maintainers would be open to a PR.

@jaredmcqueen is that a yes?

I'll help out.  I've written a few services with tonic now and know my way around reflection.  I can't merge PRs though

👀

Also agree with this, the example is incomplete.
I cannot figure out how to implement tonic::include_file_descriptor_set!("test")
It just complains that there's no file, but the 'bin' isn't generated for the directory it's looking in, so I'm completely stuck.

are you calling this tonic::include_file_descriptor_set!("test") in the same crate that you have the descriptor set in?

i somehow missed this thread even though i scoured the entire repo for references to reflection 😂 and ended up making my own issue (#1600). ive got a PR out as well with an example (#1601), but am happy to include some actual documentation for how to get it working from scratch without relying on cloning the tonic repo for its build process


vtkWithQt4: remove package

for #33248 (Qt4 deprecation)

Not used anywhere in Nixpkgs.
I haven't added an alias but could do so.

Motivation for this change
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

Then also need to remove qtLib parameter.
cc @knedlsepp

The package builds fine with qt5.qtbase, so I could change the name and Qt version instead of removing?  Or remove the package but leave these flags to allow custom realizations?

If you need vtk with qt5 support, you could implement it (you will need to test that it is working). Otherwise, we could say "if it is not built on Hydra, it doesn't exist".

Closing for now, may upgrade to Qt5 if I have a chance sometime.


Update typos

Fixes a typo that nagged me every time I opened the documentation

Fine.  If that's the worst thing you can find, I'll take it.

Fine. If that's the worst thing you can find, I'll take it.

Woah woah woah - let’s not go too crazy here


fix(react-router): revert the Assign type to what was used prior in 1.26.11

The new Assign type introduced in 1.26.12 (#1436) was removing the optional part of keys in the route search schema.

Looks like simple fixes aren't the solution for this.
Will wait till Chris is available to administer the right fix for this.


Unable to debug .NET Framework application with single project packaging

Describe the bug
Trying to onboard the Windows App Sdk for the single project with packaging capability - which mostly works, except that debugging does not.
I cannot just use the "MsixPackage"-target and hit Run, I have to deploy, then use Debug -> Debug Installed App Package in order to get managed debugging to work.
Steps to reproduce the bug

Create WindowsAppSdk project, change to WPF
Set breakpoint in MainWindow constructor
Debug
Observe nothing stops

See attached repro.
Expected behavior
Step debugging works.
Screenshots
No response
NuGet package version
1.1.5
Packaging type
Packaged (MSIX)
Windows version
Windows 11 version 22H2 (22621)
IDE
Visual Studio 2022
Additional context
App3.zip

I'm unable to repro it now. I will close this issue and please let me know if this still has been an issue now with latest WinAppSDK


Unclear how to use caching with documented ARM template schema

Caching is not covered in any of the documented api versions of the front door ARM template schema.
Adding @tfitzmac since he is mentioned as author of the ARM template schema.
When exporting the template, it exports the caching configuration but that is using an undocumented API version (2018-08-01).

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 2973ab88-1258-8a80-1344-07a9e0063951
Version Independent ID: c0cff4d9-d7fd-7b25-52da-9cbe5dda3ba8
Content: Azure Front Door Service - caching
Content Source: articles/frontdoor/front-door-caching.md
Service: frontdoor
GitHub Login: @sharad4u
Microsoft Alias: sharadag

@hansmbakker There are a few parts of Front Door's Arm Templates which are not properly documented, see #42215 for example. I typically create a front door using the portal or PowerShell, and then use resources.azure.com to view the JSON of the resources in different API versions. This will typically give the needed info.
You can also go a step above and packet capture what the portal sends out when you create the front door with caching rules.
To add an example in the future, I am assigning this issue to the author to update as appropriate.

#reassign: amitsriva

#reassign: amitsriva

#reassign: jessie-jyy


Prevent Plot Re-Rendering when Forms Window is up

Right now the plot tries to re-render when the forms window is up. This is likely because the form-rerendering function is attached to the window as an event. It could have some kind of boolean that indicates that the form window is up to prevent that, or have it detach when the forms window comes up and re-attach when it goes back away.

It seems that this was fixed.


Out of Memory Error

During CPUHeavyGenrule //:BoomTile_route, openroad is being killed by the OOM killer. Which constraints can I change so this will consume less memory? I appreciate that this can be fixed by just getting more RAM, but my current machine has
64 GiB of RAM. Its a 20 core i7 running Ubuntu 22.04.4 LTS
megaboom$ bazel build BoomTile_final --verbose_failures
INFO: Analyzed target //:BoomTile_final (80 packages loaded, 1533 targets configured).
ERROR: /home/davidbyrd/Documents/src/github.com/The-OpenROAD-Project/megaboom/BUILD.bazel:832:15: Executing CPUHeavyGenrule //:BoomTile_route failed: (Exit 2): bash failed: error executing CPUHeavyGenrule command (from target //:BoomTile_route)
  (cd /home/davidbyrd/.cache/bazel/_bazel_davidbyrd/6129d0cbed3c216c13ad15fa0aba969d/execroot/_main && \
  exec env - \
  /bin/bash -c '\
OR_IMAGE=openroad/flow-ubuntu22.04-builder:latest  DESIGN_CONFIG=bazel-out/k8-fastbuild/bin/BoomTile_config.mk  STAGE_CONFIG=bazel-out/k8-fastbuild/bin/BoomTile_route_config.mk  MAKE_PATTERN=external/bazel-orfs~override/route-bazel.mk  RULEDIR=bazel-out/k8-fastbuild/bin  bazel-out/k8-fastbuild/bin/external/bazel-orfs~override/docker_shell  make --silent bazel-route elapsed')
# Configuration:<PHONE_NUMBER>caac83f2686a71e0317d4d9a62c6a78f5b323d7478d3ae7e7a8bf9
# Execution platform: @@local_config_platform//:host
make[1]: *** [/OpenROAD-flow-scripts/flow//Makefile:774: do-5_3_route] Error 137
make: *** [/home/davidbyrd/.cache/bazel/_bazel_davidbyrd/6129d0cbed3c216c13ad15fa0aba969d/external/bazel-orfs~override/route-bazel.mk:10: bazel-route] Error 2
[INFO-FLOW] ASU ASAP7 - version 2
Default PVT selection: BC
grep -q 1 /home/davidbyrd/.cache/bazel/_bazel_davidbyrd/6129d0cbed3c216c13ad15fa0aba969d/execroot/_main/bazel-out/k8-fastbuild/bin/results/asap7/BoomTile/base/grt.ok
Running fillcell.tcl, stage 5_2_fillcell
[WARNING STA-0450] virtual clock clock_vir can not be propagated.
[INFO DPL-0001] Placed 7135933 filler instances.
Elapsed time: 1:00.72[h:]min:sec. CPU time: user 52.00 sys 8.42 (99%). Peak memory: 13242068KB.
Running detail_route.tcl, stage 5_3_route
[WARNING STA-0450] virtual clock clock_vir can not be propagated.
[INFO ORD-0030] Using 20 thread(s).
detailed_route -output_drc /home/davidbyrd/.cache/bazel/_bazel_davidbyrd/6129d0cbed3c216c13ad15fa0aba969d/execroot/_main/bazel-out/k8-fastbuild/bin/reports/asap7/BoomTile/base/5_route_drc.rpt -output_maze /home/davidbyrd/.cache/bazel/_bazel_davidbyrd/6129d0cbed3c216c13ad15fa0aba969d/execroot/_main/bazel-out/k8-fastbuild/bin/results/asap7/BoomTile/base/maze.log -bottom_routing_layer M2 -top_routing_layer M9 -save_guide_updates -verbose 1 -drc_report_iter_step 5
[INFO DRT-0149] Reading tech and libs.
[WARNING DRT-0340] LEF58_ENCLOSURE EOL is not supported. Skipping for layer V3
[WARNING DRT-0340] LEF58_ENCLOSURE EOL is not supported. Skipping for layer V3
[WARNING DRT-0340] LEF58_ENCLOSURE EOL is not supported. Skipping for layer V4
[WARNING DRT-0340] LEF58_ENCLOSURE EOL is not supported. Skipping for layer V5
[WARNING DRT-0340] LEF58_ENCLOSURE EOL is not supported. Skipping for layer V6

Units:                1000
Number of layers:     21
Number of macros:     229
Number of vias:       93
Number of viarulegen: 11

[INFO DRT-0150] Reading design.

Design:                   BoomTile
Die area:                 ( 0 0 ) ( 1387092 1387092 )
Number of track patterns: 32
Number of DEF vias:       0
Number of components:     9212750
Number of terminals:      388
Number of snets:          2
Number of nets:           1942672

[INFO DRT-0167] List of default vias:
  Layer V2
    default via: VIA23
  Layer V3
    default via: VIA34
  Layer V4
    default via: VIA45
  Layer V5
    default via: VIA56
  Layer V6
    default via: VIA67
  Layer V7
    default via: VIA78
  Layer V8
    default via: VIA89
  Layer V9
    default via: VIA9Pad
[INFO DRT-0162] Library cell analysis.
[INFO DRT-0163] Instance analysis.
  Complete 100000 instances.
  Complete 200000 instances.
  Complete 300000 instances.
  Complete 400000 instances.
  Complete 500000 instances.
  Complete 600000 instances.
  Complete 700000 instances.
  Complete 800000 instances.
  Complete 900000 instances.
  Complete 1000000 instances.
  Complete 2000000 instances.
  Complete 3000000 instances.
  Complete 4000000 instances.
  Complete 5000000 instances.
  Complete 6000000 instances.
  Complete 7000000 instances.
  Complete 8000000 instances.
  Complete 9000000 instances.
[INFO DRT-0164] Number of unique instances = 483.
[INFO DRT-0168] Init region query.
[INFO DRT-0018]   Complete 100000 insts.
[INFO DRT-0018]   Complete 200000 insts.
[INFO DRT-0018]   Complete 300000 insts.
[INFO DRT-0018]   Complete 400000 insts.
[INFO DRT-0018]   Complete 500000 insts.
[INFO DRT-0018]   Complete 600000 insts.
[INFO DRT-0018]   Complete 700000 insts.
[INFO DRT-0018]   Complete 800000 insts.
[INFO DRT-0018]   Complete 900000 insts.
[INFO DRT-0019]   Complete 1000000 insts.
[INFO DRT-0019]   Complete 2000000 insts.
[INFO DRT-0019]   Complete 3000000 insts.
[INFO DRT-0019]   Complete 4000000 insts.
[INFO DRT-0019]   Complete 5000000 insts.
[INFO DRT-0019]   Complete 6000000 insts.
[INFO DRT-0019]   Complete 7000000 insts.
[INFO DRT-0019]   Complete 8000000 insts.
[INFO DRT-0019]   Complete 9000000 insts.
[INFO DRT-0024]   Complete Active.
[INFO DRT-0024]   Complete V0.
[INFO DRT-0024]   Complete M1.
[INFO DRT-0024]   Complete V1.
[INFO DRT-0024]   Complete M2.
[INFO DRT-0024]   Complete V2.
[INFO DRT-0024]   Complete M3.
[INFO DRT-0024]   Complete V3.
[INFO DRT-0024]   Complete M4.
[INFO DRT-0024]   Complete V4.
[INFO DRT-0024]   Complete M5.
[INFO DRT-0024]   Complete V5.
[INFO DRT-0024]   Complete M6.
[INFO DRT-0024]   Complete V6.
[INFO DRT-0024]   Complete M7.
[INFO DRT-0024]   Complete V7.
[INFO DRT-0024]   Complete M8.
[INFO DRT-0024]   Complete V8.
[INFO DRT-0024]   Complete M9.
[INFO DRT-0024]   Complete V9.
[INFO DRT-0024]   Complete Pad.
[INFO DRT-0033] Active shape region query size = 0.
[INFO DRT-0033] V0 shape region query size = 0.
[INFO DRT-0033] M1 shape region query size = 71179657.
[INFO DRT-0033] V1 shape region query size = 181268871.
[INFO DRT-0033] M2 shape region query size = 10138396.
[INFO DRT-0033] V2 shape region query size = 9205197.
[INFO DRT-0033] M3 shape region query size = 18410466.
[INFO DRT-0033] V3 shape region query size = 6136798.
[INFO DRT-0033] M4 shape region query size = 15356680.
[INFO DRT-0033] V4 shape region query size = 6137088.
[INFO DRT-0033] M5 shape region query size = 6536384.
[INFO DRT-0033] V5 shape region query size = 788800.
[INFO DRT-0033] M6 shape region query size = 398622.
[INFO DRT-0033] V6 shape region query size = 0.
[INFO DRT-0033] M7 shape region query size = 0.
[INFO DRT-0033] V7 shape region query size = 0.
[INFO DRT-0033] M8 shape region query size = 0.
[INFO DRT-0033] V8 shape region query size = 0.
[INFO DRT-0033] M9 shape region query size = 0.
[INFO DRT-0033] V9 shape region query size = 0.
[INFO DRT-0033] Pad shape region query size = 0.
[INFO DRT-0165] Start pin access.
[INFO DRT-0076]   Complete 1000 pins.
[INFO DRT-0076]   Complete 2000 pins.
[INFO DRT-0076]   Complete 3000 pins.
[INFO DRT-0076]   Complete 4000 pins.
[INFO DRT-0076]   Complete 5000 pins.
[INFO DRT-0076]   Complete 6000 pins.
[INFO DRT-0076]   Complete 7000 pins.
[INFO DRT-0076]   Complete 8000 pins.
[INFO DRT-0076]   Complete 9000 pins.
[INFO DRT-0077]   Complete 10000 pins.
[INFO DRT-0078]   Complete 11353 pins.
[INFO DRT-0079]   Complete 100 unique inst patterns.
[INFO DRT-0079]   Complete 200 unique inst patterns.
[INFO DRT-0079]   Complete 300 unique inst patterns.
[INFO DRT-0081]   Complete 375 unique inst patterns.
[INFO DRT-0082]   Complete 10000 groups.
[INFO DRT-0082]   Complete 20000 groups.
[INFO DRT-0082]   Complete 30000 groups.
[INFO DRT-0082]   Complete 40000 groups.
[INFO DRT-0082]   Complete 50000 groups.
[INFO DRT-0082]   Complete 60000 groups.
[INFO DRT-0082]   Complete 70000 groups.
[INFO DRT-0082]   Complete 80000 groups.
[INFO DRT-0082]   Complete 90000 groups.
[INFO DRT-0083]   Complete 100000 groups.
[INFO DRT-0083]   Complete 200000 groups.
[INFO DRT-0083]   Complete 300000 groups.
[INFO DRT-0083]   Complete 400000 groups.
[INFO DRT-0083]   Complete 500000 groups.
[INFO DRT-0083]   Complete 600000 groups.
[INFO DRT-0083]   Complete 700000 groups.
[INFO DRT-0083]   Complete 800000 groups.
[INFO DRT-0083]   Complete 900000 groups.
[INFO DRT-0083]   Complete 1000000 groups.
[INFO DRT-0083]   Complete 1100000 groups.
[INFO DRT-0083]   Complete 1200000 groups.
[INFO DRT-0083]   Complete 1300000 groups.
[INFO DRT-0083]   Complete 1400000 groups.
[INFO DRT-0083]   Complete 1500000 groups.
[INFO DRT-0083]   Complete 1600000 groups.
[INFO DRT-0083]   Complete 1700000 groups.
[INFO DRT-0083]   Complete 1800000 groups.
[INFO DRT-0083]   Complete 1900000 groups.
[INFO DRT-0084]   Complete 1911881 groups.
#scanned instances     = 9212750
#unique  instances     = 443
#stdCellGenAp          = 14348
#stdCellValidPlanarAp  = 120
#stdCellValidViaAp     = 11890
#stdCellPinNoAp        = 0
#stdCellPinCnt         = 6875679
#instTermValidViaApCnt = 0
#macroGenAp            = 33839
#macroValidPlanarAp    = 33839
#macroValidViaAp       = 0
#macroNoAp             = 0
[INFO DRT-0166] Complete pin access.
[INFO DRT-0267] cpu time = 00:01:55, elapsed time = 00:00:17, memory = 30707.38 (MB), peak = 35993.86 (MB)
[INFO DRT-0156] guideIn read 100000 guides.
[INFO DRT-0156] guideIn read 200000 guides.
[INFO DRT-0156] guideIn read 300000 guides.
[INFO DRT-0156] guideIn read 400000 guides.
[INFO DRT-0156] guideIn read 500000 guides.
[INFO DRT-0156] guideIn read 600000 guides.
[INFO DRT-0156] guideIn read 700000 guides.
[INFO DRT-0156] guideIn read 800000 guides.
[INFO DRT-0156] guideIn read 900000 guides.
[INFO DRT-0157] guideIn read 1000000 guides.
[INFO DRT-0157] guideIn read 2000000 guides.
[INFO DRT-0157] guideIn read 3000000 guides.
[INFO DRT-0157] guideIn read 4000000 guides.
[INFO DRT-0157] guideIn read 5000000 guides.
[INFO DRT-0157] guideIn read 6000000 guides.
[INFO DRT-0157] guideIn read 7000000 guides.
[INFO DRT-0157] guideIn read 8000000 guides.
[INFO DRT-0157] guideIn read 9000000 guides.
[INFO DRT-0157] guideIn read 10000000 guides.
[INFO DRT-0157] guideIn read 11000000 guides.
[INFO DRT-0157] guideIn read 12000000 guides.
[INFO DRT-0157] guideIn read 13000000 guides.
[INFO DRT-0157] guideIn read 14000000 guides.
[INFO DRT-0157] guideIn read 15000000 guides.
[INFO DRT-0157] guideIn read 16000000 guides.
[INFO DRT-0157] guideIn read 17000000 guides.
[INFO DRT-0157] guideIn read 18000000 guides.
[INFO DRT-0157] guideIn read 19000000 guides.
[INFO DRT-0157] guideIn read 20000000 guides.
[INFO DRT-0157] guideIn read 21000000 guides.
[INFO DRT-0157] guideIn read 22000000 guides.
[INFO DRT-0157] guideIn read 23000000 guides.

Number of guides:     23998525

[INFO DRT-0169] Post process guides.
[INFO DRT-0176] GCELLGRID X 0 DO 2568 STEP 540 ;
[INFO DRT-0177] GCELLGRID Y 0 DO 2568 STEP 540 ;
[INFO DRT-0026]   Complete 100000 origin guides.
[INFO DRT-0026]   Complete 200000 origin guides.
[INFO DRT-0026]   Complete 300000 origin guides.
[INFO DRT-0026]   Complete 400000 origin guides.
[INFO DRT-0026]   Complete 500000 origin guides.
[INFO DRT-0026]   Complete 600000 origin guides.
[INFO DRT-0026]   Complete 700000 origin guides.
[INFO DRT-0026]   Complete 800000 origin guides.
[INFO DRT-0026]   Complete 900000 origin guides.
[INFO DRT-0027]   Complete 1000000 origin guides.
[INFO DRT-0027]   Complete 2000000 origin guides.
[INFO DRT-0027]   Complete 3000000 origin guides.
[INFO DRT-0027]   Complete 4000000 origin guides.
[INFO DRT-0027]   Complete 5000000 origin guides.
[INFO DRT-0027]   Complete 6000000 origin guides.
[INFO DRT-0027]   Complete 7000000 origin guides.
[INFO DRT-0027]   Complete 8000000 origin guides.
[INFO DRT-0027]   Complete 9000000 origin guides.
[INFO DRT-0027]   Complete 10000000 origin guides.
[INFO DRT-0027]   Complete 11000000 origin guides.
[INFO DRT-0027]   Complete 12000000 origin guides.
[INFO DRT-0027]   Complete 13000000 origin guides.
[INFO DRT-0027]   Complete 14000000 origin guides.
[INFO DRT-0027]   Complete 15000000 origin guides.
[INFO DRT-0027]   Complete 16000000 origin guides.
[INFO DRT-0027]   Complete 17000000 origin guides.
[INFO DRT-0027]   Complete 18000000 origin guides.
[INFO DRT-0027]   Complete 19000000 origin guides.
[INFO DRT-0027]   Complete 20000000 origin guides.
[INFO DRT-0027]   Complete 21000000 origin guides.
[INFO DRT-0027]   Complete 22000000 origin guides.
[INFO DRT-0027]   Complete 23000000 origin guides.
[INFO DRT-0028]   Complete Active.
[INFO DRT-0028]   Complete V0.
[INFO DRT-0028]   Complete M1.
[INFO DRT-0028]   Complete V1.
[INFO DRT-0028]   Complete M2.
[INFO DRT-0028]   Complete V2.
[INFO DRT-0028]   Complete M3.
[INFO DRT-0028]   Complete V3.
[INFO DRT-0028]   Complete M4.
[INFO DRT-0028]   Complete V4.
[INFO DRT-0028]   Complete M5.
[INFO DRT-0028]   Complete V5.
[INFO DRT-0028]   Complete M6.
[INFO DRT-0028]   Complete V6.
[INFO DRT-0028]   Complete M7.
[INFO DRT-0028]   Complete V7.
[INFO DRT-0028]   Complete M8.
[INFO DRT-0028]   Complete V8.
[INFO DRT-0028]   Complete M9.
[INFO DRT-0028]   Complete V9.

[INFO DRT-0178] Init guide query.
[INFO DRT-0029]   Complete 100000 nets (guide).
[INFO DRT-0029]   Complete 200000 nets (guide).
[INFO DRT-0029]   Complete 300000 nets (guide).
[INFO DRT-0029]   Complete 400000 nets (guide).
[INFO DRT-0029]   Complete 500000 nets (guide).
[INFO DRT-0029]   Complete 600000 nets (guide).
[INFO DRT-0029]   Complete 700000 nets (guide).
[INFO DRT-0029]   Complete 800000 nets (guide).
[INFO DRT-0029]   Complete 900000 nets (guide).
[INFO DRT-0030]   Complete 1000000 nets (guide).
[INFO DRT-0035]   Complete Active (guide).
[INFO DRT-0035]   Complete V0 (guide).
[INFO DRT-0035]   Complete M1 (guide).
[INFO DRT-0035]   Complete V1 (guide).
[INFO DRT-0035]   Complete M2 (guide).
[INFO DRT-0035]   Complete V2 (guide).
[INFO DRT-0035]   Complete M3 (guide).
[INFO DRT-0035]   Complete V3 (guide).
[INFO DRT-0035]   Complete M4 (guide).
[INFO DRT-0035]   Complete V4 (guide).
[INFO DRT-0035]   Complete M5 (guide).
[INFO DRT-0035]   Complete V5 (guide).
[INFO DRT-0035]   Complete M6 (guide).
[INFO DRT-0035]   Complete V6 (guide).
[INFO DRT-0035]   Complete M7 (guide).
[INFO DRT-0035]   Complete V7 (guide).
[INFO DRT-0035]   Complete M8 (guide).
[INFO DRT-0035]   Complete V8 (guide).
[INFO DRT-0035]   Complete M9 (guide).
[INFO DRT-0035]   Complete V9 (guide).
[INFO DRT-0035]   Complete Pad (guide).
[INFO DRT-0036] Active guide region query size = 0.
[INFO DRT-0036] V0 guide region query size = 0.
[INFO DRT-0036] M1 guide region query size = 6358434.
[INFO DRT-0036] V1 guide region query size = 0.
[INFO DRT-0036] M2 guide region query size = 6406898.
[INFO DRT-0036] V2 guide region query size = 0.
[INFO DRT-0036] M3 guide region query size = 4298172.
[INFO DRT-0036] V3 guide region query size = 0.
[INFO DRT-0036] M4 guide region query size = 1094108.
[INFO DRT-0036] V4 guide region query size = 0.
[INFO DRT-0036] M5 guide region query size = 507259.
[INFO DRT-0036] V5 guide region query size = 0.
[INFO DRT-0036] M6 guide region query size = 208666.
[INFO DRT-0036] V6 guide region query size = 0.
[INFO DRT-0036] M7 guide region query size = 69899.
[INFO DRT-0036] V7 guide region query size = 0.
[INFO DRT-0036] M8 guide region query size = 0.
[INFO DRT-0036] V8 guide region query size = 0.
[INFO DRT-0036] M9 guide region query size = 0.
[INFO DRT-0036] V9 guide region query size = 0.
[INFO DRT-0036] Pad guide region query size = 0.
[INFO DRT-0179] Init gr pin query.
[INFO DRT-0185] Post process initialize RPin region query.
[INFO DRT-0181] Start track assignment.
[INFO DRT-0184] Done with 11233764 vertical wires in 52 frboxes and 7709672 horizontal wires in 52 frboxes.
[INFO DRT-0186] Done with 1231459 vertical wires in 52 frboxes and 1528070 horizontal wires in 52 frboxes.
[INFO DRT-0182] Complete track assignment.
[INFO DRT-0267] cpu time = 00:40:49, elapsed time = 00:07:03, memory = 51627.06 (MB), peak = 52074.85 (MB)
[INFO DRT-0187] Start routing data preparation.
Command terminated by signal 9
Elapsed time: 10:13.10[h:]min:sec. CPU time: user 2698.31 sys 37.74 (446%). Peak memory: 58177308KB.
Target //:BoomTile_final failed to build
ERROR: /home/davidbyrd/Documents/src/github.com/The-OpenROAD-Project/megaboom/BUILD.bazel:832:15 Executing genrule //:BoomTile_final failed: (Exit 2): bash failed: error executing CPUHeavyGenrule command (from target //:BoomTile_route)
  (cd /home/davidbyrd/.cache/bazel/_bazel_davidbyrd/6129d0cbed3c216c13ad15fa0aba969d/execroot/_main && \
  exec env - \
  /bin/bash -c '\
OR_IMAGE=openroad/flow-ubuntu22.04-builder:latest  DESIGN_CONFIG=bazel-out/k8-fastbuild/bin/BoomTile_config.mk  STAGE_CONFIG=bazel-out/k8-fastbuild/bin/BoomTile_route_config.mk  MAKE_PATTERN=external/bazel-orfs~override/route-bazel.mk  RULEDIR=bazel-out/k8-fastbuild/bin  bazel-out/k8-fastbuild/bin/external/bazel-orfs~override/docker_shell  make --silent bazel-route elapsed')
# Configuration:<PHONE_NUMBER>caac83f2686a71e0317d4d9a62c6a78f5b323d7478d3ae7e7a8bf9
# Execution platform: @@local_config_platform//:host
INFO: Elapsed time: 685.371s, Critical Path: 683.85s
INFO: 2 processes: 2 internal.
 As I recall, reducing number of cores wont help much.

This is a large design:
Number of components:     9212750
Number of nets:           1942672

though it is a bit odd that you have so many more components.  Does this design have a lot of dead space?  I doubt your problem relates to constraints.

I added 30GB of swap and the same thing happened, it just took longer. I'm also running it on an ec2 instance with 124GB of Ram and that's still running. What were the specs of the machine you confirmed it worked on?
Elapsed time: 3:45:22[h:]min:sec. CPU time: user 230047.12 sys 2971.37 (1723%). Peak memory: 63405600KB.

What you want to know, how much vm is required, should be possible to measure with https://github.com/The-OpenROAD-Project/OpenROAD/pull/5260

As I recall, I have 256GB vm, 96gb ram and 48 cores(96 threads).

@thebyrd FYI https://github.com/The-OpenROAD-Project/megaboom/pull/66. Running a build now to test...

Okay, it completed with a VM with 124GB; it looks like the peak memory was 84.65 GB. It took ~28 hours, so I think more RAM is the solution.


@conda-forge-admin,please add bot automerge

Additional comment:
No response

Hi! This is the friendly automated conda-forge-webservice.
I just wanted to let you know that I added bot automerge in conda-forge/dvc-gs-feedstock#4.


plot_2d breaks with matplotlib-3.5

Describe the bug
Calling mcmc.plot_2d(['omegabh2','omegach2','H0'], types={'lower':'kde','diagonal':'kde'}); returns
Traceback (most recent call last):
  File "/tmp/test.py", line 15, in <module>
    mcmc.plot_2d(['omegabh2','omegach2','H0'], types={'lower':'kde','diagonal':'kde'});
  File "/home/stefan/rps/codes/anesthetic/anesthetic/samples.py", line 330, in plot_2d
    fig, axes = make_2d_axes(axes, tex=self.tex,
  File "/home/stefan/rps/codes/anesthetic/anesthetic/plot.py", line 296, in make_2d_axes
    axes[x][y].lines = axes[x][y].twin.lines
AttributeError: can't set attribute

To Reproduce
Install new matplotlib pip install matplotlib==3.5, then use plot_2d. MWE from plot gallery:
import requests
import tarfile
import matplotlib.pyplot as plt
import numpy as np

for filename in ["plikHM_TTTEEE_lowl_lowE_lensing.tar.gz","plikHM_TTTEEE_lowl_lowE_lensing_NS.tar.gz"]:
    github_url = "https://github.com/williamjameshandley/cosmo_example/raw/master/"
    url = github_url + filename
    open(filename, 'wb').write(requests.get(url).content)
    tarfile.open(filename).extractall()

from anesthetic import MCMCSamples
mcmc_root = 'plikHM_TTTEEE_lowl_lowE_lensing/base_plikHM_TTTEEE_lowl_lowE_lensing'
mcmc = MCMCSamples(root=mcmc_root)
mcmc.plot_2d(['omegabh2','omegach2','H0'], types={'lower':'kde','diagonal':'kde'});
plt.show()

Expected behavior
Plot works, like with matplotlib-3.4.1. Edit: matplotlib==3.4.3 also works
Additional context
python version 3.9.7, anesthetic git version (commit 6f80b6a8b598708ee2409fe461ecfeddce941fb7 from 15th November)

I confirm this is also broken on my machines. The recommended fix for now is to use matplotlib 3.4.3 or lower.
I'll try and have a look into fixing this once term is over, but it looks like this breaking API change is the cause of the issues.

Thanks @Stefan-Heimersheim for flagging this up. Should be fixed with #184.


Release new version of ml-wrappers

Releasing new version with the predict_proba fix to unblock https://github.com/microsoft/responsible-ai-toolbox/pull/2203

Codecov Report

Merging #142 (7ede2d7) into main (bb11f0f) will decrease coverage by 6.49%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #142      +/-   ##
==========================================
- Coverage   41.47%   34.99%   -6.49%
==========================================
  Files          27       27
  Lines        1789     1789
==========================================
- Hits          742      626     -116
- Misses       1047     1163     +116

Flag
Coverage Δ

unittests
34.99% <100.00%> (-6.49%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed
Coverage Δ

python/ml_wrappers/version.py
100.00% <100.00%> (ø)

... and 5 files with indirect coverage changes


Build fails on Ubuntu older than 18.04 due to old cmake version

Example:
cmake .
CMake Error at CMakeLists.txt:1 (cmake_minimum_required):
  CMake 3.7 or higher is required.  You are running version 2.8.7

Proposed fix by @chipitsine should be applied.

The current error is:
wget https://cmake.org/files/v/cmake-3.12.3.tar.gz && tar -xzf cmake-3.12.3.tar.gz
--2018-10-09 14:13:38--  https://cmake.org/files/v/cmake-3.12.3.tar.gz
Resolving cmake.org (cmake.org)... failed: Name or service not known.
wget: unable to resolve host address ‘cmake.org’

This is getting tricky.

I played a bit with launchpad, seems it runs builds without network connectivity

It's indeed so, thanks for help! I've worked this around already and fixing newly found problems.

I've had to embed cmake source into debian folder and build it from source in case built in version is older than required, and also I've had to downgrade cmake to be able to build it with GCC without g++11 support under 12.04. Now it builds like a charm, and I see no problems left, feel free to reopen or create new issues in case of trouble.


Firebase signInWithCredential error occur

I am using Firebase in iOS using swift language. I am trying to signIn with Facebook. I am getting FBSDKAccessToken using with graphRequest. When I am using Firebase  signIn method of Facebook then getting following error

> UserInfo={NSUnderlyingError=0x608000059560 {Error Domain=FIRAuthInternalErrorDomain Code=3 "(null)"

UserInfo={FIRAuthErrorUserInfoDeserializedResponseKey={
code = 400;
errors =     (
{
domain = global;
message = "Unsuccessful debug_token response from Facebook: {"error":{"message":"(#4) Application request limit
reached","type":"OAuthException","is_transient":true,"code":4,"fbtrace_id":"En2h2Q0L7TP"}}";
reason = invalid;
}
);
message = "Unsuccessful debug_token response from Facebook: {"error":{"message":"(#4) Application request limit
reached","type":"OAuthException","is_transient":true,"code":4,"fbtrace_id":"En2h2Q0L7TP"}}";
}}}, error_name=ERROR_INTERNAL_ERROR, NSLocalizedDescription=An internal error has occurred, print and inspect the error details for
more information.}

And my complete code is

    {
        let login : FBSDKLoginManager = FBSDKLoginManager()
        login.logOut()
        login.logInWithReadPermissions(["public_profile","email"], fromViewController: self, handler:  { (result, error) -> Void in
            if (error == nil){
                let request: FBSDKGraphRequest = FBSDKGraphRequest(graphPath: "me", parameters: ["fields": "id,email,first_name,middle_name,last_name,picture"], HTTPMethod: "GET")
                request.startWithCompletionHandler({ (connection : FBSDKGraphRequestConnection!, result : AnyObject!, error : NSError!) -> Void in
                    if error == nil
                    {
                        let userDetail : NSDictionary = result as! NSDictionary

                    let token = FBSDKAccessToken.currentAccessToken().tokenString

                       let credential =  FIRFacebookAuthProvider.credentialWithAccessToken(token)
                       FIRAuth.auth()?.signInWithCredential(credential, completion: { (user, error) in

                            if error == nil
                            {

                                let homeVC = self.storyboard?.instantiateViewControllerWithIdentifier("PageController") as? PageController
                                self.navigationController?.pushViewController(homeVC!, animated: true)

                                print(userDetail)
                            }
                            else
                            {
                                 print("Signing  Failed ")

                            }

                        })

                    }
                    else
                    {
                        print("Error Getting Info \(error)");
                    }
                })
            }else{
                print("Login Failed")
            }
        })

        return;
    }

Looks like you're running into the Facebook API rate limit.

message = "Unsuccessful debug_token response from Facebook: {"error":{"message":"(#4) >Application request limit reached"

Try posting this question over at StackOverflow.

I have posted already http://stackoverflow.com/questions/43519458/firebase-signinwithcredential-giving-error-using-facebook-sdk
But what is the solution. Sorry I didn't get

I have same probleme

And I have same probleme

but in my other application, Facebook works fine

I solve this problem by change the FacebookAppID in info.plist and URL. :)
I used the one which is unconnected with Firebase application. (old one)


Using openui with AzureOpenai

I have Azureopenai Credentials and i was wondering if the openui can be used with Azureopenai

I'm using the native OpenAI client which I imagine supports it.  You can just change the code here.

i tried changing the code as you mentioned ,but still facing a lot of errors

Thanks @vanpelt works perfectly for me with gpt-4o.
@NikitaRawat2003 the fastest way to try it, is to create on your azure instance a deployment that as the same name as the model name in openai ("gpt-3.5-turbo", "gpt-4o", "gpt-4-turbo") and then import from AsyncAzureOpenAI
and replace:
# openai = AsyncOpenAI(
#     base_url=config.OPENAI_BASE_URL,
#     api_key=config.OPENAI_API_KEY)

openai = AsyncAzureOpenAI(
    api_key=config.OPENAI_API_KEY,
    api_version="2024-02-01",
    azure_endpoint=config.OPENAI_BASE_URL)

Thanks @vanpelt works perfectly for me with gpt-4o. @NikitaRawat2003 the fastest way to try it, is to create on your azure instance a deployment that as the same name as the model name in openai ("gpt-3.5-turbo", "gpt-4o", "gpt-4-turbo") and then import AsyncAzureOpenAI from openai and replace:
# openai = AsyncOpenAI(
#     base_url=config.OPENAI_BASE_URL,
#     api_key=config.OPENAI_API_KEY)

openai = AsyncAzureOpenAI(
    api_key=config.OPENAI_API_KEY,
    api_version="2024-02-01",
    azure_endpoint=config.OPENAI_BASE_URL)

did you add your endpoint and key in the config.py file? can you show me how you did it?

I created a pull request for use Azure OpenAI.
https://github.com/wandb/openui/pull/216


Feature/replace jdbi with hibernate

Current coverage is 75.93% (diff: 34.04%)

Merging #19 into dev will decrease coverage by 2.27%

@@                dev        #19   diff @@
==========================================
  Files            30         30
  Lines           569        590    +21
  Methods           0          0
  Messages          0          0
  Branches         57         61     +4
==========================================
+ Hits            445        448     +3
- Misses          107        123    +16
- Partials         17         19     +2

Powered by Codecov. Last update 92cf25e...a39cf25


Postgres ->count() always returns 1 when grouping

Laravel Version: 5.3.29
PHP Version: 7.1
Database Driver & Version: PostgreSQL 9.6.1 on x86_64-apple-darwin16.1.0, compiled by Apple LLVM version 8.0.0 (clang-8<IP_ADDRESS>), 64-bit

Description:
Laravel will always return 1 when you try to call the ->count() method on a query that is being grouped. See below.
Steps To Reproduce:
If I want to get the amount of users in a system who have posted an article I might want to do something like this:
$userPostCount = DB::table('posts')
                                ->whereNotNull('user_id')
                                ->groupBy('user_id')
                                ->groupBy('id')
                                ->count();

But this will always return 1. At least whilst testing, I had 7 posts, each with a unique user_id but the query above will return 1.
As a result, I've had to get around this by doing:
$userPostCount = DB::table('posts')
                                ->whereNotNull('user_id')
                                ->groupBy('user_id')
                                ->groupBy('id')
                                ->select('user_id')
                                ->get()->unique('user_id')->count();

I don't mind doing this, but obviously at scale I'm pulling all records here and having to then count those which can't be too good. I would switch to MySQL but Postgres is kinda shitty about that.
I also posted this on #14123

Could you use the querylog/debugbar/Clockwork to check what actual sql statement is generated and executed?

@sisve I will do. I will also build a super simple example project and provide a ZIP of that for you.

i don't think this is laravel issue.
i think the problem is on your query.
why you need to group by the id? i believe the id is uniq each row.
even if you are using mySQL, i think the above query will return the same. CMIIW.
with the above code, your query will look like:
select count(*) as aggregate from `posts` where `user_id` is not null group by `user_id`, `id`

try to run the query and see the result.

@alfa6661  I may be wrong, I often am, but it appeared I couldn't actually run the query without grouping by ID. In any case, grouping by the ID shouldn't really do anything because they are all unique anyway. Postgres is very funny about this and for some reason requests it be done this way.

@davzie Basically what you need to take notice about is this
$userPostCount = DB::table('posts')                                 ->whereNotNull('user_id')                                 ->groupBy('user_id')                                 ->groupBy('id')                                 ->count();
This line of code is doing next : Group Columns by user_id,id returns only one row -> which will always be 1 ( ** because each user when posts will have post with distinct id ** ), and when you call count() after that, Laravel will return you only one row.  Which is always one because you are doing only counting on one row. That is why when you use get() and after that count() you are basically doing counting of SELECT for all rows.
Idea:
amount of users in a system who have posted an article
If you are looking only for only number of users who posted
$userPostCount = DB::table('posts')             -select('user_id')             ->whereNotNull('user_id')             ->distinct()             ->get()             ->count('user_id');
No need for group by, you already know that distinct ids of users are in table.

@movingSone This is great. Does your final example not have the issue though that I'm still retrieving tonnes of rows from the database? What if there are 1,000,000 users, I'll be pulling 1,000,000 records from the DB and counting the result rather than harnessing the DB's built in counting functions.

@davzie That will all depend on your implementation inside the system. If you will have million users then make testing of different methods. Good way to get speed would be to store that information in database where it is always populated when users create posts. That way you get results much faster.
Or try using
DB::select('custom SQL');


kstep detection for xftf

The group.kwin recorded in the mismatch with the group.k when the when the kstep in the autobk and xftf is different.
This will not affect the calculation of the R, since the R is calculated using the interpolated chi(k) and kwin, but it will affect on the plotting.
Sample code to reproduce the problem.
import matplotlib.pyplot as plt
from larch.xafs import pre_edge, autobk, xftf, ftwindow

group = Group()
group.energy = energy
group.mu = mu

pre_edge(group)

# The problem only occurs when we use non-default values for autobk or xftf
autobk(group, kstep=0.1)

# I am using hanning, kmin=2, dk=2 for clarity
xftf(group, window="hanning", kmin=1, dk=1)

correct_kwin = ftwindow(group.k, xmin=1, dx=1)

plt.plot(group.k, group.chi*group.k**2, label='chi(k)')
plt.plot(group.k, group.kwin, label='chi(k)*kwin')
plt.plot(group.k, correct_kwin, label='correct kwin')
plt.legend()
plt.xlim(0, 4)
plt.xlabel('k (A$^{-1}$)')
plt.ylabel('chi(k)')

plt.savefig('xftf.png', dpi=300)

Before correction:

After correction:

@Ameyanagi Thanks -- that's a good catch!


AAD Domain Services: add  06-01-2022 API version

MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Changelog
Add a changelog entry for this PR by answering the following questions:

What's the purpose of the update?

[ ] new service onboarding
[x] new API version
[ ] update existing version for new feature
[ ] update existing version to fix swagger quality issue in s360
[ ] Other, please clarify

When are you targeting to deploy the new service/feature to public regions? Please provide the date or, if the date is not yet available, the month.  6/13/2022
When do you expect to publish the swagger? Please provide date or, the the date is not yet available, the month. 6/13/2022
If updating an existing version, please select the specific language SDKs and CLIs that must be refreshed after the swagger is published.

[ ] SDK of .NET (need service team to ensure code readiness)
[ ] SDK of Python
[ ] SDK of Java
[ ] SDK of Js
[ ] SDK of Go
[ ] PowerShell
[ ] CLI
[ ] Terraform
[x] No refresh required for updates in this PR

Contribution checklist:

[x] I commit to follow the Breaking Change Policy of "no breaking changes"
[x] I have reviewed the documentation for the workflow.
[x] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

Applicability: :warning:
If your changes encompass only the following scenarios, you should SKIP this section, as these scenarios do not require ARM review.

Change to data plane APIs
Adding new properties
All removals

Otherwise your PR may be subject to ARM review requirements. Complete the following:

[x] Check this box if any of the following appy to the PR so that the label "ARMReview" and "WaitForARMFeedback" will be added by bot to kick off ARM API Review.  Missing to check this box in the following scenario may result in delays to the ARM manifest review and deployment.

Adding a new service
Adding new API(s)
Adding a new API version
-[ ] To review changes efficiently, ensure you are using OpenAPIHub to initialize the PR for adding a new version. More details, refer to the wiki.

[x] Ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[x] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If any of the following scenarios apply to the PR, request approval from the Breaking Change Review Board as defined in the Breaking Change Policy.

[ ] Removing API(s) in a stable version
[ ] Removing properties in a stable version
[ ] Removing API version(s) in a stable version
[ ] Updating API in a stable or public preview version with Breaking Change Validation errors
[ ] Updating API(s) in public preview over 1 year (refer to Retirement of Previews)

Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Addition details on the process and office hours are on the Breaking change Wiki.
Please follow the link to find more details on PR review process.

The Swagger ModelValidation failures are from examples that were not changed outside of updating the API version. I'm not opposed to changing them, however I am not sure what is the right fix here. Should this be changed to date-time instead of date-time-rfc1123? How do I determine?

The Swagger ModelValidation failures are from examples that were not changed outside of updating the API version. I'm not opposed to changing them, however I am not sure what is the right fix here. Should this be changed to date-time instead of date-time-rfc1123? How do I determine?

Hi @jodyhendrix , for the time format, it depends on the implementation of service codes. If the time format is as written in examples, then "format": "date-time" is preferred. For lro, please add the location in response header to fix the error.

Since there is cross version breaking change, we will require breaking change review board to review this pr.


Update blob-inventory-how-to.md

We have seen few instance wherein customer got confused and thought the container being selected as the source for report. So just adding some clarification to the documentation as to the container selected represent as the destination container or target container for report generation.

Thank you @Amrinder-Singh29!
Reviewers: can you please modify that line to say - "Choose the container that will store inventory reports."
Thank you!
#sign-off


Create Html for Transaction table/view

This would be the page to make a transaction, Basic stuff like sending to, amount etc... @Mohnish2004

Done


Payments per day card with a breakdown between dues packages

A separate card for each package or preferably one card with multiple lines on the line graph for the different dues packages

IMO we should split this out to a dues dashboard

Analytics moved to Metabase


Package the example code

What

Create Dockerfile to package the example
refactor console output to use pino library
additional npm run targets to work with the simulator

Why

Package the example code for ease of distribution and use
make logging consistent with the simulator
provide a simple way to run the simulator for local use

@philf999 this is the branch I referred to this morning as a draft pr