EF Core + AutoMapper ProjectTo - SQL code not projected correctly

Is there an existing issue for this?

[X] I have searched the existing issues

Describe the bug
I'm using EF Core (v 5.0.9) and Automapper (v 10.1.1) with a very simple example database (the one described here).
I've set up a GraphQL server with HotChocolate and defined some simple GraphQL queries with [UseDbContext] and [UseProjection]. The generated SQL code is projected depending on the selection of fields, exactly as expected.
However when I add a layer of simple DTO classes, define maps with Automapper and then use ProjectTo in the queries, the projection stops working properly when there's a 1-to-1 relationship (interestingly it doesn't seem to happen for 1-to-many).
So for example the following query:
{
  postsDto {
    postId,
    title,
    blog {
      url
    }
  }
}

Results in the following SQL code:
SELECT [p].[PostId], [p].[Title], CAST(0 AS bit), [b].[BlogId], [b].[Url], [p0].[BlogId], [p0].[Content], [p0].[PostId], [p0].[Title]
FROM [Posts] AS [p]
INNER JOIN [Blogs] AS [b] ON [p].[BlogId] = [b].[BlogId]
LEFT JOIN [Posts] AS [p0] ON [b].[BlogId] = [p0].[BlogId]
ORDER BY [p].[PostId], [b].[BlogId], [p0].[PostId]

Which has fields that I didn't ask for plus it makes and extra JOIN back to the Posts table (because the Blog Dto exposes a Posts property, even though I haven't asked for it in the GraphQL query).
I also experience the problem described here https://github.com/ChilliCream/hotchocolate/issues/3008.
Steps to reproduce

Create a simple EF Core DbContext and Entities exactly as described here .
Create a ASP.NET 5.0 project and configure HotChocolate as described here
Install Automapper 10.1.1 and create Dto classes for Blog and Posts, and then configure simple mappings:

    public class PostDto
    {
        public int PostId { get; set; }
        public string Title { get; set; }
        public string Content { get; set; }
        public int BlogId { get; set; }
        public BlogDto Blog { get; set; }
    }

	public class PostProfile : Profile
	{
		public PostProfile()
		{
			CreateMap<Post, PostDto>();
		}
	}

        public void ConfigureServices(IServiceCollection services)
        {
            services
                .AddGraphQLServer()
                .AddProjections()
                .AddQueryType<Query>();

            services.AddMiniProfiler
               (options =>
               { options.RouteBasePath = "/profiler"; })
                     .AddEntityFramework();

            services.AddPooledDbContextFactory<BloggingContext>(options =>
            {
               ...
            });

            services.AddAutoMapper(typeof(BlogProfile));
        }

Define a simple projected query using UseDbContext and UseProjection:

        [UseDbContext(typeof(BloggingContext))]
        [UseProjection]
        public IQueryable<BlogDto> GetBlogsDto(
               [ScopedService] BloggingContext context)
               => context.Blogs.ProjectTo<BlogDto>(mapper.ConfigurationProvider);

        [UseDbContext(typeof(BloggingContext))]
        [UseProjection]
        public IQueryable<PostDto> GetPostsDto(
               [ScopedService] BloggingContext context)
               => context.Posts.ProjectTo<PostDto>(mapper.ConfigurationProvider);

Run the following query and check the generated SQL code using the miniprofiler

{
  postsDto {
    postId,
    title,
    blog {
      url
    }
  }
}

The generated SQL code is incorrect for me:

SELECT [p].[PostId], [p].[Title], CAST(0 AS bit), [b].[BlogId], [b].[Url], [p0].[BlogId], [p0].[Content], [p0].[PostId], [p0].[Title]
FROM [Posts] AS [p]
INNER JOIN [Blogs] AS [b] ON [p].[BlogId] = [b].[BlogId]
LEFT JOIN [Posts] AS [p0] ON [b].[BlogId] = [p0].[BlogId]
ORDER BY [p].[PostId], [b].[BlogId], [p0].[PostId]

Note this issue doesn't happen if the EF Core entities are use directly (no DTOs or automapper), and also if the relationship is one-to-many such as the inverse query:

{
  blogsDto {
    blogId,
    posts {
      postId,
      title
    }
  }
}

Relevant log output
No response
Additional Context?
I have tested this with the latest stable version (11.3.5) and the latest perelease version (12.0.0-preview30), getting the same result.
Product
Hot Chocolate
Version
11.3.5

Hey @tobias-tengler, I also have this issue, and it is reproducing even if I make a manual projection without AutoMapper to a DTO like this:
public IQueryable<ContactDto> GetContacts()
{
    return DbContext.Set<Contact>().Select(c => new ContactDto
    {
        Id = c.Id,
        Name = c.Name,
        Age = c.Age,
        Children = c.Children.Select(cc => new ChildrenDto
        {
            Id = c.Id,
            Name = c.Name,
            Age = c.Age
        })
    });
}

Hey @eduardomhg , have you found any workaround on this one?

This issue also happens without AutoMapper, using just regular naive manual mapping. It looks like the null check described in #2373 is the issue. When querying for a single attribute on a collection (where the projection works as expected) you will see that it projects the ID of the other entity implicitly, which I'm guessing means that the null check is done only on the ID, while for single entities it does the null check on the entire entity.

Same here, using .Net 6, HotChocolate 12.3.2 and AutoMapper 8.1.1
Flattening the target models (using automapper) fixes the problem but this is not a viable solution.
This is a game stopper everywhere we want to use nested objects without data loaders.

Can someone provide a reproduction of bug in form of a very simple project? so we can have a look at it?

So I have investigated what is going on here.
Given the configuration in the issue description and the following query:
    {
      postsdto {
        postid
        title
        blog {
          url
        }
      }
    }

This is what projections generate:
.Lambda #Lambda1<System.Func`2[HotChocolate.Data.Projections.AutomapperTests+PostDto,HotChocolate.Data.Projections.AutomapperTests+PostDto]>(HotChocolate.Data.Projections.AutomapperTests+PostDto $_s1)
{
    .New HotChocolate.Data.Projections.AutomapperTests+PostDto(){
        PostId = $_s1.PostId,
        Title = $_s1.Title,
        Blog = .If ($_s1.Blog != null) {
            .New HotChocolate.Data.Projections.AutomapperTests+BlogDto(){
                Url = ($_s1.Blog).Url
            }
        } .Else {
            .Default(HotChocolate.Data.Projections.AutomapperTests+BlogDto)
        }
    }
}

As soon as this is applied to the IQueryable that was mapped with automapper:
DbSet<Post>()
    .Select(dtoPost => new PostDto{
        Blog = dtoPost.Blog == null ? null : new BlogDto{
            BlogId = dtoPost.Blog.BlogId,
            Name = dtoPost.Blog.Name,
            Posts = dtoPost.Blog.Posts
                .Select(dtoPost => new PostDto{
                    BlogId = dtoPost.BlogId,
                    Content = dtoPost.Content,
                    PostId = dtoPost.PostId,
                    Title = dtoPost.Title
                }
                )
                .ToList(),
            Url = dtoPost.Blog.Url
        }
        ,
        BlogId = dtoPost.BlogId,
        Content = dtoPost.Content,
        PostId = dtoPost.PostId,
        Title = dtoPost.Title
    }
    )
    .Select(_s1 => new PostDto{
        PostId = _s1.PostId,
        Title = _s1.Title,
        Blog = _s1.Blog != null ? new BlogDto{ Url = _s1.Blog.Url }
         : default(BlogDto)
    }
    )

So you see that the root of the problem is, that the projections of auto mapper are not merged with the projections of the UseProjections.
There are a couple of possibilities we have here.

We could rewrite the expression tree based on what we have projected, but if you start writing an expression tree rewriter you open pandoras box.
We could also let the projection provider read the automapper configuration and project based on that, but this would require a significant change in the projection visitation logic and possible breaking changes
EF Core could understand chained selects better but this would require investigation why they do not do it already, a PR ant then maybe a possible fix in EF Core 7

So i tried out a few things (pretty much in the order that is listed above).
But then, i had a deeper look into the the automapper documentation. And voila:
https://docs.automapper.org/en/stable/Queryable-Extensions.html#explicit-expansion
So if we configure the Profile like this:
    public class PostProfile : Profile
    {
        public PostProfile()
        {
            CreateMap<Post, PostDto>().ForAllMembers(x => x.ExplicitExpansion());
        }
    }

and do some magic in the resolver:
        [UseDbContext(typeof(BloggingContext))]
        [UseSqlLogging]
        [UseProjection]
        public IQueryable<PostDto> GetPostsWorking(
            [Service] IMapper mapper,
            [ScopedService] BloggingContext dbContext,
            IResolverContext context)
        {
            // ensure sorting is only applied once
            context.LocalContextData =
                context.LocalContextData.SetItem(QueryableProjectionProvider.SkipProjectionKey,
                    true);

            var visitorContext =
                new QueryableProjectionContext(
                    context,
                    context.ObjectType,
                    context.Selection.Field.Type.UnwrapRuntimeType());
            var visitor = new QueryableProjectionVisitor();
            visitor.Visit(visitorContext);

            Expression<Func<PostDto, object>> projection =
                visitorContext.Project<PostDto, object>();

            var res = dbContext.Posts
                .ProjectTo<PostDto>(mapper.ConfigurationProvider, projection);
            return res;
        }

We get this result:
DbSet<Post>()
    .Select(dtoPost => new PostDto{
        Blog = dtoPost.Blog == null ? null : new BlogDto{ Url = dtoPost.Blog.Url }
        ,
        PostId = dtoPost.PostId,
        Title = dtoPost.Title
    }
    )

That generates the following SQL
SELECT 0, "b"."Url", "p"."PostId", "p"."Title"
FROM "Posts" AS "p"
INNER JOIN "Blogs" AS "b" ON "p"."BlogId" = "b"."BlogId"

This requires a very minor API change in projections. I guess we could also create a HotChocolate.Data.Projections.AutoMapper package that abstracts this a little.
Could everyone involved life with this API:
        [UseDbContext(typeof(BloggingContext))]
        [UseProjection]
        public IQueryable<PostDto> GetPostsWorking(
            [ScopedService] BloggingContext dbContext,
            IResolverContext context)
        {
            return dbContext.Posts.ProjectTo<Post, PostDto>(context);
        }

Hi @PascalSenn, yes, I can confirm that downgrading Automapper to version 10.1.1 (and in my case AutoMapper.Extensions.Microsoft.DependencyInjection to 8.1.1) does fix the issue and works as I expect (and you intended).
So do you think this is a bug that should be raised in Automapper and not a behavior change that has caused your ProjectTo to not work properly anymore? Just asking to see what the next step should be.

@eduardomhg #4724 (comment)
It seems like the problem is automapper 11. Projections to work with 10.1. This is something that should be fixed updstream in the automapper repository. But watch out. the resulting expressions are often to complex for EF to understand. So if you want to use this in combination with filtering, you could run into issues

From what I see in the AutoMapper documentation examples regarding explicit expansion (https://docs.automapper.org/en/stable/Queryable-Extensions.html#explicit-expansion), I think, that the projection expression you've fed to AutoMapper's ProjectTo method never supposed to work.
AutoMapper's ProjectTo method wants an expression for each expanded member separately and not one expression, containing the entire initialization.
Here is the sample GraphQL query from my test project:
{
  orders {
      id
      name
      description
      customer {
        id,
        contacts{
          position,
          name,
          value
        }
      },
      items{
        position
      }
  }
}

Your current ProjectTo method from HotChocolate.Data.AutoMapper package generates this single projection expression in my setup:
{
	_s1 => Convert(
	new OrderDto()
	{
		Id = _s1.Id,
		Name = _s1.Name,
		Description = _s1.Description,
		Customer = new CustomerDto()
		{
			Id = _s1.Customer.Id,
			Contacts = _s1.Customer.Contacts.Select(p2 => new CustomerContactDto()
			{
				Position = p2.Position,
				Name = p2.Name,
				Value = p2.Value
			}).ToArray()
		},
		Items = _s1.Items.Select(p1 => new OrderItemDto()
		{
			Position = p1.Position
		}).ToArray()
	}, Object)
}

I've managed to rewrite your code and break this projection expression into multiple separate expand expressions:
p_0 => Convert(p_0.Id, Object)
p_0 => p_0.Name
p_0 => p_0.Description
p_0 => Convert(p_0.Customer.Id, Object)
p_0 => p_0.Customer.Contacts.Select(p_1 => Convert(p_1.Position, Object))
p_0 => p_0.Customer.Contacts.Select(p_1 => p_1.Name)
p_0 => p_0.Customer.Contacts.Select(p_1 => p_1.Value)
p_0 => p_0.Items.Select(p_1 => Convert(p_1.Position, Object))

This way I've got a correct projection SQL code even with AutoMapper 12.0.1:
SELECT t."Id", t."Name", t."Description", FALSE, c."Id", c0."Position", c0."Name", c0."Value", c0."CustomerId", o0."Position", o0."OrderId"
      FROM (
          SELECT o."Id", o."CustomerId", o."Description", o."Name"
          FROM "Orders" AS o
          LIMIT @__p_0
      ) AS t
      INNER JOIN "Customers" AS c ON t."CustomerId" = c."Id"
      LEFT JOIN "CustomerContacts" AS c0 ON c."Id" = c0."CustomerId"
      LEFT JOIN "OrderItem" AS o0 ON t."Id" = o0."OrderId"
      ORDER BY t."Id", c."Id", c0."CustomerId", c0."Position", o0."OrderId"

The projection separation is somewhat naive, and will possibly break on some edge cases, but here it is, feel free to test and enhance it:
public static class AutoMapperQueryableExtensions
{
    public static IQueryable<TDest> ProjectTo<TSource, TDest>(this IQueryable<TSource> query, IResolverContext context)
    {
        IMapper mapper = context.Service<IMapper>();

        context.LocalContextData =
            context.LocalContextData.SetItem(QueryableProjectionProvider.SkipProjectionKey,
                true);

        var visitorContext =
            new QueryableProjectionContext(
                context,
                context.ObjectType,
                context.Selection.Field.Type.UnwrapRuntimeType(), false);
        var visitor = new QueryableProjectionVisitor();
        visitor.Visit(visitorContext);

#pragma warning disable CS8631
        Expression<Func<TDest, object?>> projection = visitorContext.Project<TDest, object?>();
#pragma warning restore CS8631

        List<Expression<Func<TDest, object?>>> membersToExpand = new();

        VisitRoot(projection, membersToExpand);

        return query.ProjectTo(mapper.ConfigurationProvider, membersToExpand.ToArray());
    }

    private static void VisitRoot<TDest>(Expression<Func<TDest, object?>> root, List<Expression<Func<TDest, object?>>> expressions)
    {
        UnaryExpression lambdaBody = (UnaryExpression)root.Body;

        MemberInitExpression memberInit = (MemberInitExpression)lambdaBody.Operand;

        int level = 0;

        ParameterExpression parameter = MakeParameter(memberInit.Type, level);

        foreach (var expression in VisitMemberInit(memberInit, parameter, level))
        {
            var lambda = Expression.Lambda<Func<TDest, object?>>(expression, parameter);

            expressions.Add(lambda);
        }
    }

    private static IEnumerable<Expression> VisitMemberInit(MemberInitExpression memberInit,
        Expression path, int level)
    {
        var memberAssignments = memberInit.Bindings.Cast<MemberAssignment>();

        foreach (MemberAssignment memberAssignment in memberAssignments)
        {
            foreach (Expression expression in VisitMemberAssignment(memberAssignment, path, level))
            {
                yield return expression;
            }
        }
    }

    private static IEnumerable<Expression> VisitMemberAssignment(MemberAssignment memberAssignment, Expression path, int level)
    {
        var memberExpression = memberAssignment.Expression;

        var member = (PropertyInfo)memberAssignment.Member;

        if (memberExpression is MemberExpression)
        {
            yield return VisitPrimitive(member, path);
        }

        if (memberExpression is MemberInitExpression nestedMemberInit)
        {
            foreach (Expression expression in VisitNestedObject(nestedMemberInit, member, path, level))
            {
                yield return expression;
            }
        }

        if (memberExpression is MethodCallExpression toArrayCallExpr)
        {
            foreach (Expression expression in VisitNestedCollection(toArrayCallExpr, member, path, level))
            {
                yield return expression;
            }
        }
    }

    private static Expression VisitPrimitive(PropertyInfo member, Expression path)
    {
        Expression memberAccess = Expression.MakeMemberAccess(path, member);

        if (member is { PropertyType.IsValueType: true })
        {
            memberAccess = Expression.Convert(memberAccess, typeof(object));
        }

        return memberAccess;
    }

    private static IEnumerable<Expression> VisitNestedObject(MemberInitExpression nestedMemberInit, PropertyInfo member, Expression path, int level)
    {
        var nestedPath = Expression.MakeMemberAccess(path, member);

        foreach (Expression expression in VisitMemberInit(nestedMemberInit, nestedPath, level))
        {
            yield return expression;
        }
    }

    private static IEnumerable<Expression> VisitNestedCollection(MethodCallExpression toArrayCall, PropertyInfo member, Expression path, int level)
    {
        if (toArrayCall.Arguments[0] is not MethodCallExpression selectCall)
            yield break;

        if (selectCall.Arguments[1] is not LambdaExpression selectLambda)
            yield break;

        var nestedPath = Expression.MakeMemberAccess(path, member);

        var memberInit = (MemberInitExpression)selectLambda.Body;

        level++;

        ParameterExpression parameter = MakeParameter(memberInit.Type, level);

        var selectMethod = typeof(Enumerable)
            .GetMethods(BindingFlags.Static | BindingFlags.Public)
            .Single(mi => mi.Name == nameof(Enumerable.Select) &&
                          mi.GetParameters()[1].ParameterType.GetGenericArguments().Length == 2)
            .MakeGenericMethod(parameter.Type, typeof(object));

        foreach (var expression in VisitMemberInit(memberInit, parameter, level))
        {
            LambdaExpression lambda = Expression.Lambda(expression, parameter);

            yield return Expression.Call(null, selectMethod, nestedPath, lambda);
        }
    }

    private static ParameterExpression MakeParameter(Type type, int lambdaLevel)
    {
        return Expression.Parameter(type, "p_" + lambdaLevel);
    }
}

P.S.:
I'm using this setup:
AutoMapper - 12.0.1
HotChocolate.AspNetCore - 13.0.5
HotChocolate.Data.AutoMapper - 13.0.5
HotChocolate.Data.EntityFramework - 13.0.5
Microsoft.EntityFrameworkCore - 7.0.5


Lifted a restriction

The restriction for "ADLS Gen 2 account must be deployed in the same region as your Power BI tenant. An error occurs if the locations of the resources are not in the same region" is updated. Customers can now use ADLS Gen2 account for different regions.

@emiharadapredmore : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Fix broken headings in Markdown files

GitHub changed the way Markdown headings are parsed, so this change fixes it.
See bryant1410/readmesfix for more information.
Tackles bryant1410/readmesfix#1

Change only what changed. Closing (as this is obviously not a human)

Sorry! :see_no_evil: This was affected by bryant1410/readmesfix#4 I'm a human!

This should be affected only by persons. Not bots pretending to be real persons.
I'm really bad that you are doing this.. Is this just a way to get more commits and stats on github?
At least state on your pull request that it is an automated pull request. Don't make the github community worse with bots.

Hey. This was automated, yes. And I'm not trying to get more commits nor stats on GH. If I tried to get more stats on GH, I'd just add random old commits to public pointless repos.
Ok, maybe I could have stated this was automated, but I don't see how the community gets worse because of this. I'm just trying to fix repos after the GH breaking change.

I see your point, but get my: We all hate spam on email, even thought they have something to offer.
When span is a obviously newsletter, we can filter. But when is a bot disguised, it's hard to tell the difference and know if the person really matter the subject.. In this case, I don't think you even know what is the purpose of this library if you didn't stopped by because of me replying to it...
Anyway, if you are doing this automated, it's called spam even if it's with the intention of something good. So, make sure to let people know that ;)

I disagree with the definition of spam :slightly_smiling_face: For me, spam is undesired email, regardless of it being automated or not.
And I could perfectly have created this PR manually without knowing the library, by just passing by (following a link or whatever) and seeing that the readme is broken.
Also, people know it's automated if they follow the first link.

That would have been great, but as you could see, your "automation" didn't help at all. Will you still fix your PR? if not, let it be.
Just make sure to say it's automated if it's not you pushing the buttons.


Headers

Any plans for adding headers to the request and also reading headers from the response?

Sorry to comment on a  closed issue, but response.headers doesn't work for me
(lldb) frame variable -L response.headers
0x7aee0ccc: ([String : String]?) response.headers = Some
(lldb) frame variable -L response.headers["Content-Type"]
error: unable to find any variable expression path that matches 'response.headers[Content-Type]'

hmmm, not sure why, is the request completing successfully? This example prints out the whole request, including the headers.
let task = HTTPTask()
task.GET("http://vluxe.io", parameters: nil, completionHandler: { (response: HTTPResponse) in
    println("response is \(response.description)")
})

The debugger in Swift isn't always great at inspecting variable so you might want to try this as well:
if let heads = self.headers {
    for (key, value) in heads {
        println("\(key): \(value)\n")
    }
} else {
	println("no headers found")
}

This seems to have been lost when you refactored the API.  I don't see this in HTTPSerializeProtocol or JSONParameterSerializer.
I did a quick review of the code and I did not see any way with the new API to set the Authorization header.  I am also looking to use AWT for my app that is using SwiftHTTP.


Improve errors

Background
We currently generate errors with the follow format:
E  openapi_tester.exceptions.DocumentationError: Mismatched types, expected a str type.
E
E  Expected: str
E
E  Received: {"color": "Red", ...}
E
E  Sequence: init.array.item.object:key:...

This works well in some cases, and seems like a useful format (though we could rework this, if anyone makes a better suggestion) since it provides users with information on:

What is wrong
What we expected the response data to look like, based on the schema
What the response data actually looked like
Where in the schema the error is

The issue
Message only
In some cases, like when raising errors related to:

Any reference handling
multipleOf
pattern compiling

The original format breaks down, for many reasons, but the main thing is that points 2-4 from the list above no longer apply. All we need is the message:
E  openapi_tester.exceptions.DocumentationError: Expected data to match one or more of the documented anyOf schema types, but found no matches.

Custom expected examples
In other cases, like for:

pattern validation
enum validation

Only the second point breaks down. In these cases it would be nice to, not only remove the expected attribute, but provide a custom value to represent what we expected based on the schema in these cases. Something like this:
E  openapi_tester.exceptions.DocumentationError: The response value is not valid with respect to the enum specified in the schema.
E
E  Expected: "Cat" or "Dog"
E
E  Received: "Turtle"

Things to consider
Error messages don't have to end up looking like the examples above, but we should rework the way we raise errors, and we should build in some flexibility to provide different formats, without needing to create new exception types.
We will add the possibility of adding custom validators soon, and the custom validators will need to be given the same flexibility our internal validators get. This implicitly makes our error handling a part of the packages public API and means breaking changes to error formatting will require major version bumps.

I would like to suggest that we remove the error message:
BEFORE
E  openapi_tester.exceptions.DocumentationError: Mismatched types, expected a str type.
E
E  Expected: str
E
E  Received: {"color": "Red", ...}
E
E  Sequence: init.array.item.object:key:...

AFTER
E  openapi_tester.exceptions.DocumentationError:
E
E  Expected: str
E
E  Received: {"color": "Red", ...}
E
E  Sequence: init.array.item.object:key:...

I would also like to suggest we update Sequence - we need to track the property keys more precisely and to be clear this is a map, so instead of init something like
Sequence: path:method:array.item:key:...

OR
Sequence: path:method:object:key:...

Yes let's definitely rework the sequence in a separate PR - it's a feature I put in place in 5 minutes that's never really been polished, so I agree there's definitely a lot of room for improvement.
As for removing the messages, I don't have any reservations against it as long as it doesn't make it harder for users to figure out what the problem is. It seems like we're losing information if we just remove the messages everywhere. Perhaps we could combine it with more specific error types or something.

I don't think we need to lose info. Rather, expected and received should be clear - if they are not already.
I would rename ''sequence'' into something more informative, e.g. ''location'', or ''identifier'' or ''path''.

Resolved in #216. Reference/sequence will be redone in a separate PR.


fix "this is null" in Popper.js

concerning issue https://github.com/Akryum/v-tooltip/issues/814

Good catch!


Cryptic warning on poor class names

The waring on poor class names is not too great:
The class name 'minion' doesn't match '[A-Z][a-zA-Z0-9]*'

Fixed in latest push.


Realm-graphql-service closes the realm and shows weird errors

I have a problem here with the realm-graphql-service.
Its randomly happens, sometimes frequent, sometimes appears only once a day.
This is the logs:
0|npm      | info: Got signal SIGINT. Shutting down
0|npm      | info: Loaded feature token capabilities=[Sync], expires=Wed Apr 19 2017 14:15:29 GMT+0000 (UTC)
0|npm      | info: Realm Object Server version 2.8.1 is starting
0|npm      | info: [sync] Realm sync server started ([realm-core-5.1.2], [realm-sync-2.2.15])
0|npm      | info: [sync] Directory holding persistent state: /home/ec2-user/xeleqt-object-server/data/sync/user_data
0|npm      | info: [sync] Backup mode: master_with_no_slave
0|npm      | info: [sync] Log level: info
0|npm      | info: [sync] Download log compaction is enabled
0|npm      | info: [sync] Max download size: 32000000 bytes
0|npm      | info: [sync] Listening on <IP_ADDRESS>:33831 (sync protocol version 22)
0|npm      | info: [sync] HTTP Connection[1]: Connection is closed after HTTP response.
0|npm      | info: [sync] HTTP Connection[3]: Connection is closed after HTTP response.
0|npm      | info: [sync] HTTP Connection[5]: Connection is closed after HTTP response.
0|npm      | info: [sync] HTTP Connection[7]: Connection is closed after HTTP response.
0|npm      | info: [sync] HTTP Connection[9]: Connection is closed after HTTP response.
0|npm      | info: [sync] HTTP Connection[10]: Connection is closed after HTTP response.
0|npm      | info: [sync] HTTP Connection[13]: Connection is closed after HTTP response.
0|npm      | info: [sync] HTTP Connection[15]: Connection is closed after HTTP response.
0|npm      | info: [sync] HTTP Connection[16]: Connection is closed after HTTP response.
0|npm      | info: [sync] HTTP Connection[19]: Connection is closed after HTTP response.
0|npm      | info: [sync] HTTP Connection[22]: Connection is closed after HTTP response.
0|npm      | info: Realm Object Server has started and is listening on http://<IP_ADDRESS>:9080
0|npm      | info: [sync] HTTP Connection[25]: Connection is closed after HTTP response.
0|npm      | info: [sync] HTTP Connection[27]: Connection is closed after HTTP response.
0|npm      | Error: Cannot access realm that has been closed.
0|npm      |     at queryResolver.(anonymous function) (/home/ec2-user/xeleqt-object-server/node_modules/realm-graphql-service/dist/service.js:244:40)
0|npm      |     at resolveFieldValueOrError (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:498:12)
0|npm      |     at resolveField (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:462:16)
0|npm      |     at /home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:311:18
0|npm      |     at Array.reduce ()
0|npm      |     at executeFields (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:308:42)
0|npm      |     at executeOperation (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:236:122)
0|npm      |     at executeImpl (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:133:26)
0|npm      |     at Object.execute (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:110:229)
0|npm      |     at doRunQuery (/home/ec2-user/xeleqt-object-server/node_modules/apollo-server-core/dist/runQuery.js:122:42)
0|npm      | Error: Cannot access realm that has been closed.
0|npm      |     at queryResolver.(anonymous function) (/home/ec2-user/xeleqt-object-server/node_modules/realm-graphql-service/dist/service.js:244:40)
0|npm      |     at resolveFieldValueOrError (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:498:12)
0|npm      |     at resolveField (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:462:16)
0|npm      |     at /home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:311:18
0|npm      |     at Array.reduce ()
0|npm      |     at executeFields (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:308:42)
0|npm      |     at executeOperation (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:236:122)
0|npm      |     at executeImpl (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:133:26)
0|npm      |     at Object.execute (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:110:229)
0|npm      |     at doRunQuery (/home/ec2-user/xeleqt-object-server/node_modules/apollo-server-core/dist/runQuery.js:122:42)
0|npm      | Error: Cannot access realm that has been closed.
0|npm      |     at queryResolver.(anonymous function) (/home/ec2-user/xeleqt-object-server/node_modules/realm-graphql-service/dist/service.js:244:40)
0|npm      |     at resolveFieldValueOrError (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:498:12)
0|npm      |     at resolveField (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:462:16)
0|npm      |     at /home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:311:18
0|npm      |     at Array.reduce ()
0|npm      |     at executeFields (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:308:42)
0|npm      |     at executeOperation (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:236:122)
0|npm      |     at executeImpl (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:133:26)
0|npm      |     at Object.execute (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:110:229)
0|npm      |     at doRunQuery (/home/ec2-user/xeleqt-object-server/node_modules/apollo-server-core/dist/runQuery.js:122:42)
0|npm      | Error: Cannot access realm that has been closed.
0|npm      |     at queryResolver.(anonymous function) (/home/ec2-user/xeleqt-object-server/node_modules/realm-graphql-service/dist/service.js:244:40)
0|npm      |     at resolveFieldValueOrError (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:498:12)
0|npm      |     at resolveField (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:462:16)
0|npm      |     at /home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:311:18
0|npm      |     at Array.reduce ()
0|npm      |     at executeFields (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:308:42)
0|npm      |     at executeOperation (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:236:122)
0|npm      |     at executeImpl (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:133:26)
0|npm      |     at Object.execute (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:110:229)
0|npm      |     at doRunQuery (/home/ec2-user/xeleqt-object-server/node_modules/apollo-server-core/dist/runQuery.js:122:42)
0|npm      | Error: Cannot access realm that has been closed.
0|npm      |     at queryResolver.(anonymous function) (/home/ec2-user/xeleqt-object-server/node_modules/realm-graphql-service/dist/service.js:244:40)
0|npm      |     at resolveFieldValueOrError (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:498:12)
0|npm      |     at resolveField (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:462:16)
0|npm      |     at /home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:311:18
0|npm      |     at Array.reduce ()
0|npm      |     at executeFields (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:308:42)
0|npm      |     at executeOperation (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:236:122)
0|npm      |     at executeImpl (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:133:26)
0|npm      |     at Object.execute (/home/ec2-user/xeleqt-object-server/node_modules/graphql/execution/execute.js:110:229)
0|npm      |     at doRunQuery (/home/ec2-user/xeleqt-object-server/node_modules/apollo-server-core/dist/runQuery.js:122:42)
0|npm      | info: Got signal SIGINT. Shutting down
0|npm      | info: Loaded feature token capabilities=[Sync], expires=Wed Apr 19 2017 14:15:29 GMT+0000 (UTC)
0|npm      | info: Realm Object Server version 2.8.1 is starting
0|npm      | info: [sync] Realm sync server started ([realm-core-5.1.2], [realm-sync-2.2.15])
0|npm      | info: [sync] Directory holding persistent state: /home/ec2-user/xeleqt-object-server/data/sync/user_data
0|npm      | info: [sync] Backup mode: master_with_no_slave
0|npm      | info: [sync] Log level: info
0|npm      | info: [sync] Download log compaction is enabled
0|npm      | info: [sync] Max download size: 32000000 bytes
0|npm      | info: [sync] Listening on <IP_ADDRESS>:46101 (sync protocol version 22)
0|npm      | info: [sync] HTTP Connection[1]: Connection is closed after HTTP response.
0|npm      | info: [sync] HTTP Connection[3]: Connection is closed after HTTP response.
0|npm      | info: [sync] HTTP Connection[5]: Connection is closed after HTTP response.
0|npm      | info: [sync] HTTP Connection[7]: Connection is closed after HTTP response.
0|npm      | info: [sync] HTTP Connection[9]: Connection is closed after HTTP response.
0|npm      | info: [sync] HTTP Connection[10]: Connection is closed after HTTP response.
0|npm      | info: [sync] HTTP Connection[13]: Connection is closed after HTTP response.
0|npm      | info: [sync] HTTP Connection[15]: Connection is closed after HTTP response.
0|npm      | info: [sync] HTTP Connection[17]: Connection is closed after HTTP response.
0|npm      | info: [sync] HTTP Connection[18]: Connection is closed after HTTP response.
0|npm      | info: [sync] HTTP Connection[22]: Connection is closed after HTTP response.
0|npm      | info: Realm Object Server has started and is listening on http://<IP_ADDRESS>:9080
0|npm      | info: [sync] HTTP Connection[25]: Connection is closed after HTTP response.
0|npm      | info: [sync] HTTP Connection[27]: Connection is closed after HTTP response.
0|npm      | info: [sync] HTTP Connection[31]: Connection is closed after HTTP response.
0|npm      | info: [sync] HTTP Connection[33]: Connection is closed after HTTP response.
0|npm      | { FetchError: network timeout at: http://<IP_ADDRESS>:46101/xeleqt-production/realms/files/%2Fxeleqt-production
0|npm      |     at Timeout._onTimeout (/home/ec2-user/xeleqt-object-server/node_modules/node-fetch/index.js:126:13)
0|npm      |     at ontimeout (timers.js:482:11)
0|npm      |     at tryOnTimeout (timers.js:317:5)
0|npm      |     at Timer.listOnTimeout (timers.js:277:5)
0|npm      |   name: 'FetchError',
0|npm      |   message: 'network timeout at: http://<IP_ADDRESS>:46101/xeleqt-production/realms/files/%2Fxeleqt-production',
0|npm      |   type: 'request-timeout' }
^[
When this error appears, i really have to restart the ROS, which i think a bad practice.
Thanks in advance for the help :D

Follow up

We still don't have a way to reproduce this which hinders any debugging efforts 😔

Duplicate of https://github.com/realm/realm-graphql-service/issues/39


NSOperation's completionBlock is called twice.

Try the following test.
func testCompletionBlock() {
    let queue = OperationQueue()
    let blockOp = BlockOperation()
    let exp = expectationWithDescription("Block finished")
    blockOp.addCompletionBlock {
        exp.fulfill()
    }
    queue.addOperation(blockOp)
    waitForExpectationsWithTimeout(5, handler: nil)
}

Then the test runner will log:

Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'API violation - multiple calls made to -[XCTestExpectation fulfill] for Block finished.'

The same error appear even if I set the completionBlock directly, which I think make no difference when only one completionBlock exists.
Discussion Xcode would not always stop the test as the error pops up, I think this is due to a race condition that the second call of fulfill() may happen either before or after waitForExpectationsWithTimeout has seen the effect of the first call (in case of after, the test has finished running and succeeded). Anyway, this is probably irrelevant of the issue. Isn't the completionBlock guaranteed to execute only once by Foundation framework?

Hi @difujia, thanks for the feedback. This is a great spot, and may explain some of the flaky unit tests in #119. Will look into it today.

Looking into this more closely. Have re-produces your exception. Hmm, I think you're right about a race condition with regard to fulfill - which explains why some of the tests are inconsistent (I need to refactor them).
But, also, there is this test: https://github.com/danthorpe/Operations/blob/development/Tests/Core/OperationTests.swift#L142 which admittedly doesn't test against multiple invocation of the completion block, but according to this bug, should be failing.
Also, I'll just add, that most of the tests fulfill their expectation using a BlockObserver (https://github.com/danthorpe/Operations/blob/development/Tests/Core/OperationTests.swift#L100) instead of a completion handler. This is because it's a little ill-defined exactly when that completion handler will be run by NSOperation.
Anyway, I will continue to investigate...

Okay, so more info. This bug doesn't have anything to do with chaining completion blocks (as @difujia said). Changing the test case a little bit to see what's happening into this:
func test__block_operation_with_default_block_runs_completion_block_once() {
    let _queue = OperationQueue()
    let expectation = expectationWithDescription("Test: \(__FUNCTION__)")
    let operation = BlockOperation()
    operation.log.severity = .Verbose

    operation.completionBlock = {
        print("*** I'm in a completion block on \(String.fromCString(dispatch_queue_get_label(DISPATCH_CURRENT_QUEUE_LABEL)))")
    }

    operation.addObserver(BlockObserver { op, errors in
        expectation.fulfill()
    })

    _queue.addOperation(operation)
    waitForExpectationsWithTimeout(3, handler: nil)
}

Will output the following in the test runner logs:
Test Case '-[iOS_OperationsTests.BlockOperationTests test__block_operation_with_default_block_runs_completion_block_once]' started.
Unnamed Block Operation: Initialized -> Pending
Unnamed Block Operation: Pending -> EvaluatingConditions
Unnamed Block Operation: EvaluatingConditions -> Ready
Unnamed Block Operation: Ready -> Executing
Unnamed Block Operation: did start
Unnamed Block Operation: Executing -> Finishing
Unnamed Block Operation: did Test Case '-[iOS_OperationsTests.BlockOperationTests test__block_operation_with_default_block_runs_completion_block_once]' passed (0.001 seconds).
finish with no errors.
*** I'm in a completion block on Optional("com.apple.root.utility-qos")
*** I'm in a completion block on Optional("com.apple.root.utility-qos")
Unnamed Block Operation: Finishing -> Finished

Notice that as the operation transitions Finishing -> Finished is when the Test Case passes and the completion block is executed - and yes, it's executed twice (on the same queue). This is all essentially happening at the same time but on different queues, but of course the console displays lines (intrinsically serial) so we read these events ordered thanks to race conditions between the queues. This is not a problem, just helps to understand what is going on.

Okay, think I'm getting to the bottom of it. I've added a KV observer to the isFinished property. When this is asserted the completion block is executed. So I have this in my observer (the test case):
override func observeValueForKeyPath(keyPath: String?, ofObject object: AnyObject?, change: [String : AnyObject]?, context: UnsafeMutablePointer<Void>) {
    if context != &completionBlockObservationContext {
       super.observeValueForKeyPath(keyPath, ofObject: object, change: change, context: context)
    }

    guard let
        oldIsFinished = change?[NSKeyValueChangeOldKey] as? Bool,
        newIsFinished = change?[NSKeyValueChangeNewKey] as? Bool
        else { return }

    switch (oldIsFinished, newIsFinished) {
    case (false, true):
        print("Asserting isFinished for the first time.")
    case (_, true):
        print("Asserting isFinished again.")
    default:
        break
    }
}

And if you put a break point on the 2nd assertion that isFinished is true, it's from the start method of Operation where we call super.start(). FYI - this is exactly what the docs say not to do:

IMPORTANT

At no time in your start method should you ever call super. When you define a concurrent operation, you take it upon yourself to provide the same behavior that the default start method provides, which includes starting the task and generating the appropriate KVO notifications. Your start method should also check to see if the operation itself was cancelled before actually starting the task. For more information about cancellation semantics, see Responding to the Cancel Command.

Which is from: https://developer.apple.com/library/mac/documentation/Cocoa/Reference/NSOperation_class/
However, during the development of this framework, there was quite a bit of back and forth with @davedelong and (via him) a Foundation engineer who wrote/maintains NSOperation, about what start() in NSOperation does. If you take a look at the sample code from WWDC: https://developer.apple.com/sample-code/wwdc/2015/?q=NSOperations it is exactly the same. I'm going to go through my emails to remember what the discussion was about.
Anyway, regardless, I suspect that there is some KVC happening on the state property, which is triggering start to be called after it's already finished - so... narrowing down the bug to the state transition...

Okay, so, the override of start is not relevant. In fact, I think this code is actually redundant, as the equivalent cancellation logic exists inside main(). So the start override can be removed. All the tests still pass, except the one with the completion block being run twice.

Hi @difujia - pretty sure I've fixed this now. The bug was not as wide reaching as I initially thought it might be, as I could only re-produce it using BlockOperation and in particular the style of block which is used for the default block implementation.
If you're able to install Operations from the development branch and re-check, that would be great. I'll leave this issue here until it's verified fixed.
Again, thanks for the feedback!

I am able to re-produce this problem on both the development and release branches with an Operation class (as opposed to just BlockOperation). It's strange though - sometimes the code will run correctly for a few goes but if you Clean and Re-build it will have problems.
I did a bunch of testing on this tonight and I figured some stuff out. You were definitely right about the start method on the NSOperation superclass triggering a willChangeValueForKeyPath("isFinalized") which causes the completionBlock to run. Then when the state changes, an additional willChangeValueForKeyPath("isFinalized") triggers which causes the completionBlock to run again. This seems to be the crux of the issue.
Based on my testing, it looks like you have to override the start method but you can't call super.start(). For example, this is what the start method should look like:
public override final func start() {
	if cancelled {
		finish()
	} else {
		main()
	}
}

This seems to be the approach recommended by apple in the past for subclassing NSOperation (see https://gist.github.com/LoganWright/eb844fdec0873182abd8). It's very odd that the "Advanced NSOperations" Sample Code makes a point of calling super.start()
If you are curious, here is a really simple way to re-produce the double-completion block calling:
class DoubleCompletionOp : NSOperation {
    var _finished = false
    override var finished: Bool {
        return _finished
    }
    override final func main() {
        willChangeValueForKey("isFinished")
        _finished = true
        didChangeValueForKey("isFinished")
    }
}

And as soon as you override start it works fine:
class SingleCompletionOp : NSOperation {
    var _finished = false
    override var finished: Bool {
        return _finished
    }
    override final func start() {
        main()
    }
    override final func main() {
        willChangeValueForKey("isFinished")
        _finished = true
        didChangeValueForKey("isFinished")
    }
}

The tests seem to all run correctly with this new start method.

Thank you guys for investigating this issue.
I agree that the call on super in override of start() method is the key. The problem can still be reproduced like this:
func testCompletionBlock() {
        measureBlock {
            let queue = OperationQueue()
            let op = DummyOperation()
            let expectation = self.expectationWithDescription("Test: \(__FUNCTION__)")
            op.addCompletionBlock {
                expectation.fulfill()
            }
            queue.addOperation(op)
            self.waitForExpectationsWithTimeout(10, handler: nil)
        }
}
class DummyOperation: Operation {

}

Here measureBlock is just used to let the test run multiple times in each run.
The sample code from Apple has actually been changed a couple of times. I have a very early version of the sample (June 10, 2015) in my computer. The override of start() in that version was:
override final func start() {
    assert(state == .Ready, "This operation must be performed on an operation queue.")

    state = .Executing

    for observer in observers {
        observer.operationDidStart(self)
    }

    execute()   // No override of main in this version.
}

I don't have time to go deeper into this issue right now. Hope this snippet could help.

@kevinbrewster @difujia okay, great stuff. I've left the start() code commented out at the moment. I knew this was the issue, didn't have any concrete info from @davedelong about why we ended up calling super.start(). Will refactor this now.
Also - great idea on using measureBlock.

@difujia @kevinbrewster proposed fix: https://github.com/danthorpe/Operations/commit/689c29e3822533e806ea495fd2d28992b77df5e8 has been merged into development. Will leave this issue open until either of you can confirm that the completion block is no longer called twice. My test for this passes.

@danthorpe - I can confirm that the completion block is no longer called twice with the latest development code.

Thanks for testing @OperationKit  - will leave the issue open for @difujia and @kevinbrewster as they found/diagnosed the bug.

(sorry i was logged in under the wrong account earlier. changing to avoid confusion).
@danthorpe - I can confirm that the completion block is no longer called twice with the latest development code.

@kevinbrewster are you the mysterious @OperationKit? ;)

Okay, closing this issue now as fix has been verified. Thanks @kevinbrewster & @difujia!

haha - my secret identity has been revealed!!
I had actually been working on my own framework for Operations before I discovered this one only a few days ago.

Ha! Cool. I know there are a couple of other projects. If you have any more feature requests or suggestions - feel free to create an issue - I'm eager to develop this more to increase its utility and fix annoyances people are having.
I've not got time to look at your other issue now, but I will do tomorrow.

Thanks @danthorpe. I too believe this issue has been addressed. For other improvements, I think a generic way of piping operation results could be great. I will create an issue for discussion.


Feature toggle for round-robin assignment

Hi nice workflow which we would like to use with an additional functionality:
When specifying a team as either reviewer or assignee and then configure numberOfAssignees, it always take the first user from the list. It would be nice to have some round-robin functionality here.
Expected Behavior
Have a feature-toggle like randomReviewers and randomAssignees that can be set to true to assign random ppl from the team or list of people.
Possible Solution
Some changes around https://github.com/wow-actions/auto-assign/blob/c3701c76c743672a7e7054993b075bd167707652/src/util.ts#L67

Hi @plowin Thanks for opening your issue here!
The code here using lodash's sampleSize method to randomly pick count user from teams.  So it already has random functionality.
It would be very helpful if you could provide the contents of your workflow.

Hi, thanks a lot for the quick reply!
With a team of 3-4 people, it is probably very unlikely to always have the same assignee:

My workflow is here, I tried using a team even though this is not documented:
    steps:
      - uses: wow-actions/auto-assign@v2
        with:
          GITHUB_TOKEN: ${{ secrets.TEST_TOKEN }}
          reviewers: org-plowin/team-plowin
          assignees: org-plowin/team-plowin # b1tamara,plowin,stefanlay
          numberOfReviewers: 1
          numberOfAssignees: 1
          skipKeywords: wip, draft

@plowin
I catch the issue. Adding a team to assignees is not supported by GitHub. So I'm trying to fetch the members of a team, then add the members as assignees in this PR.
In order to test the action I create a team "authors" with two members: "bubkoo" and "srm-bot". And update my test workflow with:
        with:
          GITHUB_TOKEN: ${{ env.BOT_TOKEN }}
          addReviewers: true
          addAssignees: true
          numberOfReviewers: 1
          numberOfAssignees: 1
          reviewers: |
            /authors
          assignees: |
            this-is-an-unknown-assignee
            /authors
          skipKeywords: wip
          excludeLabels: dependencies, wip

The result looks like what you'd expect.

Note that the default ${{ secrets.GITHUB_TOKEN }} does not have the permission to add teams as reviewers or to list members of a team. As a workaround:

First, create a personal access token (PAT) with repo and admin:org permissions.
Then, make the PAT available to our actions by adding the token as a secret.
Finally, replace the GITHUB_TOKEN with the new secret, e.g. GITHUB_TOKEN: ${{ secrets.NAME_OF_MY_SECRET_CONTAINING_PAT_WITH_REPO_ACCESS }} instead of GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}.

Or with a cool but slightly cumbersome solution: create a private github app for your org with custom permissions and avatar, then use the app token in out workflow, e.g. wow-actions-bot.

cool, I'll try it out once you release a v3!

Hi, in my test-action I tried multiple variations and in the end even copied the exact action you pasted above but I keep getting the same error:
runTypeError: Cannot read properties of undefined (reading 'length')
--
runCannot read properties of undefined (reading 'length')

I am not familiar with debugging GH actions and error output does not help a lot here. v2 worked with the variations I tried.
The GH history of this file shows you what I tried, they all fail with the exact same error. The PAT has org:admin scope.

@plowin  v3.0.2 was released. Try to rerun your workflow.


git write-back for helm application does not work

Describe the bug
I deploy an application as following. In charts/superset path, there is a helm chart manifests and it is deployed correctly.
But its sourceType is none so argo image updater skips this application even if I set the annotations.
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  annotations:
    argocd-image-updater.argoproj.io/write-back-method: git
    argocd-image-updater.argoproj.io/git-branch: main
    argocd-image-updater.argoproj.io/image-list: main=ortege01.azurecr.io/ortege/superset:ortege
    argocd-image-updater.argoproj.io/main.update-strategy: digest
    argocd-image-updater.argoproj.io/main.helm.image-name: image.repository
    argocd-image-updater.argoproj.io/main.helm.image-tag: image.tag
  name: superset
  namespace: argo-cd
spec:
  project: default
  destination:
    server: "https://kubernetes.default.svc"
    namespace: superset
  source:
    path: charts/superset
    repoURL: https://github.com/ortege-xyz/gitops
    targetRevision: HEAD
    helm:
      releaseName: superset

To mitigate this sourceType issue, I released the chart to gh pages and updated the app source like this.
spec:
  source:
    chart: superset
    repoURL: https://ortege-xyz.github.io/gitops
    targetRevision: 0.10.10
    helm:
      releaseName: superset

Then the sourceType is detected as Helm so image updater detects this. But now it is failing because of auth error.
level=error msg="Could not update application spec: could not get creds for repo 'https://ortege-xyz.github.io/gitops': credentials for 'https://ortege-xyz.github.io/gitops' are not configured in Argo CD settings" application=superset

This ortege-xyz.github.io gh release page url is not the git url in fact.
How can I use git write-back for helm application?
To Reproduce
I mentioned above.
Expected behavior
New image tag should be committed to git repo.
Additional context
Add any other context about the problem here.
Version
0.12.2
Logs

argo-cd/argocd-image-updater-79cbbdc6b4-9z527[argocd-image-updater]: time="2023-11-26T18:29:41Z" level=error msg="Could not update application spec: could not get creds for repo 'https://ortege-xyz.github.io/gitops': credentials for 'https://ortege-xyz.github.io/gitops' are not configured in Argo CD settings" application=superset
argo-cd/argocd-image-updater-79cbbdc6b4-9z527[argocd-image-updater]: time="2023-11-26T18:29:41Z" level=info msg="Processing results: applications=1 images_considered=2 images_skipped=0 images_updated=0 errors=1"

Hi @josedev-union ， May I know how did you fix it? I also meet this issues.

Hi @josedev-union ， May I know how did you fix it? I also meet this issues.

@kaweit i uploaded the chart manifest in the same git repository, charts/superset path instead of publishing via gh pages. Then updated the application spec as following;
  source:
    path: charts/superset
    repoURL: git@github.com:Ortege-xyz/gitops
    targetRevision: main


Need Help

I am having the B-L475E-IOTO1A dico kit and i would like to extract data from the two microphones, i tried your code but i don't know how to get back the data with TeraTerm.

hi,
in order to receive data using TeraTerm you need send them using USART1 module. to do that you can either use printf function or HAL api for usart.
HAL_UART_Transmit(&huart1, (uint8_t *)payloadBuffers[0], payload_len,1000);
//		  HAL_UART_Transmit_DMA(&huart1, (uint8_t *)payloadBuffers[0], payload_len);
//		  while(UARTTXCompleteFlag==0);

this is a simple code that can send payload using uart1. you can add this to main loop to first four if conditions and bypass the queue entirely, i should also mention that the throughput of usart1 is very low so you need to lower the sampling rate of microphones.
if(micDma0HlfCmplt==1)
{
 micDma0HlfCmplt=0;
 HAL_UART_Transmit(&huart1, (uint8_t *)payloadBuffers[0], payload_len,1000);
}

Thanks for that it works good, with your initial code you get the data with a python script ? if I want to use the WIFI I have to create a python script ?

I send the data to a UDP server. you can use any software that can act as a UDP server or write one yourself using python. This is the code for a simple UDP server in python. You should also set SSID and password of you wifi network and IP/PORT of your UDP server in this file.

I did this code to create my server

And did this to the STM32CubeIDE but I can't get the data, and i don't really know where is the problem maybe the port is not listening ?

You need to change host parameter of python code to '<IP_ADDRESS>'. And in config.h, you should set IP address of your computer. Your computer should (preferably) be on the same network as your board. here is a simple python that works fine for me:
import socket

def get_ip():
    s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    s.settimeout(0)
    try:
        # doesn't even have to be reachable
        s.connect(('<IP_ADDRESS>', 1))
        IP = s.getsockname()[0]
    except Exception:
        IP = '<IP_ADDRESS>'
    finally:
        s.close()
    return IP

IPAddr = get_ip()
print("set this ip in config.h:" + IPAddr)

UDPServerSocket = socket.socket(family=socket.AF_INET, type=socket.SOCK_DGRAM)
UDPServerSocket.bind(('<IP_ADDRESS>', 8080))
bufferSize = 4000
print("UDP server up and listening")
while (True):
    bytesAddressPair = UDPServerSocket.recvfrom(bufferSize)

    message = bytesAddressPair[0]

    address = bytesAddressPair[1]

    clientMsg = "Message from Client:{}".format(message)
    clientIP = "Client IP Address:{}".format(address)

    print(clientMsg)
    print(clientIP)

You should also make some changes to my project. Comment out these two lines in main.c (line 314):
while(AccelGyroMeterInitialization()!=0)
   HAL_Delay(500);

You should also change these line in wifi.c to use UDP instead of TCP (line 148):
	sendCommand(UDPCommand,strlen(UDPCommand));
//	sendCommand(TCPcommand,strlen(TCPcommand));

You can use ifconfig to get ip address of your network interface and use that in config file.
The problem with these two lines is that they try to initialize AccelGyro on x-nucleo board. If you do not have any x-nucleo attached to your IOT board then the initialization would always fail. If you have a x-nucleo board then you can use these sensors without any problem.

Here is my settings i made the changes but still not receiving idk why.
And ok thanks yeah i only have the IoT board

Could you show me the output of ifconfig? Is your computer connected to the same wifi network? did you set the port correctly in both codes?

Yeah the pc and the board has the same connection, maybe the port is not set correctly in python script i think but i don't understand which line contains the port is it this :
s.connect(('<IP_ADDRESS>',1 )) and i replace 1 by my port

Yes that's right. You should replace 1 with your port number. I had some problem with wifi security mode before. Are you sure the board is connected to the wifi network?
You should check security mode of your wifi network. for me, i set it to "WPA-PSK/WPA2-PSK AUTO" in access point settings and set the security mode to 3 in WifiInit.
Security mode is defined in line 21 of wifi.h. You can set it based on this table:

I had some problem with other security mode and only 3 worked for me.

I think this wifi module doesn't support WPA3. Can you change your phone hotspot's security settings to "None"? if you use None then you can use C3=0.

none so no password i can delete the line for the password ?

I don't know. I haven't tested it myself.

it's not possible i will try with WPA2 i don't have to change anything ?

WPA2 has 2 modes: TKIP or AES. you should use 3 for AES and 5 for TKIP. i have only tested AES with mode 3.

Ok thanks I will try at home I hope it will works

last question when i have to choose a port i have to take an established port or a time_wait

Established ports are already taken by some other process so you can't use them. As far as i know these states are only relevant for statefull connection like TCP. UDP is stateless, hence there is no state for a UDP connection. Operating system always checks for free ports and if the port you choose is already taken, it will prevent your application for accessing it and your python code would throw an error.

Did it works with a WPA2-CCMP ?

I don't know. I've only tested it with a 2.4 GHz wifi in "WPA-PSK/WPA2-PSK AUTO" mode and C3=3. Does your WiFi work in 2.4 GHz band or 5 GHz ?

i will try with wifi but i am not WPA2 :( at my job we use 5 gHz

According to datasheet you can use mode 2 or 4 for wpa. Have you tried them?

tried with 2 and 4 but still nothing...

Try uncommenting these two line at the end of WifiInit function (line 156 of wifi.c).
sendCommand(SetSettingsSpace, strlen(SetSettingsSpace));
sendCommand(SaveSettings, strlen(SaveSettings));

And connect to a WPA2 wifi  with C3=3. I just tested it with my laptop's hotspot and it works.

I tried with a WPA2 personal 2,4 GHz, but nothing even with trying all the things you tell me maybe I am unlucky I don't know.

Does your board get listed on your access point's client list?

According to this post, by default these devices can't connect to an access point  on channel 11 or above.


tools: libboard: n3000: update interface glob to linux-dfl-5.11

Signed-off-by: Martin Hundebøll<EMAIL_ADDRESS>

Coverage remained the same at 75.368% when pulling 4905eddd3b768db0fa46616d90210cf0e274b186 on hundeboll/linux-5.11-eth-info into 921be4ee9a9b81cb5bd954301131eb834fe2b0d7 on master.

Coverage remained the same at 75.368% when pulling 4905eddd3b768db0fa46616d90210cf0e274b186 on hundeboll/linux-5.11-eth-info into 921be4ee9a9b81cb5bd954301131eb834fe2b0d7 on master.

can you change "dfl*./net/npac" to "dfl*/net/npac*"
dfl*.* to dfl* , glob works for both dfl-fme* and dfl_dev*

we want to match both dfl-fmeX.Y and dfl_dev.X, for which dfl*.* works.

can you change "dfl*./net/npac" to "dfl*/net/npac*"
dfl*.* to dfl* , glob works for both dfl-fme* and dfl_dev*

we want to match both dfl-fmeX.Y and dfl_dev.X, for which dfl*.* works.


(Ignore) Revert "KAFKA-15661: KIP-951: Server side changes (#14444)"

This reverts commit f38b0d886c01bf4e83a08ca7c8a4da01c2d0c686.
Trying to find the root cause of org.apache.kafka.tiered.storage.integration.ReassignReplicaShrinkTest failing in CI.
Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

JDK 21 and Scala 2.13 build failed due to https://github.com/apache/kafka/pull/14739.

org.apache.kafka.tiered.storage.integration.ReassignReplicaShrinkTest does not fail without this commit. We should revert it for the time being.

The failed build is due to https://github.com/apache/kafka/pull/14739.

org.apache.kafka.tiered.storage.integration.ReassignReplicaShrinkTest does not fail without this commit. We should revert it for the time being.

Here are existing builds where it was already failing -- I see that the error is different though. This is why I probably thought it was a non-issue.
https://ci-builds.apache.org/job/Kafka/job/kafka/job/trunk/2371/#showFailuresLink
https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14695/6/


Add CUDNN_POOLING_MAX_DETERMINISTIC

This is related to https://github.com/chainer/chainer/pull/7390

Jenkins, test this please

Jenkins CI test (for commit 434cebe4884c17d7749699f4874e37f35dc8890b, target branch master) succeeded!

LGTM!


Add icon colors to editor theme

Follow-up to #58759.
I noticed that the same issue was happening with the disabled icon color. While I'm at it I also added icon_pressed_color and icon_disabled_color to the other Button types but I'm not sure if that is actually necessary since I don't think we use those types of Button with icons in the Editor.

Thanks!


Change default branch to dev

As pull requests must be made against dev branch, maybe you could switch the default branch to dev so that this branch is pre-selected when creating pull requests.
https://help.github.com/articles/setting-the-default-branch/

This is a good idea for contributors. Not sure if its good for ppl wanting to use the boilerplate

Yes, I'm not sure if that also changes the branch used by git clone.

My understanding is that changing default to dev means a git clone will default to dev branch, which kind of defeats the purpose of having a dev branch.
Anyone know if it's possible to have separate "default" branches for clone and PR? That would be cool.
In the meantime, devs must pay attention to detail, but it's also easy to "Edit" a PR and change the base branch.

My understanding is that changing default to dev means a git clone will default to dev branch, which kind of defeats the purpose of having a dev branch.

Same, this is why I haven't done it. For now we'll just need to edit the PR base branches manually, hopefully Github adds what we need someday!


feat: additional load balancer config

Description
The landing-zone-vsi module by default created an Application load balancer with the primary network interface subnet. The actual resource ibm_is_lb allows to configure these variables using profile and subnets. This PR adds them as optional config.
Release required?

[ ] No release
[x] Patch release (x.x.X)
[ ] Minor release (x.X.x)
[ ] Major release (X.x.x)

Release notes content

Run the pipeline
If the CI pipeline doesn't run when you create the PR, the PR requires a user with GitHub collaborators access to run the pipeline.
Run the CI pipeline when the PR is ready for review and you expect tests to pass. Add a comment to the PR with the following text:
/run pipeline

Checklist for reviewers

[ ] If relevant, a test for the change is included or updated with this PR.
[ ] If relevant, documentation for the change is included or updated with this PR.

Merge actions for mergers

When merging, use a relevant conventional commit message that is based on the PR contents and any release notes provided by the PR author. The commit message determines whether a new version of the module is needed, and if so, which semver increment to use (major, minor, or patch).
Merge by using "Squash and merge".

/run pipeline

/run pipeline

/run pipeline

/run pipeline

@rajatagarwal-ibm It seems we have no examples or tests that actually test adding load balancers to VSIs. Can we add an example (or maybe just update the fscloud example) to do this? That way consumers can see how to use the variable, and we test it as part of our tests

@ocofaigh sure, let me update fscloud example to consume LB

@rajatagarwal-ibm maybe actually it makes sense to create a new example. Perhaps a custom example where we provision the module with extra customisations such as load balancer and additional data volumes. Check if we have other gaps too.
I added additional data volumes to the fscloud example to cover that gap a while ago, but perhaps there are more

@ocofaigh sure, other than Loadbalancer, there are "Secondary Interface Variables" which are not used in the fscloud example. Perhaps I can add Load Balancer and "Secondary Interface Variables" to the new example. Should I also remove "block_storage_volumes" from the FSCloud example to the new custom example?

I think you can leave it in the fscloud example, since it would test being encrypted using HPCS.

As discussed with @ocofaigh, I will be creating a new example "complete" which creates LB and Secondary Interface. I will also use Key Protect for encryption in this example which will enable parallel execution of FSCloud tests(uses HPCS) with the complete tests.
Along with that, I will also update tests:

Add complete test to the PR test
Replace FSCloud Upgrade test with complete Upgrade test in the PR Test
Run FSCloud standard test in the PR test (no change needed).

@rajatagarwal-ibm In https://github.com/terraform-ibm-modules/terraform-ibm-landing-zone-vsi/pull/539 Steve has added placement group support and updated the basic example to use it. I think it should be moved the complete example you are going to create

/run pipeline

/run pipeline

/run pipeline

/run pipeline

/run pipeline

/run pipeline

/run pipeline

:tada: This PR is included in version 2.9.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Update Plugin Artifacts Links for v0.177.4

Automated changes by create-pull-request GitHub action

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Add a A-Pose character in Misc

Add it so we can just skip the steps on how to make a class A-pose

This isn't really something I could implement unless you literally want me to try and make it 1) killbind, 2) respawn by changing classes twice 3) emptying the ammo and holding the attack key and then 4) walking to and picking up the dropped weapon from where you kill binded. This also doesn't work on all classes, and there's also the dispenser option. It just isn't worth the time investment to add this no offense

no.


Causal Effect of confounder on outcome

Could you please tell me if it is possible to determine Causal Effect of confounder on the outcome?
Hope to hear from you soon. Thank you in advance!

@amit-sharma Could you please help me with the above question?
Sincerely,
Asha


If ITestListener.onStart(ITestContext) throws in a suite with multiple, parallel  sections, exception is swallowed

Steps to reproduce

have a parallel <suite> with multiple <test> sections
implement ITestListener.onStart(ITestContext) so that it throws in e.g. second <test>
run the suite using  Ant task

expected

ant task failure, breaking the build

observed

<test> for which listener threw, gets ignored
others are executed, so everything looks fine (thus if this is a CI job, the job remains green)

Could you share a runnable example?

No reproducible sample available to proceed further. Closing this ticket


State persist exception

Hi, just wonder how to do the exception if I don't want to persist the whole state.
Or how can I specify the module properties to save.
I am using js-cookie.Thanks.

Hi please read the readme thoroughly, the issue queue is not a help forum.


Teste para responder por email

Teste bem loko
Att,
William Oliveira
[image: http://]
http://woliveiras.com.br http://www.woliveiras.com.br/
http://www.woliveiras.com.br/
On Thu, Aug 25, 2016 at 3:55 PM, Teles<EMAIL_ADDRESS>wrote:

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/woliveiras/woliveiras.github.io/issues/11, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AEDAkWuAzWb7-bj_84xAUaJAFbA5vBXnks5qjeUrgaJpZM4JtY4P
.

XABLAU!!!
https://woliveiras.com.br
Att,
William Oliveira
[image: http://]
http://woliveiras.com.br http://www.woliveiras.com.br/
http://www.woliveiras.com.br/
On Thu, Aug 25, 2016 at 3:56 PM, William Oliveira<EMAIL_ADDRESS>wrote:

Teste bem loko
Att,
William Oliveira
[image: http://]
http://woliveiras.com.br http://www.woliveiras.com.br/
http://www.woliveiras.com.br/
On Thu, Aug 25, 2016 at 3:55 PM, Teles<EMAIL_ADDRESS>wrote:

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/woliveiras/woliveiras.github.io/issues/11, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AEDAkWuAzWb7-bj_84xAUaJAFbA5vBXnks5qjeUrgaJpZM4JtY4P
.

Primeiramente fora Temer

Segundamente, receba essa atualização no seu email
Att,
https://woliveiras.com.br

Fechando o teste bem loko


Cleanups

Initial clean ups for user-defined features which are now redundant as those can be found using the presence of the 4CCs which we already parse.
~add new issue template, improve existing templates~ (cherry picked, merged)
make unrecognized features fail the pipeline
fix all user-defined features in all files
add missing boxes (WIP)

closes #39

@DenizUgur could you take a look how to resolve the failing test. I moved out sample entries defined in ISOBMFF to codecs.json in this commit 5ef2d49, and it seems to not like it.

This happened because in that commit you removed the only entry that listed its type as SampleEntry. I'll have to check if this error can be silenced. There is another hierarchical check for these kind of indirect type matches. Maybe this was a corner case.
{
    "fourcc": "encs",
    "description": "Encrypted Systems stream",
-    "type": "SampleEntry",
    "containers": ["stsd"],
    "syntax": "TODO: Bug. see 'rest'"
}


Add EntityMut system param

Closes #162

Codecov Report
Patch coverage: 100.00% and no project coverage change.

Comparison is base (aec50c0) 100.00% compared to head (eb9fdcb) 100.00%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@            Coverage Diff            @@
##              main      #193   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files          123       124    +1
  Lines         8578      8629   +51
=========================================
+ Hits          8578      8629   +51

Impacted Files
Coverage Δ

crates/modor/src/lib.rs
100.00% <ø> (ø)

crates/modor/src/system_params/mod.rs
100.00% <ø> (ø)

crates/modor/src/app.rs
100.00% <100.00%> (ø)

crates/modor/src/system_params/entity_mut.rs
100.00% <100.00%> (ø)

crates/modor/src/system_params/world.rs
100.00% <100.00%> (ø)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Dependabot Bumps

Description
Bumps the following packages:

Energinet.DataHub.Core.FunctionApp.TestCommon from 2.3.3 to 2.3.4.
Energinet.DataHub.Core.FunctionApp.TestCommon from 2.3.3 to 2.3.4.

References

#0000

Codecov Report

Merging #63 (09512ba) into main (21a1a72) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main      #63   +/-   ##
=======================================
  Coverage   80.54%   80.54%
=======================================
  Files          25       25
  Lines         293      293
  Branches       12       12
=======================================
  Hits          236      236
  Misses         51       51
  Partials        6        6

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more


DM-45088: Revert "Update Butler library dependencies"

Roll back to the previous Butler version, since the new one is unusable due to DM-45088.
This reverts commit 2e231e389c5afc07adab5820d3a04409ba98d761.

The fix for this bug won't be released in a form consumable by datalinker and vo-cutouts until weekly 27, so rolling back for now in case someone needs to do something with these projects in the meantime.


STORAGE: Removing project from Connection.

Only needed for create_bucket() and get_all_buckets().
Fixes #726.

Coverage remained the same at 100.0% when pulling 068b9fad90c2e1e9a588f6e27ebf6858d6a07091 on dhermes:detach-project-from-connection into d6680440d9215eb2064bf99aefa1cc35db3fcb4e on GoogleCloudPlatform:master.

Maybe we should add project as a keyword argument to _BucketIterator.

LGTM

Changes Unknown when pulling b91b2af678b4ef684fafe592346b9d054e16b618 on dhermes:detach-project-from-connection into ** on GoogleCloudPlatform:master**.


as a user, I want to install tw4a using google play

The instructions to build tw4a seem not-impossible, but for me (and other non-android-deleloper users) installing tw4a from google play would be the easiest/ best way.

This is planned as soon as we reach 1.0. Currently there is precompiled 0.1 version available for downloading right from our GitHub.

TW4A is FLOSS so don't forget about F-Droid. Here's their inclusion policy and how-to. AFAIU they usually build their apps by themselves (and there are like, I dunno, 4 guys so kudos to them), but we could send them metadata. Manual for reference.


how to handle the number of anchor box?

i am a beginner class of detection with deep learning so i don't know how to handle anchor box.
i try to change filter numbers of loc_layers(21 to 11)  in multibox in faceboxes.py  but i have an error
"Runtime error : the expanded size of the tensor (11584) must match the exisitng size(21824) at non singleton dimension1. Target sizes : [32 ,11584 ,4]. Tensor sizes : [32 ,21824 ,1]"
i think 21824 is number of default(anchor) box and 11584 is number of prediction box
so what should i do to decrease number of filters?
and i have one more qusetion
this paper has anchor densification module but i don't want to use this module fully, i want to use half of anchor densification  to decrease my university project's latency so i have change prior_box.py 33th line
"dense_cx = [xself.steps[k]/self.image_size[1] for x in [j+0, j+0.25, j+0.5, j+0.75]]
--> dense_cx = [xself.steps[k]/self.image_size[1] for x in [j+0, j+0.5]]"
but i have an similar error above problem so i need your advice

It's probably easier to take out one of the Inception layers. This will give a ~16% speed up for very little accuracy drop off.

what is 3 dense and 2 normal anchors? as i know, 3232 feature map have 4 dense anchor box and  6464 feature map have 2 dense anchor box.  128128,256256,512*512 feature maps has normal anchor box. Is this wrong?

Sorry, 3 normal anchors and 2 are dense. One is 4 and one is 2, yes.
The size depends on the input resolution.

i am a beginner class of detection with deep learning so i don't know how to handle anchor box. i try to change filter numbers of loc_layers(21 to 11) in multibox in faceboxes.py but i have an error "Runtime error : the expanded size of the tensor (11584) must match the exisitng size(21824) at non singleton dimension1. Target sizes : [32 ,11584 ,4]. Tensor sizes : [32 ,21824 ,1]" i think 21824 is number of default(anchor) box and 11584 is number of prediction box so what should i do to decrease number of filters?
and i have one more qusetion
this paper has anchor densification module but i don't want to use this module fully, i want to use half of anchor densification to decrease my university project's latency so i have change prior_box.py 33th line
"dense_cx = [x_self.steps[k]/self.image_size[1] for x in [j+0, j+0.25, j+0.5, j+0.75]] --> dense_cx = [x_self.steps[k]/self.image_size[1] for x in [j+0, j+0.5]]"
but i have an similar error above problem so i need your advice

hi did you solve your problem successfully？Could you please tell me how you changed the dense?


Problem with building for M1 with arm64

Hi everyone
I used 15798ac9c2611d5c7f9ba832e2c9159bdd8945f2 of cryptopp-cmake with cryptopp v8.7.0
=> Project : cryptopp v8.7.0 (91409cd8f6fc)
-- Crypto++ from user-specified location at: <PATH>/cryptopp
=> Module : cryptopp
-- [cryptopp] CMake version 3.25.2
-- [cryptopp] System Darwin
-- [cryptopp] Processor arm64
-- [cryptopp]     CMAKE_OSX_ARCHITECTURES :
-- [cryptopp] CMAKE_HOST_SYSTEM_PROCESSOR : arm64
-- [cryptopp]      CMAKE_SYSTEM_PROCESSOR : arm64
-- [cryptopp] Target architecture detected as: arm64 -> CRYPTOPP_ARMV8
-- [cryptopp] Performing Test CRYPTOPP_ARM_NEON_HEADER
-- [cryptopp] Performing Test CRYPTOPP_ARM_NEON_HEADER - Success
-- [cryptopp] Performing Test CRYPTOPP_ARM_ACLE_HEADER
-- [cryptopp] Performing Test CRYPTOPP_ARM_ACLE_HEADER - Success
-- [cryptopp] Performing Test CRYPTOPP_ARM_SIMD
-- [cryptopp] Performing Test CRYPTOPP_ARM_SIMD - Success
-- [cryptopp] Performing Test CRYPTOPP_HAVE_ARM_CRC32
-- [cryptopp] Performing Test CRYPTOPP_HAVE_ARM_CRC32 - Failed
-- [cryptopp] Performing Test CRYPTOPP_HAVE_ARM_AES
-- [cryptopp] Performing Test CRYPTOPP_HAVE_ARM_AES - Success
-- [cryptopp] Performing Test CRYPTOPP_HAVE_ARM_PMULL
-- [cryptopp] Performing Test CRYPTOPP_HAVE_ARM_PMULL - Failed
-- [cryptopp] Performing Test CRYPTOPP_HAVE_ARM_SHA1
-- [cryptopp] Performing Test CRYPTOPP_HAVE_ARM_SHA1 - Success
-- [cryptopp] Performing Test CRYPTOPP_HAVE_ARM_SHA2
-- [cryptopp] Performing Test CRYPTOPP_HAVE_ARM_SHA2 - Success
-- [cryptopp] Performing Test CRYPTOPP_HAVE_ARM_SM3
-- [cryptopp] Performing Test CRYPTOPP_HAVE_ARM_SM3 - Failed
-- [cryptopp] Performing Test CRYPTOPP_HAVE_ARM_SHA3
-- [cryptopp] Performing Test CRYPTOPP_HAVE_ARM_SHA3 - Failed
-- [cryptopp] Performing Test CRYPTOPP_HAVE_ARM_SHA512
-- [cryptopp] Performing Test CRYPTOPP_HAVE_ARM_SHA512 - Failed
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- [cryptopp] Platform: ARMv8
-- [cryptopp] Compiler definitions:  CRYPTOPP_DATA_DIR="<PATH>/cryptopp";CRYPTOPP_ARM_NEON_HEADER=1;CRYPTOPP_ARM_ACLE_HEADER=1;CRYPTOPP_DISABLE_ARM_CRC32=1;CRYPTOPP_DISABLE_ARM_PMULL=1
-- [cryptopp] Compiler options:
-- [cryptopp] Build type: Release

And after thousands of lines
  ld: error: undefined symbol: cryptogams_AES_set_encrypt_key
  >>> referenced by rijndael.cpp:351 (<PATH>/cryptopp/rijndael.cpp:351)
  >>>               rijndael.cpp.o:(CryptoPP::Rijndael::Base::UncheckedSetKey(unsigned char const*, unsigned int, CryptoPP::NameValuePairs const&)) in archive <PATH>/cryptopp-cmake-modern/cryptopp/libcryptopp.a

  ld: error: undefined symbol: cryptogams_AES_set_decrypt_key
  >>> referenced by rijndael.cpp:355 (<PATH>/cryptopp/rijndael.cpp:355)
  >>>               rijndael.cpp.o:(CryptoPP::Rijndael::Base::UncheckedSetKey(unsigned char const*, unsigned int, CryptoPP::NameValuePairs const&)) in archive <PATH>/cryptopp-cmake-modern/cryptopp/libcryptopp.a

  ld: error: undefined symbol: cryptogams_AES_encrypt_block
  >>> referenced by rijndael.cpp:359 (<PATH>/cryptopp/rijndael.cpp:359)
  >>>               rijndael.cpp.o:(CryptoPP::Rijndael::Enc::ProcessAndXorBlock(unsigned char const*, unsigned char const*, unsigned char*) const) in archive <PATH>/cryptopp-cmake-modern/cryptopp/libcryptopp.a

  ld: error: undefined symbol: cryptogams_sha1_block_data_order_neon
  >>> referenced by sha.cpp:352 (<PATH>/cryptopp/sha.cpp:352)
  >>>               sha.cpp.o:(CryptoPP::SHA1::HashMultipleBlocks(unsigned int const*, unsigned int)) in archive <PATH>/cryptopp-cmake-modern/cryptopp/libcryptopp.a

  ld: error: undefined symbol: cryptogams_sha1_block_data_order
  >>> referenced by sha.cpp:359 (<PATH>/cryptopp/sha.cpp:359)
  >>>               sha.cpp.o:(CryptoPP::SHA1::HashMultipleBlocks(unsigned int const*, unsigned int)) in archive <PATH>/cryptopp-cmake-modern/cryptopp/libcryptopp.a

  ld: error: undefined symbol: cryptogams_sha256_block_data_order_neon
  >>> referenced by sha.cpp:940 (<PATH>/cryptopp/sha.cpp:940)
  >>>               sha.cpp.o:(CryptoPP::SHA256::HashMultipleBlocks(unsigned int const*, unsigned int)) in archive <PATH>/cryptopp-cmake-modern/cryptopp/libcryptopp.a

  ld: error: undefined symbol: cryptogams_sha256_block_data_order
  >>> referenced by sha.cpp:947 (<PATH>/cryptopp/sha.cpp:947)
  >>>               sha.cpp.o:(CryptoPP::SHA256::HashMultipleBlocks(unsigned int const*, unsigned int)) in archive <PATH>/cryptopp-cmake-modern/cryptopp/libcryptopp.a
  clang++: error: linker command failed with exit code 1 (use -v to see invocation)
  ninja: build stopped: subcommand failed.

Related to: https://github.com/abdes/cryptopp-cmake/pull/38
and possible https://github.com/abdes/cryptopp-cmake/commit/3a2474943c8f10cda7f2c8321f2bbad64b457307
CRYPTOGAMS_ARM_AES defined in config_asm.h:377 and not used any cmake based variable except CRYPTOPP_DISABLE_ARM_NEON
I am not so deep in ARM arch but seems like we need to include cryptopp_SOURCES_ASM for arm64 too
Also i think it can be related to compile for Android

I'm sorry, but I mixed the logs from two different builds and the problem seems to be really related to android
I'll open another issue when I get deeper.

Ok. I fixed my problem
My patch is:
 if(CRYPTOPP_ARM32
-   AND CMAKE_SYSTEM_NAME STREQUAL "Linux"
+   AND (CMAKE_SYSTEM_NAME STREQUAL "Linux" OR CMAKE_SYSTEM_NAME STREQUAL "Android" )
    AND NOT CRYPTOPP_DISABLE_ARM_NEON
    AND NOT CRYPTOPP_DISABLE_ASM)
+  enable_language(ASM)
   list(
     APPEND cryptopp_SOURCES_ASM ${CRYPTOPP_PROJECT_DIR}/aes_armv4.S
     ${CRYPTOPP_PROJECT_DIR}/sha1_armv4.S ${CRYPTOPP_PROJECT_DIR}/sha256_armv4.S

Forgot about Android is also linux and trigger preprocessor around config_asm.h:377
Forgot to enable ASM and it leads to next errors:

  CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
  Missing variable is:
  CMAKE_ASM_COMPILE_OBJECT

According to cmake docs CMAKE_SYSTEM_NAME STREQUAL "Android" can be replaced by ANDROID variable since Cmake 3.7


Milestone 3 feedback

Write comments below, not in a separate thread.
P.S. This is my favorite team I'm giving them all A+s

Team JOANNA - K3
Nic
I like - I really like the app idea, and how it was presented with the humourous storyboards. Personally, I think it’s going to be really hard to gather all of that data, but if done correctly this app would be amazing.
I wish - I’m not sure how this could be improved, since it seems like there’s a very clear idea in mind. Maybe it would be cool to expand this idea and see what else could be implemented in the app. Is there any way you could find a way to save money on UberEATS as well?
Joseph
I like - I like how this app takes very commonly used services and makes them better and more convenient for people.
I wish - You could maybe add a speed feature that determines where and which app will provide the fastest service for people who want to get somewhere faster.
Ashkon
I like - how it is taking advantage of applications that are very commonly used in day to day life. I can see myself and a lot of others using this because of how tedious it is to constantly flip through applications. I also like how it factors the decisions in for your relative location.
I wish - that this would give you multiple options based on if you are riding in a group or solo. Sometimes taking a rideshare may be cheaper than all paying for a public transportation service. Right now it seems to only handle the solo case.
Anish
I like - how this will not only be comparing prices of Uber and Lyft. The thing that makes this application different is that it also recommends how to make the ride cheaper by giving a time to wait or a distance to walk. This is a really interesting idea and I hope it will not be challenging to implement the recommendation part.
I wish - that there would be a way to also recommend for different number of people as well as compare UberX with UberPool and other travel options

Zhiyu:
I like: the idea is very useful in real life.
I want: A paper model showing how users interact with the app would be great. Because how to implement the app in a user-friendly way is very important to attract users. Also, there is already a web application ride.guru has similar functionality. Would be better to think about how to add an unique feature that makes your app stands out.

Feedback from K6 Hoooookah
Claire:
This app directly speaks to me.  This is a great concept. I wish there was a way to compare Lyft as well. That would make the application more functional.
April:
I like that this app would genuinely be useful to people, especially those who want to save money.
I wish there was a way to differentiate between Uber and Lyft in your app's UI. I'm not exactly sure how the app is going to work. Will Uber and Lyft be compared with each other or separately?
Devon
I like that this app would help decide when to leave a location and find cheaper times to travel.
I wish we had more information on how surge information is displayed.  It is sometimes hard to predict and can be affected by weather, events, etc.  Would you be displaying previous surge trends?  Current trends?  Also, a similar app (different focus) exists: ride.guru  Just make sure different enough from this application.
I have one concern and that is safety.  The cheaper locations during a surge (or in general) may not be safest locations to wander to, especially late at night and/or while drunk/drugged.  Maybe integrate crime stats into the app to avoid this?

It's a good idea, but it's not clear where you will get your data from. Think of what you could pivot to, since there is no novelty in case this data isn't available. Also, be kinder to constructive criticism!
PS: These are just some questions, some of which I'm sure you would have already thought about. Don't take it as criticism! Good work with the storyboards. Make sure to iterate on and not commit to the sketches early on (they are just to make the idea concrete enough to present for feedback). Feel free to ping me on Messenger to discuss.


[HUDI-5697] Revisiting refreshing of Hudi relations after write operations on the tables

Change Logs
Currently, after most DML operations in Spark SQL, Hudi invokes Catalog.refreshTable
Prior to Spark 3.2, this was essentially doing the following:

Invalidating relation cache (forcing next time for relation to be re-resolved, creating new FileIndex, listing files, etc)
Trigger cascading invalidation (re-caching) of the cached data (in CacheManager)

As of Spark 3.2 it now additionally does LogicalRelation.refresh for ALL tables (previously this was only done for Temporary Views), therefore entailing

Whole table to be re-listed again by triggering FileIndex.refresh which might be costly operation.

This change attempts to revert back to previous behavior from Spark 3.1, where LogicalRelation are not refreshed until they are actually read again.
Impact
Avoiding costly FileIndex.refresh operations on the write path
Risk level (write none, low medium or high below)
High (we need to carefully explore repercussions of adjusting refreshing behavior on all Spark versions)
Documentation Update
N/A
Contributor's checklist

[ ] Read through contributor's guide
[ ] Change Logs and Impact were stated clearly
[ ] Adequate tests were added if applicable
[ ] CI passed

Cherry-picked this fix on Hudi 0.12.1 and tested with Spark 3.3.0 and it looks good! No listing requests after the hive sync when Spark refresh the table

CI is green:

https://dev.azure.com/apache-hudi-ci-org/apache-hudi-ci/_build/results?buildId=15252&view=results


Add Microsoft MPI

Hi!
It would be great if you add Microsoft MPI.
Link:
https://www.microsoft.com/en-us/download/details.aspx?id=41634
Thanks in advance!

Microsoft MPI is distributed as binaries only and the source code is not available. The most recent version can usually be found at https://msdn.microsoft.com/en-us/library/bb524831(v=vs.85).aspx and currently is 7.1 (available at https://www.microsoft.com/en-us/download/details.aspx?id=52981).
The problem that I encountered is that the binaries are redistributed via installers only and a simple archive to extract is not available. There are two installers 'msmpisdk.msi' which includes the developer components (.lib and header files) and 'MSMpiSetup.exe' which includes the runtime components (.dll and mpiexec.exe).
I was able to create a portfile that downloads the developer tools and extracts those by using 'msiexec' (which should be available on any windows machine?!) but I was not able to extract the runtime components without installing it (which i don't think is desired since it adds registry entries and may have conflicts if the msmpi runtime has been installed on the machine already.). In general the developer components are sufficient to compile any library that depends on msmpi but executing those files would still require the runtime components to be available.
Currently I do have the runtime components installed system-wide on my machine, but this comes with the risk that the versions of the runtime and the developer components do not match. I solution where the runtime is installed in the vcpkg directories would be preferred.
Does anyone have a solution or knows how to extract an .exe installer without installing it and without relying on tools like 7z (which itself has to be installed system-wide?)
Or is the current solution that installs the developer tools only sufficient enough and I should submit a pull request for the port?

If you require 7zip to do the extract, you can check for the existence of 7zip. If it is not installed, you can either:

fail the build and ask the user to install it
download a portable version of it (recommended).

We do this for various tools that are needed in order to perform the build (and are not required for the library's functionality after the build).
Take a look at scripts\vcpkg_find_acquire_program.cmake.
PYTHON2 uses option 1. Everything else uses option 2.
I think it would be reasonable to add 7zip there.


⚠ Samsung has degraded performance

In 85512ed, Samsung (https://samsung.com) experienced degraded performance:

HTTP code: 200
Response time: 2441 ms

Resolved: Samsung performance has improved in 224eccc.


Define alerting rules for CPU, Disk and RAM usage

Is this a BUG REPORT or FEATURE REQUEST?
FEATURE REQUEST
Define and add alerting rules for alertmanager for the following

RAM (if RAM usage >75%)
CPU (if CPU usage > 75%)
Disk (if Disk usage > 75%)
Send Alerts on slack / email

Closed via #1658


Slice data in data viewer

Show Slice Data panel for >1D tensors and ndarrays
Select first dimension by default
Show shape of tensor and currently-selected axis
Axis dropdown
Index dropdown
Refetch dataframe rows on axis/index selection change

Messaging is implemented, blocked on design of actual UI controls for slicing.

There is probably a lot to fully test here. But I pulled up a 3x3 and tried to test all the basics. All looked good to me.

Slicing using the expression worked as expected
Slicing using Axis / Index boxes also worked as expected
Axis / Index and the slice expression correctly updated based on the other changing
When slice expression could not be reflected in Axis / Index it was disabled
Could stop slicing data then re-enable and it saved the slice that I had.

Validated.


Use Snap install instructions for Ubuntu

PR Summary
This pull request modifies the installation instructions for Ubuntu to reference Snapcraft. The "Many Linux distributions" entry which references Snapcraft has also been moved into the supported platforms table as we've (I work on the Snapcraft team) confirmed the PowerShell snap is working across all Snap compatible distributions.
PR Checklist

[x] PR has a meaningful title

[x] Summarized changes

[x] Change is not breaking

[x] This PR is ready to merge and is not Work in Progress.

User-facing changes

[x] Not Applicable

Testing - New and feature

[x] Not Applicable or can only be tested interactively

I'm going to defer this to @TravisEz13. The change is kinda inconsistent with the other items in the table - going from a file extension (like the rest) to Snapcraft... but maybe there was a conversation that went on elsewhere that said this is fine.

@joeyaiello Can you comment?  I'm not sure we want to change the primary may we are asking people to install PowerShell to snap.  At least not yet.  We just fixed an issue when VS Code would not run from the snap.  I'm still listing the snap as experimental.

The snap install doesn't work ATM for WSL!
WSL is currently the easiest way to try LINUX distributions like Ubuntu on their own computer.
https://github.com/Microsoft/WSL/issues/2374

Also, in containers, but often containers and WSL require additional steps

@joeyaiello  Can you comment?

Yeah, we obviously love Snap, but I think we'd want to do a full test pass across Ubuntu distros using the Snap before we move them all over.
Also, has Snap been pushed into the older LTS versions as part of servicing? Or do you have any kind of idea what percent of users are pulling it in? My hypothesis is that folks on older LTS are less likely to want new dependencies (eg. Snap), but I don't have any real data there.
Lastly, the "more Linux distros" line makes sense to me in principle, and I feel like we went through the formatting and wording already with AppImage. Could we pull that up and see if it makes sense to match what we had there?

One problem still is that we have found that distributions not supported by DotNet, we cannot guarantee are supported.  So, the line about more distributions would need to move back down to the community table.