Recursive Directory list causes dart to abort

I have the following code which I'm using to do a scan of the complete file system under linux.
It appears that when the recursive list iterates through  /proc (or in that area) that an unexpeted st_mode (384) is returned and the dart aot aborts.
To reproduce the problem:
Save the following code to listing.dart
import 'dart:io';

void main(List<String> args) {
  var lister = Directory('/').list(recursive: true);
  lister.listen((entity) {
    print(entity.path);
  });
}

Compile the script:
dart2native listing.dart
Generated: /tmp/temp/listing.exe

Run the code using sudo
sudo ./listing.exe

....
/proc/slabinfo
/proc/softirqs
../../runtime/bin/directory_linux.cc: 204: error: Unexpected st_mode: 384

/proc/zoneinfo
/proc/buddyinfo
/proc/diskstats
/proc/key-users
....

System information:
dcli doctor
DCli version      0.21.0

OS                linux
OS Version        Linux 5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020
Path separator    /

dart version      2.9.1

dcli path         /<HOME>/.pub-cache/bin/dcli
dart exe path     /usr/bin/dart
dart path         /usr/bin/dart                                           which: /usr/bin/dart
dart2Native path  /usr/lib/dart/bin/dart2native                           which: /usr/lib/dart/bin/dart2native

pub path          /usr/lib/dart/bin/pub                                   which: /usr/lib/dart/bin/pub
Pub cache         /<HOME>/.pub-cache

Package Config    /tmp/temp/file:/<HOME>/git/dcli/.packages

PATH
                  /usr/local/sbin
                  /usr/local/bin
                  /usr/sbin
                  /usr/bin
                  /sbin
                  /bin
                  /usr/games
                  /usr/local/games
                  /snap/bin
                  /usr/lib/dart/bin
                  /<HOME>/apps/flutter/bin
                  /<HOME>/.pub-cache/bin
                  /<HOME>/.dcli/bin

$SHELL            /bin/bash
Detected SHELL    bash
Shell Start Scrip Not Found

Dart location(s)
                  /usr/bin/dart

Permissions
HOME              rwxr-xr-x <user>:<user> /<HOME>
.dcli             rwxrwxr-x <user>:<user> /<HOME>/.dcli
cache             rwxrwxr-x <user>:<user> /<HOME>/.dcli/cache
dependencies.yaml rw-rw-r-- <user>:<user> /<HOME>/.dcli/dependencies.yaml
templates         rwxrwxr-x <user>:<user> /<HOME>/.dcli/templates

I managed to produce a variation on the above error without the use of sudo.
void main(List<String> args) {
  var lister = Directory('/').list(recursive: true);

  lister.listen((entity) {
  })
   .onError((Object e, StackTrace st) {
    print(e);
    print(st);
  });

Yields:
FileSystemException: Directory listing failed, path = '/lib/ssl/private' (OS Error: Permission denied, errno = 13)

FileSystemException: Directory listing failed, path = '/proc/tty/driver' (OS Error: Permission denied, errno = 13)

../../runtime/bin/directory_linux.cc: 204: error: Unexpected st_mode: 384

version=2.9.1 (stable) (Unknown timestamp) on "linux_x64"
pid=574321, thread=574327, isolate_group=(nil)((nil)), isolate=(nil)((nil))
isolate_instructions=0, vm_instructions=559e77f5ff80
  pc 0x0000559e781e9eac fp 0x00007fc063efaf90 dart::Profiler::DumpStackTrace(void*)+0x7c
  pc 0x0000559e77f600b2 fp 0x00007fc063efb070 dart::Assert::Fail(char const*, ...)+0x82
  pc 0x0000559e78089d5a fp 0x00007fc063efb1f0 /usr/lib/dart/bin/dart+0x1a57d5a
  pc 0x0000559e78087d0c fp 0x00007fc063efb240 dart::bin::Directory::List(dart::bin::DirectoryListing*)+0x7c
  pc 0x0000559e7808894f fp 0x00007fc063efb270 dart::bin::Directory::ListNextRequest(dart::bin::CObjectArray const&)+0x11f
  pc 0x0000559e77f4f891 fp 0x00007fc063efb2c0 dart::bin::IOServiceCallback(long, _Dart_CObject*)+0x3a1
  pc 0x0000559e78164bf3 fp 0x00007fc063efbc30 dart::NativeMessageHandler::HandleMessage(std::__2::unique_ptr<dart::Message, std::__2::default_delete<dart::Message> >)+0xd3
  pc 0x0000559e78162836 fp 0x00007fc063efbca0 dart::MessageHandler::HandleMessages(dart::MonitorLocker*, bool, bool)+0x146
  pc 0x0000559e78162eea fp 0x00007fc063efbd00 dart::MessageHandler::TaskCallback()+0x1da
  pc 0x0000559e7826d8c8 fp 0x00007fc063efbd80 dart::ThreadPool::WorkerLoop(dart::ThreadPool::Worker*)+0x148
  pc 0x0000559e7826dd9c fp 0x00007fc063efbdb0 dart::ThreadPool::Worker::Main(unsigned long)+0x5c
  pc 0x0000559e781e605d fp 0x00007fc063efbe70 /usr/lib/dart/bin/dart+0x1bb405d
-- End of DumpStackTrace
Aborted (core dumped)

@bsutton, thanks for the report!
I see that you can reproduce this when the repro case is compiled via AOT; does this also reproduce when just run via dart (using JIT)?

The second example i provided in the comments occurs without compiling and
without Sudo.
On Wed, 26 Aug 2020, 6:03 am Devon Carew<EMAIL_ADDRESS>wrote:

@bsutton https://github.com/bsutton, thanks for the report!
I see that you can reproduce this when the repro case is compiled via AOT;
does this also reproduce when just run via dart (using JIT)?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dart-lang/sdk/issues/43176#issuecomment-680241180,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAG32ODOEVCBYN5YLW7ROFLSCQKHXANCNFSM4QKTMVNA
.

It broke due to unrecognized mask. I don't think this is a bug of us.
st_mode: 384 (which is 0600) doesn't fall into any valid category (Reg file, links, directories, sockets and so on). Checkout the man page: https://man7.org/linux/man-pages/man7/inode.7.html
Can you identify the file (It is probably not even a file though) that causing the problem? @bsutton

Its definitely a bug.
You can't have something as simple as a Directory.list causing dart to abend.
I believe the /proc directory is where linux mounts hardware.
https://man7.org/linux/man-pages/man5/proc.5.html
I have a vague recollection that proc uses additional bits but this isn't my area of expertise.
For what its worth I'm running a zfs file system but I'm sceptical that this is the issue.
I can't tell which file it is given as the above logs fail to report the filename.
If I run the stat command across the directory it reports no errors.
I've had a go at running a stat command over the directory but I'm not quite certain what I'm looking for.
If you have a command you would like me to run, happy to do so.

I don't think there is an easy way to debug. You have to write a simple c program. Here is a pseudo code version, which using almost the same system calls the VM uses.
// Open the directory
int dir_fd = opendir(pathname);

// Iterate all elements in the directory.
dirent* entry = readdir(dir_fd);

// entry->d_name is the file name

// Use fstatat() or stat() to check the result
int result = stat(path, &stat);
if (stat->st_mode == 0600) {
  // This is the entity we are looking for
  // In the VM, we throw the error.
}

Looks like some files in the /dev directory will yield the same error.

I've just reproduced the problem on a second pc.
The PC is running mandaro and ext4.
The crash also happens in the /proc directory on the above system.
So I think its safe to conclude:
This isn't a problem with the zfs file system.
This isn't a problem with a corrupt file system.

Looks like some files in /snap and /run show the same symptoms.

So I think the question now is what does 600 mean and what type of file/process object does it represent?

My C coding is rather rusty but I found the file:
DIRECTORY: /proc/1/task/1
DIRECTORY: /proc/1/task/1/fd
/proc/1/task/1/fd/4
st_mode 384
ino 11676 uid 0 gid 0  4
FOUND
Here is the script I ran:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <dirent.h>
#include <sys/stat.h>

void scan(char * dirname);

int main(int argc, char* argv[])
{
	printf("scanning dirname %s\n", argv[1]);
	scan(argv[1]);
}

void scan(char * dirname)
{
	DIR *dir;
    struct dirent *entry;
    int i;
    struct stat fileStat;
    dir = opendir(dirname);
    if (!dir)
    {
        printf("cannot open directory : %s\n",dirname);
	return;
    }

    do
    {
    	entry = readdir(dir);
	if (entry == NULL)
	{
		//printf("entry is null\n");
		break;
	}
	char entityPath [1024];
	sprintf(entityPath, "%s/%s", dirname, entry->d_name);
	if (strcmp(entry->d_name, ".")  ==0
	|| strcmp(entry->d_name, "..")  ==0
	|| strstr(entityPath, "self")  != NULL)
	{
		//printf("entriy is . or ..\n");
		continue;
	}
        int result = stat(entityPath, &fileStat);
	//printf("result %d\n", result);

	if (result == 0)
	{

		if (fileStat.st_mode == 0600)
		{
       			printf("%s/%s\n", dirname, entry->d_name);
       			printf("st_mode %ld\n", (long)fileStat.st_mode);
			printf("ino %ld ",(long)fileStat.st_ino);
			printf("uid %ld ",(long)fileStat.st_uid);
			printf("gid %ld ",(long)fileStat.st_gid);
        		printf(" %s\n", entry->d_name);
			printf("FOUND\n");
			exit(0);
		}
        	if (S_ISLNK(fileStat.st_mode))
        	{
       		printf("LINKD: %s/%s\n", dirname, entry->d_name);
        	}
        	else if (S_ISDIR(fileStat.st_mode))
        	{
       			printf("DIRECTORY: %s/%s\n", dirname, entry->d_name);
			char newdir [1024];
			sprintf(newdir, "%s/%s", dirname, entry->d_name);
			scan(newdir);
        	}
        	else if (S_ISREG(fileStat.st_mode))
        	{
       			printf("FILE: %s/%s\n", dirname, entry->d_name);
        	}
        	//printf(" %s\n", entry->d_name);
	}
	else {
		printf("can't stat %s\n", entry->d_name);
	}
	//printf("looping \n");
    } while (entry != NULL);
	closedir(dir);
}

I'm not certain what you code is doing but I refer to the readdir doco which stats that you must be able to handle an unknown file type.
I can't find any doco but is the issue that the file type mask is zero and the code aborts because it can't determine the file type.
If that is the case then the below doco indicates that you need to be prepared to return an unknown file type.
https://www.man7.org/linux/man-pages/man3/readdir.3.html
d_type This field contains a value indicating the file type, making
              it possible to avoid the expense of calling lstat(2) if fur‐
              ther actions depend on the type of the file.

              When a suitable feature test macro is defined (_DEFAULT_SOURCE
              on glibc versions since 2.19, or _BSD_SOURCE on glibc versions
              2.19 and earlier), glibc defines the following macro constants
              for the value returned in d_type:

              DT_BLK      This is a block device.

              DT_CHR      This is a character device.

              DT_DIR      This is a directory.

              DT_FIFO     This is a named pipe (FIFO).

              DT_LNK      This is a symbolic link.

              DT_REG      This is a regular file.

              DT_SOCK     This is a UNIX domain socket.

              DT_UNKNOWN  The file type could not be determined.

              Currently, only some filesystems (among them: Btrfs, ext2,
              ext3, and ext4) have full support for returning the file type
              in d_type.  All applications must properly handle a return of
              DT_UNKNOWN.

WIP: https://dart-review.googlesource.com/c/sdk/+/160705

Checkout /proc/[pid]/fd/ section: https://man7.org/linux/man-pages/man5/proc.5.html

Excellent and thanks.
Is there an eta on what version it will be released in?

Most likely September! If the change managed to be submitted before the deadline.

OK thanks.
And thanks for the help in getting it resolved.
On Fri, 28 Aug 2020 at 14:03, zichangg<EMAIL_ADDRESS>wrote:

Most likely September! If the change managed to be submitted before the
deadline.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dart-lang/sdk/issues/43176#issuecomment-682312225,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAG32OB53VKZ7TMWVE4MVVLSC4T7JANCNFSM4QKTMVNA
.


New command "deck file kong2kic" Kong configuration file to Kong Ingress Controller manifests.

New command "deck file kong2kic" Kong configuration file to Kong Ingress Controller manifests.
Currently, only ingress controller manifests are supported. Output format is always YAML.
Usage:
  deck file kong2kic [flags]

Flags:
  -a, --api string           [ingress|gateway] (default "ingress")
  -h, --help                 help for kong2kic
  -i, --input-file string    Kong spec file to process. Use - to read from stdin. (default "-")
  -o, --output-file string   Output file to write. Use - to write to stdout. (default "-")

Global Plugins map to KongClusterPlugins
Kong Services map to K8s Services. Kong Routes map to K8s Ingresses. To specify Kong specific features, K8s Ingress and Service resources eventually have a KongIngress resource associated to them.
Kong Consumer entities map to KongConsumer manifests.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.0 out of 2 committers have signed the CLA.:x: mod42:x: battlebyteYou have signed the CLA already but the status is still pending? Let us recheck it.

Last commit adds JSON output and can generate manifests using annotations for proxy and route (now default) or using KongIngress CRD. Code refactored to use builder + director pattern.

replaced by #1050


[TASK] Release Building Blocks 1.19.0

Description
Please provide a detailed description of the issue.
Example: As a (user or developer), I want to be able to (feature or function) so that (reason to implement).
Acceptance Criteria
Please describe what it means for this issue to be considered complete.

Closing this, as there is a different task for deployment.

Reopening as main to develop PR is open.


Null Pointer Exception in FastImageSource.java Glide lib doesn't allow null or empty string to be passed to GlideUrl object.

I have fix this issue and trying to raise the PR. but facing issue while pushing the code with below command:
git push<EMAIL_ADDRESS>Error : While I try to push my code , I get an error that access is denied to me (anchalarora).
Can anyone please help me to resolve this issue?

@DylanVann : Can you please provide me access to the repository , I am unable to push the fix(Null pointer exception fix) in the library.
While I try to push my code , I get an error that access is denied to me (anchalarora).

@anchalarora You need to fork this repo, push the fix into your forked repo, and then PR the fix to this repo.

@DylanVann : I have raised the PR, Please review it and merge.


feat(table): add row expansion

Summary

Added table row expansion

Change List (commits, features, bugs, etc)

(see commits in PR)

Related Issues

(none)

Acceptance Test (how to verify the PR)

View the "Table --> expansion" story in storybook.  Toggle a few rows.

:tada: This PR is included in version 1.6.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.6.0 :tada:
The release is available on npm package (@release-1.x dist-tag)
Your semantic-release bot :package::rocket:


Comfy example not working anymore

Hello everyone,
I was trying to get the comfyui example working on modal. However I am getting a weird behaviour :
I run the following commands :

in a terminal I run : modal serve comfyapp.py
in a second terminal I run : python3 comfyclient.py --dev --modal-workspace myworkspace--prompt "a white heron"

However as I see the logs in the first terminal, eveerything seems to be doubled (or run twice) and I get a weird error regarding the creation of /root/user/
It will also print continuously : Server still standing up...
Complete message below :
`Total VRAM 15095 MB, total RAM 347440 MB
Total VRAM 15095 MB, total RAM 347440 MB
xformers version: 0.0.26.post1
xformers version: 0.0.26.post1
Set vram state to: NORMAL_VRAM
Device: cuda:0 Tesla T4 : cudaMallocAsync
VAE dtype: torch.float32
Set vram state to: NORMAL_VRAM
Device: cuda:0 Tesla T4 : cudaMallocAsync
VAE dtype: torch.float32
Using xformers cross attention
Using xformers cross attention
Traceback (most recent call last):
File "/root/main.py", line 206, in
server = server.PromptServer(loop)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/server.py", line 71, in init
self.user_manager = UserManager()
^^^^^^^^^^^^^
File "/root/app/user_manager.py", line 20, in init
os.mkdir(user_directory)
FileExistsError: [Errno 17] File exists: '/root/user'
Import times for custom nodes:
0.0 seconds: /root/custom_nodes/websocket_image_save.py
0.0 seconds: /root/custom_nodes/image-resize-comfyui
downloading https://raw.githubusercontent.com/comfyanonymous/ComfyUI_examples/master/inpaint/yosemite_inpaint_example.png ... to /root/input
100%|██████████| 1.41M/1.41M [00:00<00:00, 189MB/s]
Server still standing up...
Server still standing up...
Server still standing up...
Server still standing up...
Server still standing up...
Server still standing up...
`
Also, the link to the UI is working correctly.
Here is my version of modal : 0.63.28
Thanks for your help !

If someone runs into the same issue just checkout to the commit : d33e74e6990f6f55c0bcb74e6decabfcd0b0a205
and it will get fixed


Operation not supported by RestApiOperationRunner

Hello! I'm working on a plugin to call an API. I have a POST method written in C#. Given that is only a POST action I don't get a response back so I had to add that 'plain/text' on OpenApiResponseWithBody. When testing in Swagger UI the content is printed as string saying that the POST method was successfully done. However when using that same function on the chat copilot app and with Sequential Planner I get the following error:
Unable to generate bot response. Details: Error: 400: Bad Request => Function invoke error: Error occurred while running plan step: The media type of the RestartVirtualMachine operation is not supported by RestApiOperationRunner.
The result in Swagger UI is this one:

What could I change to avoid this error? Thank you

It does not work with the Stepwise Planner either, I'm not sure how to send a POST method to Copilot and that it then tells the status of the request to the API. I've tried that my output is a json object and a plain text but neither of those work.

It looks like this exception is being thrown in Semantic Kernel code, not Chat Copilot code. Chat Copilot takes a dependency on Microsoft.SemanticKernel.Skills.OpenAPI, which is where the RestApiOperationRunner & related code lives. You might have better luck debugging this issue by cloning the Semantic Kernel repo and importing your plugin with the debugger attached, using one of the ImportAIPluginAsync functions.
Here is where the exception you're seeing is being thrown: RestApiOperationRunner.cs#L152
This looks to me like the mediaType variable might be empty, so I'm trying to trace where that gets populated from.
Alternatively, if you are willing to share your plugin with me, I can try to step through and see what exactly is happening to cause this error.
FYI @SergeyMenshykh if you have a chance to take a look here

I'm guessing it might be the type of output from the call to the API and the way it's being done but I'm not sure how could it be fixed. When trying in Postman everything works fine:
using System.Net;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.Functions.Worker.Http;
using Microsoft.Extensions.Logging;
using Microsoft.Azure.WebJobs.Extensions.OpenApi.Core.Attributes;
using Microsoft.OpenApi.Models;
using Microsoft.Identity.Client;
using System.Net.Http.Headers;
using Microsoft.Extensions.Caching.Memory;

namespace azuremanagementapi
{
    public class RestartAzureVM
    {
        private static readonly string ClientId = "";
        private static readonly string TenantId = "";
        private static readonly string ClientSecret = "";

        private readonly ILogger _logger;

        public RestartAzureVM(ILoggerFactory loggerFactory)
        {
            _logger = loggerFactory.CreateLogger<RestartAzureVM>();
        }

        [OpenApiOperation(operationId: "RestartVirtualMachine", tags: new[] { "ExecuteFunction" }, Description = "Restarts a specified virtual machine in Azure.")]
        [OpenApiParameter(name: "vmName", Description = "The name of the virtual machine to restart", Required = true, In = ParameterLocation.Query)]
        [OpenApiResponseWithBody(statusCode: HttpStatusCode.OK, contentType: "text/plain", bodyType: typeof(string), Description = "The status of the Virtual Machine restart operation")]
        [Function("RestartVirtualMachine")]

        public async Task<HttpResponseData> Run([HttpTrigger(AuthorizationLevel.Function, "post")] HttpRequestData req)
        {
            try
            {
                string vmName = req.Query["vmName"];
                if (string.IsNullOrEmpty(vmName))
                {
                    var badReqResponse = req.CreateResponse(HttpStatusCode.BadRequest);
                    badReqResponse.Headers.Add("Content-Type", "text/plain");
                    badReqResponse.WriteString("Virtual Machine name is required.");
                    return badReqResponse;
                }

                // Get access token
                var app = ConfidentialClientApplicationBuilder.Create(ClientId)
                    .WithClientSecret(ClientSecret)
                    .WithAuthority(AzureCloudInstance.AzurePublic, TenantId)
                    .Build();
                var result = await app.AcquireTokenForClient(new[] { "https://management.azure.com/.default" }).ExecuteAsync();
                string token = result.AccessToken;

                // Call the Azure management API to restart VM
                using (var httpClient = new HttpClient())
                {
                    httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", token);
                    var response = await httpClient.PostAsync($"https://management.azure.com/subscriptions/<subscriptionid>/resourceGroups/<resourcegroupname>/providers/Microsoft.Compute/virtualMachines/{vmName}/restart?api-version=2023-03-01", null);
                    string content = await response.Content.ReadAsStringAsync();

                    Console.Write($"Response Headers: {response.Headers}");
                    Console.Write($"Response Content: {content}");

                    var azureFunctionResponse = req.CreateResponse(response.IsSuccessStatusCode ? HttpStatusCode.OK : HttpStatusCode.BadRequest);
                    azureFunctionResponse.Headers.Add("Content-Type", "text/plain");

                    if (!response.IsSuccessStatusCode)
                    {
                        azureFunctionResponse.WriteString($"There was an error: {content}");
                        return azureFunctionResponse;
                    }
                    else
                    {
                        azureFunctionResponse.WriteString($"The {vmName} virtual machine was restarted successfully");
                        return azureFunctionResponse;
                    }
                }
            }
            catch (Exception ex)
            {
                _logger.LogError(ex, "An error occurred while trying to restart the VM.");
                var internalErrorResponse = req.CreateResponse(HttpStatusCode.InternalServerError);
                internalErrorResponse.Headers.Add("Content-Type", "text/plain");
                internalErrorResponse.WriteString("An error occurred. Please try again later.");
                return internalErrorResponse;

            }
        }
    }
}

@miguelhermar, since this issue is coming from the Semantic Kernel RestApiOperationRunner, I'm going to transfer this issue to that repo where they can help resolve it. Thanks!


Add vc creation, signing, and verification to web5 cmd

Initial vc package created here https://github.com/TBD54566975/web5-go/pull/60
Add support to the web5 cmd to create, sign and verify VCs

Its merged in! 👍


⚠ Bofrost.at has degraded performance

In 85f6c4e, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1992 ms

Resolved: Bofrost.at performance has improved in edc9da2.


Get ortools_vendor to use ament_vendor properly

ament_vendor has ament_environment_hooks that setup the LD_LIBRARY_PATH and PYTHONPATH properly for a ROS2 workspace.
I figured out how to use https://github.com/jspricke/ros-deb-builder-action which I had to patch to allow the sbuild tool that it uses, to access the network (Use ExternalProject/FetchContent).  See https://github.com/aosmw/ros-deb-builder-action/tree/feature/mw/enable_network
I then figured out the minor patch required to ortools so that it will build under strict dpkg-buildflags conditions.
Here is the github action running on my fork of ortools_vendor which should give confidence that a debian package is being produced.
https://github.com/aosmw/ortools_vendor/actions
There are options to upload the debian package and even serve it but I have not enabled those.
I have also run the action steps locally and have successfully run a Filelds2Cover build using using the debian package it produces.  (on ubuntu 22.04 humble so far)

@Gonzalo-Mier I think this is ready to be bloomed IMO.  I have setup github actions to prove that a debian package can be built successfully for both humble on jammy(22.04) and jazzy on noble(24.04)
This needs to be available on testing before the Fields2Cover packaging can use it.

Let's give it a try. Thanks @aosmw a lot for the effort!


Function Binding Rework

Currently we are working on a rework of how functions/function binding works:

Functions should not have a MatchesArgument function, instead they should have a list of input types with a set of auto cast rules (e.g. TINYINT + SMALLINT can be auto casted to SMALLINT + SMALLINT).
Functions should support overloading, i.e. there can be multiple functions with the same name but different input parameters (TINYINT + TINYINT, SMALLINT + SMALLINT)
OperatorExpression should be removed and all current operators should be implemented as functions (e.g. the addition operator "+" will become a set of functions with overloads).
Adding new functions should be cleaner and require less code than currently

The current interface that I have in mind for adding functions is as follows:
Simple Function (no overloads)
// prototype:
void AddFunction(string name, vector<SQLType> arguments, SQLType return_type, scalar_function_t function, bool has_side_effects = false, bind_scalar_function_t bind= nullptr, dependency_function_t dep = nullptr);
// usage:
functions.AddFunction("cos", {SQLType(SQLTypeId::DOUBLE)}, SQLType(SQLTypeId::DOUBLE) cos_function);

Function with Overloads:
FunctionSet set;
set.AddFunction("add", {SQLType(SQLTypeId::INTEGER)}, SQLType(SQLTypeId:: INTEGER), add_function_integer);
set.AddFunction("add", {SQLType(SQLTypeId::DOUBLE)}, SQLType(SQLTypeId::DOUBLE), add_function_double);
// etc...

This would replace the current setup of creating the static classes entirely, and would clean up a lot of headers/etc

I think mapping prefix/infix operators to functions is the right thing to do. The distinction is purely syntactic and should be handled in the parser. (Besides the term operator is already overladed enough in a database!)
Removing the MatchesArgument function seems problematic to me because it makes it hard to specify functions that have multiple incompatible signatures. For example: IIF and MAX have signatures type IIF( Boolean, type, type ) and type MAX( type ).  This could be worked around if the functions were keyed by signature and resolving a function provided a list of matches that the binder could then choose between. The previous system I worked on did things that way.
This is related to the promotion issue. The implementations deal in physical types, but the binder (not to mention the user!) should only have to worry about logical types like integer, real and so forth. This suggests that either the binder or the implementation should handle promotion once the logical type matching has been determined. Perhaps function signatures could specify physical types for binding, and the binder could match using logical types and then add casts? Casting could also be deferred to the function implementations, which would make the cast results cache-resident and also allow the function to optimise for particular types instead of just casting them.
As for the amount of code, I think the problem is that SQL effectively requires you to define objects in a 1970s function pointer style. For scalar functions, this is not too onerous because there is usually only one function, but for aggregates it gets complex. Aggregates define an internal, persistent state that requires a number of functions (initialize, update, merge, finalize, deallocate) as well as the binding information. Windowed aggregates probably add one or two more (I've done this but it was a while ago.) Often there are defaults that can be applied, especially if the aggregate is computed in-place: initialize sets to NULL, merge just aliases to update, finalize gathers and deallocate is a NOP, but any notion of "simplifying" should bear this in mind.
Maybe the way to think about it is to imagine the DDL statement. There will likely be a number of arguments, any one of which could be optional and some of which would trigger defaults (e.g., specifying the state as some kind of struct?) This sounds like filling in the elements of some kind of record, so maybe the current approach is the right one in the long term. Maybe the complexity is just the current "static function returning function pointer" approach, and we could just simplify it to "struct initialised with function pointers and inheriting some defaults from a base class"?

Removing the MatchesArgument function seems problematic to me because it makes it hard to specify functions that have multiple incompatible signatures. For example: IIF and MAX have signatures IIF( Boolean, type, type ): type and MAX( type ): type. This could be worked around if the functions were keyed by signature and resolving a function provided a list of matches that the binder could then choose between. The previous system I worked on did things that way.

For sure, this is exactly what we want to do. We provide a set of function overloads, e.g. for MAX this would be:
MAX(TINYINT) : TINYINT
MAX(SMALLINT) : SMALLINT
MAX(INT) : INT
MAX(BIGINT) : BIGINT
etc...
Then based on the input types we choose one of the overloads, with potential upcasting if required (e.g. if only SIN(DOUBLE) : DOUBLE is defined, the binder adds a cast from INT to DOUBLE before calling the function).

Maybe the way to think about it is to imagine the DDL statement. There will likely be a number of arguments, any one of which could be optional and some of which would trigger defaults (e.g., specifying the state as some kind of struct?) This sounds like filling in the elements of some kind of record, so maybe the current approach is the right one in the long term. Maybe the complexity is just the current "static function returning function pointer" approach, and we could just simplify it to "struct initialised with function pointers and inheriting some defaults from a base class"?

Indeed this is also the plan. I think there is no way to get rid of all the function pointers. I have now partly implemented the filling out the struct thing now in the functions branch (1, 2), take a look and see how you think it looks.

2 looks great!
My concern with 1 is that it scatters the definition of the aggregate function between files. I was wondering if a set of subclasses might be better, but that can wait.
Incidentally, there are two other aggregate state functions that might be needed (merge and deallocate) and I'm also thinking about how to resurrect the AVG optimisation, but that would just require more arguments. I've also never been happy about that silly cast Boolean - do you think this argument binding could get rid of that somehow?

Yes it should. The reason it was there was before we had auto-casting performing COUNT(a) on a string column would result in a cast of string to BIGINT. In the new system we would have COUNT(STRING) : BIGINT defined, which means the input will not be cast anymore for COUNT aggregates to begin with.

Implemented in c29057f91cd041912e8362312ef7d691dd00f747


authenticatedRoute issue with nested routes

Consider the following scenario. "user" route displays public user info. "user/private" route displays info available only to members and therefore you must be logged in to view it.
Router.map(function() {
  this.route('user', function() {
    this.authenticatedRoute('private');
  });
});

The above unfortunately does not work. And authenticated-route-mixin will not be applied to the "user.private" route.
I've investigated the issue and it turns out that the below line of code located here https://github.com/Vestorly/torii/blob/master/addon/router-dsl-ext.js#L8 doesn't take into consideration the full route name.
currentMap.push(arguments[0]);
Because of that the factory can not be found https://github.com/Vestorly/torii/blob/master/addon/bootstrap/routing.js#L8 as the factoryName will be "route:private" as oppose to "route:user.private" and therefore the mixin will not be applied.
Ember's routing dsl has the unexposed method called getFullName that takes care of naming the nested routes https://github.com/emberjs/ember.js/blob/master/packages/ember-routing/lib/system/dsl.js#L101

I just ran into this - @denzo any chance you're working on a PR for this? Or should I take a swing at it?

Having the same issue. Any solutions?

@leifdejong we just ended up doing auth checks in beforeModel in the routes...
//in route.js for the secured route
...
beforeModel(){
    let session = this.get('session');
    if(!session.get('isAuthenticated')){
      this.transitionTo('signin');
    }
  }
...

@dbouwman  thanks for the tip. I did however find the problem I was facing. I was modifying the beforeModel hook without calling this._super(...arguments);. After adding this, my issues were fixed with nested routes.


[Feature] CLI to update general settings

Description
This PR introduces a CLI command to update the general settings which will hopefully help solve the 500 error on that page. Might help fix issues like in #943.
Use

Open the terminal of the container.
Run php artisan app:update-general-settings.
Follow the prompts.

Changelog
Added

cli to update general settings

good morning i have been following 500 server error and followed till this and now its saying GMT (london) is not a valid time zone

good morning i have been following 500 server error and followed till this and now its saying GMT (london) is not a valid time zone

GMT might not be valid in PHP's time zone list. Use UTC instead.

Just googled it and GMT is Europe/London and it's on the PHPS list
From: Alex Justesen @.>
Sent: Monday, December 18, 2023 11:06 AM
To: alexjustesen/speedtest-tracker @.>
Cc: adam1991tomlinson @.>; Comment @.>
Subject: Re: [alexjustesen/speedtest-tracker] [Feature] CLI to update general settings (PR #978)
good morning i have been following 500 server error and followed till this and now its saying GMT (london) is not a valid time zone
Screenshot.2023-12-18.093321.png (view on web)https://github.com/alexjustesen/speedtest-tracker/assets/154218156/c317ef12-71b8-4774-b1e0-33a3821e7828
GMT might not be valid in PHP's time zone list. Use UTC instead.

Reply to this email directly, view it on GitHubhttps://github.com/alexjustesen/speedtest-tracker/pull/978#issuecomment-1860167874, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BEYS5LDDFYC5PW7OWBW5IWLYKAPPVAVCNFSM6AAAAABAQTHDJWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRQGE3DOOBXGQ.
You are receiving this because you commented.Message ID<EMAIL_ADDRESS>

Just googled it and GMT is Europe/London and it's on the PHPS list

Interesting, use UTC for now and I'll check the validation tonight on that one.

So looks like GMT is no longer a valid time zone that's provided from the embedded method. Def suggest using Europe/London or UTC instead.
https://www.php.net/manual/en/timezones.others.php


feat(views): make mini window unfocusable by default

I already did locally :) Will push updates later

👍

done


3141 rc rebase

Pull Request Checklist:

[ ] Positive Test Case Written by Dev

[ ] Automated Testing

[ ] Sufficient User and Developer Documentation

[ ] Passing Jenkins Build

[ ] Peer Code review and approval

Positive Test Case

[Enter positive test case here]

@Maurier - Want to make sure everything is here?  I ended up cherry-picking your commits onto the RC... I could not figure out those merge conflicts for a rebase.  Never ran into something like it.

@sblack-usu That looks correct.


Problems getting SSID to work in Simulator

Hello,
I have problems getting SSID when I'm running the XCode iOS simulator. I am running it as a iphone 6S with iOS 9.2. No interfaces is found and 'error' is returned. Should it be able to work in the simulator or what can I have made wrong? I can get the IP without any problems.
thanks!

It actually will not work in the simulator.  The simulator doesn't support interfaces unfortunately.  It should work fine when running on the device though.

Ah, okay! Then I know why it didnt work in the simulator.

Is there any code to prevent the crash on simulators but that it still runs on a real device?

does it crash the simulator or is it just not working in the simulator?

It crashes the Android & iOS simulators. I've avoided it with a try and catch.


Istio Operator hangs when Deleting IstioOperator With Revision

Bug description
If I create a control plane containing Istio Revision, I cannot delete it. Istio Operator hangs.
[ ] Docs
[X] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Extensions and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
[ ] Upgrade
Expected behavior
k delete istiooperator -n istio-system should trigger Istio Operator to delete a control plane, when it's using revisions.
Steps to reproduce the bug
Note the use of revision: 1-10-2
kubectl apply -f - <<EOF
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
  namespace: istio-system
  name: gloo-istiocontrolplane
spec:
  profile: default
  tag: 1.10.2
  revision: 1-10-2
  components:
    pilot:
      k8s:
        resources:
          requests:
            memory: 10Mi
            cpu: 10m
    ingressGateways:
    - name: istio-ingressgateway
      enabled: true
      k8s:
        resources:
          requests:
            memory: 10Mi
            cpu: 10m
        service:
          ports:
            - name: http2
              port: 8080
              targetPort: 8080
            - name: https
              port: 20443
              targetPort: 8443
            - name: tcp-status-port
              port: 15021
              targetPort: 15021
            - name: tls
              port: 8443
              targetPort: 15443
            - name: tcp-istiod
              port: 15012
              targetPort: 15012
            - name: tcp-webhook
              port: 15017
              targetPort: 15017
EOF

Then check the control plane deployment. Looks good.
k get pod -n istio-system
NAME                                   READY   STATUS    RESTARTS   AGE
istiod-1-10-2-6f8bd67fbb-zn984         1/1     Running   0          13s
svclb-istio-ingressgateway-wbsp6       6/6     Running   0          6s
svclb-istio-ingressgateway-9b9bt       6/6     Running   0          6s
istio-ingressgateway-6f8c99d7f-5xlvl   1/1     Running   0          6s

Now, try to delete the control plane. Deletion is hanging.
k delete istiooperator -n istio-system gloo-istiocontrolplane
istiooperator.install.istio.io "gloo-istiocontrolplane" deleted <---- Hangs

Check the Istio Operator Logs. Deletion is hanging.
k logs -n istio-operator istio-operator-769c594554-z58h5
...
2021-06-25T22:58:25.020033Z	info	installer	Reconciling IstioOperator
2021-06-25T22:58:25.025068Z	info	installer	Deleting IstioOperator gloo-istiocontrolplane <---- Hangs

Control plane pods are still Running. They should be Terminating
k get pod -n istio-system
NAME                                   READY   STATUS    RESTARTS   AGE
istiod-1-10-2-6f8bd67fbb-zn984         1/1     Running   0          13s
svclb-istio-ingressgateway-wbsp6       6/6     Running   0          6s
svclb-istio-ingressgateway-9b9bt       6/6     Running   0          6s
istio-ingressgateway-6f8c99d7f-5xlvl   1/1     Running   0          6s

Version (include the output of istioctl version --remote and kubectl version --short and helm version --short if you used Helm)
1.10.2
How was Istio installed?
Istio Operator 1.10.2
Environment where the bug was observed (cloud vendor, OS, etc)
Civo (k3s)
Additionally, please consider running istioctl bug-report and attach the generated cluster-state tarball to this issue.
Refer cluster state archive for more details.

I also see logs in the operator stating that it expected a string for revision when I explicitly provide it a string, but it then parses it as an int and breaks itself.

Manually removing the revision via kubectl edit causes it to be cleaned up correctly.

Just ran into this same issue upgrading. The old iop objects are stuck with a deleted timestamp and the operator just continuously loops saying the Reconciling IstioOperator and Deleting IstioOperator istio-1.x.x

I also faced with similar issue. I described it here https://github.com/istio/istio/issues/35016


Fix of broken links

I renamed the github-branch instead to reflect the existing links.
Thanks for reporting!


[Improvement] OpenSearch Field types

Can we add a page to outline the various field data types that opensearch supports?

And date formats too....like https://www.elastic.co/guide/en/elasticsearch/reference/master/mapping-date-format.html

Specifically I am looking for equivalent info from this page: https://www.elastic.co/guide/en/elasticsearch/reference/7.10/mapping-types.html but for OS.

I was looking for this information today and was unable to find it. Internet searches for "opensearch data types" or "opensearch field types" had results from the SQL and PPL pages. But, this is not the same information.

Elasticsearch reference regarding the removal of mapping types: https://www.elastic.co/guide/en/elasticsearch/reference/8.2/removal-of-types.html

Hi Fanit, can you please try to tackle this issue? In addition to this request (that has been in our backlog for a while), there was a recent request in a Slack thread for documenting the data types: https://amzn-aws.slack.com/archives/C01L92M6Y92/p1654250558281419 chat. Let me know if you can't access it. Nate A. will support, as needed. Thanks!


Fossid API Scan response model has changed

After Fossid version 22.2,  hasFileExtension in Scan.kt type has changed from Int -> Boolean.
It cause Json deserialize error.
API Test at fossid api pusher version 2022.2.2#20722
request
{
    "group": "scans",
    "action": "get_information",
    "data": {
        "username": "test_user",
        "key": "test_user_key",
        "scan_code": "test_scan_code"
    }
}

response
{
  "operation": "scans_get_information",
  "status": "1",
  "data": {
    "id": 1950,
    "scanlimit": 10,
    "last_run": null,
    "code": "test_scan_code",
    "name": "test_scan_code",
    "description": "",
    "comment": null,
    "is_archived": null,
    "target_path": null,
    "is_blind_audit": null,
    "is_from_report": false,
    "files_not_scanned": null,
    "pending_items": null,
    "git_repo_url": "git@github.com:test_scan_code.git",
    "git_branch": "master",
    "imported_metadata": null,
    "has_file_extension": true,   ### changed from Int -> Boolean
    "jar_extraction": "never",
    "any_archives_expanded": null,
    "uploaded_files": null,
    "sensitivity": 10,
    "kb_version": null,
    "fid_version": null,
    "ort_version": null,
    "created": "2022-11-16 16:18:34",
    "updated": "2022-11-16 16:18:34",
    "username": "test_user",
    "project_code": "test_project",
    "user_id": 1,
    "project_id": 1
  }
}

@nnobelis can you have a look at this?

Will try to prepare a fix but it will last a couple of days.


Replacing wget with iwr in the powershell sample

I was following the powershell instructions and noticed that wget is not a default command in powershell so this PR replaces it with the iwr command.

@cmendible : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Closing since now master branch code reflects a change replacing the wget call with Invoke-WebRequest


[WFLY-16847] Upgrade Eclipse Yasson to 3.0.1

Fixes https://issues.redhat.com/browse/WFLY-16847

Please find all links to release notes, git tags, github-diff in the Jira.


Support project scaffolding or initializer

Such as https://start.spring.io/, so a user can easily start building the application using Armeria.

I shared a related post last year. 😄
See https://github.com/line/armeria/issues/2426#issuecomment-591251023

Ah, I missed that. Thanks. 😄

Also, Micronaut has one.
https://micronaut.io/launch/

I think this would be a great way to create a nice UX for devs especially for first-timers. I would like to work on this when I have spare time.


Winacme Guide

As the title says: is it possible to have a step by step guide for make it works using winacme? thank you!

Creating a winacme tutorial is currently not on our roadmap. However, we welcome any help! So, if you use winacme and figured out how to use it with deSEC, please feel free to contribute.
If you have any questions in this context, I'll be happy to answer them.


Persist and upload stroke / revolution count

#303 added display support for it. We could possibly also persist it and upload it with FIT / TCX. Since it's a cumulative metric it might be enough to just persist the final value for the activity.

TCX doesn't have field for this. We persist and upload (FIT) the cumulative end count, at the Activity level, we don't keep track of the progress at the Record level.
https://github.com/TrackMyIndoorWorkout/TrackMyIndoorWorkout/commit/ac32ac6f348be5d159ad696b61efe0871a27b508


MAID-2622 refactor/naming: aligns naming conventions between libraries

Coverage remained the same at 92.113% when pulling 0a41977147059e8848f23e2ecd367d37fc1e2eb0 on hunterlester:2622 into c2bf3fb7f1b08027a9d263cfbf822804e28b115f on maidsafe:master.

Also, as discussed, let's please review the API's that are being exposed by the other bindings so we can have all bindings to expose similar and coherent API's


Delete messages

Is there any method to remove a message older than 48 hours using a bot?

There can be no method that can do that since the Telegram Bot API itself does not support it, as referenced here: https://core.telegram.org/bots/api#deletemessage

It's not good news :-( I know method #deletetemessage  from BOT API :-)  but I need to delete messages in private channels older then 48 h. But it is impossible - return json from Telegram:
{"ok":false,"error_code":400,"description":"Bad Request: message can't be deleted"}


histogram data incorrect - entries show in multiple buckets

with a single request:
curl -v http://localhost:42422/metrics
*   Trying <IP_ADDRESS>...
* Connected to localhost (<IP_ADDRESS>) port 42422 (#0)
> GET /metrics HTTP/1.1
> Host: localhost:42422
> User-Agent: curl/7.47.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Length: 2498
< Content-Type: text/plain; version=0.0.4
< Date: Sat, 27 May 2017 00:01:26 GMT
<
# HELP request_count request count by source, target, service, and code
# TYPE request_count counter
request_count{method="/echo",response_code="200",service="unknown",source="unknown",target="foo.svc.cluster.local",version="unknown"} 1
# HELP request_duration request duration by source, target, and service
# TYPE request_duration histogram
request_duration_bucket{method="/echo",response_code="200",service="unknown",source="unknown",target="foo.svc.cluster.local",version="unknown
",le="0.005"} 1
request_duration_bucket{method="/echo",response_code="200",service="unknown",source="unknown",target="foo.svc.cluster.local",version="unknown
",le="0.01"} 1
# very basic local run
request_duration_bucket{method="/echo",response_code="200",service="unknown",source="unknown",target="foo.svc.cluster.local",version="unknown
",le="0.025"} 1
request_duration_bucket{method="/echo",response_code="200",service="unknown",source="unknown",target="foo.svc.cluster.local",version="unknown
",le="0.05"} 1
request_duration_bucket{method="/echo",response_code="200",service="unknown",source="unknown",target="foo.svc.cluster.local",version="unknown
",le="0.1"} 1
request_duration_bucket{method="/echo",response_code="200",service="unknown",source="unknown",target="foo.svc.cluster.local",version="unknown
",le="0.25"} 1
request_duration_bucket{method="/echo",response_code="200",service="unknown",source="unknown",target="foo.svc.cluster.local",version="unknown
",le="0.5"} 1
request_duration_bucket{method="/echo",response_code="200",service="unknown",source="unknown",target="foo.svc.cluster.local",version="unknown
",le="1"} 1
request_duration_bucket{method="/echo",response_code="200",service="unknown",source="unknown",target="foo.svc.cluster.local",version="unknown
",le="2.5"} 1
request_duration_bucket{method="/echo",response_code="200",service="unknown",source="unknown",target="foo.svc.cluster.local",version="unknown
",le="5"} 1
request_duration_bucket{method="/echo",response_code="200",service="unknown",source="unknown",target="foo.svc.cluster.local",version="unknown
",le="10"} 1
request_duration_bucket{method="/echo",response_code="200",service="unknown",source="unknown",target="foo.svc.cluster.local",version="unknown
",le="+Inf"} 1
request_duration_sum{method="/echo",response_code="200",service="unknown",source="unknown",target="foo.svc.cluster.local",version="unknown"}
0.003
request_duration_count{method="/echo",response_code="200",service="unknown",source="unknown",target="foo.svc.cluster.local",version="unknown"
} 1

with wrk:
request_duration_bucket{method="/echo",response_code="200",service="unknown",source="unknown",target="foo.svc.cluster.local",version="unknown
",le="0.05"} 15375
request_duration_bucket{method="/echo",response_code="200",service="unknown",source="unknown",target="foo.svc.cluster.local",version="unknown
",le="0.1"} 15375
request_duration_bucket{method="/echo",response_code="200",service="unknown",source="unknown",target="foo.svc.cluster.local",version="unknown
",le="0.25"} 15375
request_duration_bucket{method="/echo",response_code="200",service="unknown",source="unknown",target="foo.svc.cluster.local",version="unknown
",le="0.5"} 15375
request_duration_bucket{method="/echo",response_code="200",service="unknown",source="unknown",target="foo.svc.cluster.local",version="unknown
",le="1"} 15375
request_duration_bucket{method="/echo",response_code="200",service="unknown",source="unknown",target="foo.svc.cluster.local",version="unknown
",le="2.5"} 15375
request_duration_bucket{method="/echo",response_code="200",service="unknown",source="unknown",target="foo.svc.cluster.local",version="unknown
",le="5"} 15375
request_duration_bucket{method="/echo",response_code="200",service="unknown",source="unknown",target="foo.svc.cluster.local",version="unknown
",le="10"} 15375
request_duration_bucket{method="/echo",response_code="200",service="unknown",source="unknown",target="foo.svc.cluster.local",version="unknown
",le="+Inf"} 15375

The buckets are meant to represent all the data below them. These buckets are generated by Prometheus and should be accurate.

ic, it's a weird feature to have overlapping buckets in an histogram no ? is it configurable ?

what you are looking at is how Prometheus serializes represents histograms. that is not configurable, unless you are willing to rewrite Prometheus ;). they have chosen to use cumulative histograms.
More info: https://prometheus.io/docs/concepts/metric_types/#histogram and https://en.wikipedia.org/wiki/Histogram#Cumulative_histogram.

Looks like they are cumulative, not overlapping.
Using "iPhone 9";

On May 26, 2017, at 5:23 PM, Laurent Demailly<EMAIL_ADDRESS>wrote:
ic, it's a weird feature to have overlapping buckets in an histogram no ? is it configurable ?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.


Budgets - "minimum exceeded" errors despite not configuring any minimums

Versions
Angular CLI: 1.7.0-beta.3
Node: 8.6.0
OS: darwin x64
Angular: 6.0.0-beta.1
... animations, common, compiler, compiler-cli, core, forms
... http, language-service, platform-browser
... platform-browser-dynamic, router

@angular/cdk: 5.0.1
@angular/cli: 1.7.0-beta.3
@angular/material: 5.0.1
@angular-devkit/build-optimizer: 0.2.0
@angular-devkit/core: 0.2.0
@angular-devkit/schematics: 0.2.0
@ngtools/json-schema: 1.1.0
@ngtools/webpack: 1.10.0-beta.3
@schematics/angular: 0.2.0
@schematics/package-update: 0.2.0
typescript-collections: 1.2.5
typescript: 2.6.2
webpack: 3.10.0

Repro steps

Use the following setup for budgets:  "budgets": [{
    "type": "any",
    "error": "1mb"
  }]

ng build --prod
Observe errors

Observed behavior
Error output:
ERROR in budgets, minimum exceeded for assets/i18n/de/strings.json.
ERROR in budgets, minimum exceeded for assets/i18n/es/strings.json.
ERROR in budgets, minimum exceeded for assets/i18n/en/strings.json.
ERROR in budgets, minimum exceeded for assets/i18n/eu/strings.json.
ERROR in budgets, minimum exceeded for assets/i18n/fr/strings.json.
ERROR in budgets, minimum exceeded for assets/i18n/hr/strings.json.
ERROR in budgets, minimum exceeded for assets/i18n/fi/strings.json.
ERROR in budgets, minimum exceeded for assets/i18n/hu/strings.json.
ERROR in budgets, minimum exceeded for assets/i18n/it/strings.json.
ERROR in budgets, minimum exceeded for assets/i18n/iw/strings.json.
ERROR in budgets, minimum exceeded for assets/i18n/nl/strings.json.
ERROR in budgets, minimum exceeded for assets/i18n/ko/strings.json.
ERROR in budgets, minimum exceeded for assets/i18n/ja/strings.json.
ERROR in budgets, minimum exceeded for assets/i18n/no/strings.json.
ERROR in budgets, minimum exceeded for assets/i18n/pl/strings.json.
ERROR in budgets, minimum exceeded for assets/i18n/pt_BR/strings.json.
ERROR in budgets, minimum exceeded for assets/i18n/ro/strings.json.
ERROR in budgets, minimum exceeded for assets/i18n/ru/strings.json.
ERROR in budgets, minimum exceeded for assets/i18n/tr/strings.json.
ERROR in budgets, minimum exceeded for assets/i18n/zh_CN/strings.json.
ERROR in budgets, minimum exceeded for assets/i18n/sv/strings.json.
ERROR in budgets, minimum exceeded for assets/i18n/zh_TW/strings.json.
ERROR in budgets, minimum exceeded for index.html.
ERROR in budgets, minimum exceeded for 3rdpartylicenses.txt.

Desired behavior
Not sure why I'm seeing errors about minimums when I haven't configured any.
Mention any other details that might be useful (optional)
Would be great to see a few example configs here, the descriptions are pretty vague (and confusing as to how they work together) without any concrete examples.

@Matmo10 If you use error (as far as I understand, I'm not the author of this feature), it sets a threshold for maximum AND minimum. If you only care about the maximum, you should set maximumError instead. And all sizes work with a baseline: if you don't set one, it uses a baseline of 0.
So for example, setting a budget "type": "any", "baseline": "500kb", "error": "100kb" will ensure that all assets are between 400kb and 600kb, or error. A little bit of documentation would indeed help, you are welcome to offer a PR if you can.
In the upcoming beta.4, I added better error messages (see https://github.com/angular/angular-cli/commit/7f9931d28b5296421889cff4cdb69616840cdf10), so you should have something slightly more explicit like : ERROR in budgets, maximum exceeded for total scripts. Budget 500 kB was exceeded by 83.9 kB..

Thanks for the clarification - even just adding what you said to the docs makes it much clearer IMO. I guess it's obvious now in retrospect, but baseline seems to be a required field then for budgets. That wasn't immediately clear to me. I came into this expecting to set an absolute max or min limit, not doing everything relatively.
Also thanks for the improved error messages!


SIR model with exponential waiting times in Julia

How do you like something very simple, I think you don't have that yet:
https://gist.github.com/mschauer/be1196dedbf8afe7fb8a0b411672b4d6

hi @mschauer, it looks like there are a few CTMC constructions of the SIR model in this repo, just listed as "jump processes":

https://github.com/epirecipes/sir-julia/blob/master/markdown/jump_process/jump_process.md
https://github.com/epirecipes/sir-julia/blob/master/markdown/jump_process_gillespie/jump_process_gillespie.md

Think of it in the same fashion as https://discourse.julialang.org/t/improving-metropolis-hastings-code/67412/7, you could use Turing there too instead.

I think this is already covered for now, unless @mschauer, you think there's something we're missing with the jump process implementations

unless you think there's something we're missing with the jump process implementations

I would think so:
prob_jump = JumpProblem(prob,Direct(),infection_jump,recovery_jump);
sol_jump = solve(prob_jump,SSAStepper());

has a very different didactical value compared to
function sir!(us, u, T, p)
   β, c, γ = p.β, p.c, p.γ
   while true
       t, S, I, R = u
      # @show t, u
       N = +(S, I, R)
       τ = randexp()/(S * β*c*I/N), randexp()/(I*γ) # renew both times, because state has changed
       t + min(τ[1], τ[2]) > T && return us
       if τ[1] < τ[2]
           t += τ[1]
           S -= 1
           I += 1
       else
           t += τ[2]
           I -= 1
           R += 1
       end
       u = (t, S, I, R)
       push!(us, u)
   end
end

OK, I can add the plain Julia code, as I agree that there are didactic differences. This no-library approach was in the old repo, and was in an old blog post of mine too

@sdwfrost I think if we are going to add plain Julia code, do you think its worth it to do plain Julia for different flavors of how to sample the CTMC? @mschauer has posted a nice Direct method, it might be worth it to show a First Reaction Method and Anderson's Next Reaction Method paper here.

I think have something close to Algorithm 2 (the next reaction method) from the Anderson paper, except that I simply resample both times instead what is done in step (8) to reuse elements of the vector of random next event times.

@slwu89 you probably have something along these lines hanging around too?

Yeah, I'd love to write up an example with a direct method, first reaction method, and Anderson's NRM. I think I also coded Thanh's rejection sampler in R before.

Hi @slwu89 - just seeing if you're still interested in this!


Ignore error from issue #244

Hey buddy, the solution for this should be code an extension and not ignoring the error.

Yes, I wish I could understand Larastan.

Let's do it for now.


Created signin.css file

Created signin.css file from scratch

@arghadipmanna101 any issues with this pr sir , Let me know ?

@arghadipmanna101 if this current PR have any issue I am requesting you to please check I have created a new PR #337 , Requesting you to verify it , I will close this PR then . Thankyou

Since the PR #485 is merged I am closing this PR . Thanks


[MM-26622] - Change permissions for guest users to create and join channels

Summary
change guest permissions to channel permissions so they can add and create new channels
Ticket Link
https://mattermost.atlassian.net/browse/MM-26622

/update-branch


Bingo bork still after fix

Bingo still tracks incorrect remaining pokemon as well as funky UI
To Reproduce
Join a server
Run /bingo
See error
screenshot of Orion

This is happening on all servers it appears

Speaking with the dev about this, hopefully a fix will be available soon.

This should be fixed! Thank you for the report!


move logo image

Issue

close #213

Overview (Required)
move toolbar_droidkaigi_logo file
from uicomponent-compose:feed & uicomponent-compose:other to uicomponent-compose:core
Links

Screenshot

Before
After

Thanks! LGTM 👍

Thanks as every year!


Error on running application

Hello @avalanche123, super neat tool! I am really looking forward to trying this as UI's in the cassandra space are few and far between and yours seems elegant.
Few problems, I tried the application and received this in output:
# cassandra-web --hosts '10.86.xx.xx' --port '9042' --username 'cassAdmin' --password 'cassPassword' --bind <IP_ADDRESS>:9393 --server-cert 'appServer.crt' --client-cert 'appServerTruststore.ca' --pri
vate-key 'appServer.key' --passphrase 'appServer' --log-level debug
D, [2018-05-09T11:59:52.140745 #14457] DEBUG -- : Populating policies and listeners with initial endpoints
D, [2018-05-09T11:59:52.141036 #14457] DEBUG -- : Host 10.86.xx.xx is found and up
I, [2018-05-09T11:59:52.141147 #14457]  INFO -- : Establishing control connection
D, [2018-05-09T11:59:52.141427 #14457] DEBUG -- : Connecting to 10.86.xx.xx
D, [2018-05-09T11:59:52.142953 #14457] DEBUG -- : Host 10.86.xx.xx  refused connection (Cassandra::Errors::IOError: Connection reset by peer)
W, [2018-05-09T11:59:52.142998 #14457]  WARN -- : Host 10.86.xx.xx refused all connections
D, [2018-05-09T11:59:52.143038 #14457] DEBUG -- : Host 10.86.xx.xx  is down
D, [2018-05-09T11:59:52.143114 #14457] DEBUG -- : Starting to continuously refresh status of <IP_ADDRESS> in 0.5 seconds
D, [2018-05-09T11:59:52.143233 #14457] DEBUG -- : Connection to 10.86.xx.xx  failed (Cassandra::Errors::IOError: Connection reset by peer)
Cassandra::Errors::NoHostsAvailable: All attempted hosts failed: 10.86.xx.xx  (Cassandra::Errors::IOError: Connection reset by peer)

Do I need to pass certs with certain file extensions? I am not sure what could be causing this.
EDIT- Seeing this in C* logs, seems this app does not use ssl:
INFO  [epollEventLoopGroup-2-4] 2018-05-09 16:18:26,156 Message.java:623 - Unexpected exception during request; channel = [id: 0x509a9c9e, L:/10.86.xxx.xx:9042 ! R:/10.86.xxx.xx:58348]io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record:<PHONE_NUMBER>00000000

Yep that was def it, when I set in C* if I set optional true it worked
client_encryption_options:
    enabled: true
    # If enabled and optional is set to true encrypted and unencrypted connections are handled.
    optional: true

But we don't want non-encrypted traffic, any chance you can add ssl client option on tx to the C* calls? Would that be sinatra config related or?

I think I found the spot in bin/cassandra-web:
    options[:load_balancing_policy] = ::Cassandra::LoadBalancing::Policies::WhiteList.new(hosts, ::Cassandra::LoadBalancing::Policies::RoundRobin.new)
    options[:compression]           = :lz4
    options[:page_size]             = nil
    options[:ssl]                   = true

    cluster = ::Cassandra.cluster(options)


Implement Encrypted Streams

TODO

[x] Multipart continuation function
[x] Usage example at the top
[ ] push / pull with additional data

Closing in favour of something based on a pre-existing streaming library


Fix build latest wifi

Fix the build breakage with latest Wi-Fi driver PR: https://jenkins-ncs.nordicsemi.no/job/latest/job/sdk-nrf/job/PR-8495/7/testReport/

@krish2718 could you please link the PR/PRs which needs to be merged before we can merge this one? I assume:

sdk-hostap:
https://github.com/nrfconnect/sdk-hostap/pull/14
sdk-nrf:
https://github.com/nrfconnect/sdk-nrf/pull/8495

@krish2718 could you please link the PR/PRs which needs to be merged before we can merge this one? I assume:

sdk-hostap:
Merge dev branch sdk-hostap#14
sdk-nrf:
Wi-Fi: Merge dev branch sdk-nrf#8495

Yes, https://github.com/nrfconnect/sdk-nrf/pull/8495 (this already includes the sdk-hostap manifest update).

also the Darwin related CI failures doesn't seem to be Wi-Fi related, can we ignore those?

also the Darwin related CI failures doesn't seem to be Wi-Fi related, can we ignore those?

I hope so, I faced the same with my latest PR, also not related to Darwin. Let's see what's other's thoughts on that.

Also, this is a circular dependency, for the sdk-nrf CI to be passed, this needs to be merged, and for matter to build once this is merged it needs latest driver, so, once that is reviewed before merging

merge this
update manifest PR->Commit
merge sdk-nrf.

Also, this is a circular dependency, for the sdk-nrf CI to be passed, this needs to be merged
@krish2718 why? I think what needs to happen is:

CI is green on you sdk-nrf PR with this open PR referenced in west.yml, so we can be sure this PR was validated vs sdk-nrf change.
Then we can merge this PR and you need to switch pull/<pr_number/head in your sdk-nrf PR to the commit SHA we got after merging this PR.
CI is validated once again on sdk-nrf PR.

I don't see circular dependency here. If we will merge this PR to sdk-connectedhomeip master it will not work vs sdk-nrf master, because there will not be your Wi-Fi driver change there yet. However we don't recommend using sdk-nrf master with sdk-connectedhomeip master, but sdk-nrf master with specific revision of sdk-connectedhomeip, so I would say everything should be fine.

Also, this is a circular dependency, for the sdk-nrf CI to be passed, this needs to be merged

@krish2718 why? I think what needs to happen is:

CI is green on you sdk-nrf PR with this open PR referenced in west.yml, so we can be sure this PR was validated vs sdk-nrf change.
Then we can merge this PR and you need to switch pull/<pr_number/head in your sdk-nrf PR to the commit SHA we got after merging this PR.
CI is validated once again on sdk-nrf PR.

These are the same steps I was referring to.

I don't see circular dependency here. If we will merge this PR to sdk-connectedhomeip master it will not work vs sdk-nrf master, because there will not be your Wi-Fi driver change there yet. However we don't recommend using sdk-nrf master with sdk-connectedhomeip master, but sdk-nrf master with specific revision of sdk-connectedhomeip, so I would say everything should be fine.

Yes, its only a problem if we use master of sdk-connectedhomeip.

These are the same steps I was referring to.

Ok then, I misunderstood something and thought you would like to merge this PR before passing Ci on sdk-nrf.

These are the same steps I was referring to.

Ok then, I misunderstood something and thought you would like to merge this PR before passing Ci on sdk-nrf.

I was suggesting that in case we want to get both master working together, I guess we can leave that now.

CI passing now, we can merge this https://github.com/nrfconnect/sdk-nrf/pull/8495


window.opener is not a BrowserWindow, so does not match evt.source

Have one window opened by electron that opens a child window using window.open
the parent does a postMessage to the child
window.opener does not equal evt.source since evt.source is a BrowserWindow, but window.opener is not

Electron 0.36.0, windows x64

Electron 0.35.2, Windows x64

I've just seen https://github.com/atom/electron/pull/3769.. thats a bit of luck. will re-test in 0.36.0

@kevinsawicki thanks for #3769 it just solved half of this bug (edited to remove duplicate), thought I'd mention you in case you have any input


Fix wayf unconnected idp button hover

The unconnected IdP rows should show the white button with text
"request access" to make sure it's clear what happens when you click it.
https://www.pivotaltracker.com/story/show/149554887

@arnoutt, as a WAYF expert, what do you thing of this change?
Current

New

@MKodde I think you're missing the 'Previously chosen' header in the new situation?

@MKodde is the 'Press enter to select' now always visible for IdPs with access? That is a bit confusing, since obviously when you press enter, only 1 IdP (the 1 that is currently selected) will be chosen :)
Same question for the 'Request access' button: is it always visible or only when hovered?

@tvdijen thanks for pointing that out! That issue was fixed in another PR and this work will probably be rebased on that once it is merged.
@arnoutt thanks for your feedback. The 'press enter to select' text was previously only visible on hover. But should of course only be set on the first/selected option. And yes, the request access buttons will now always be visible as advertised in the screenshot. When hovering these items they will revert to their current hover state.
This is our interpretation of the request by Thijs in the pivotal story mentioned in the description of this PR.

The unconnected IdP rows should show the white button with text "request access" to make sure it's clear what happens when you click it.

@MKodde Ok I understand, but would like to see that the 'Press enter to select' is only visible when hovered. The 'Request access' button should indeed always be visible. Thanks!

@MKodde After some internal discussion, would it be possible to implement as follows:

Currently, it is possible to highlight 2 IdPs: 1 with the keyboard (arrows), another with the mouse. Could this be made into 1 possible selection? So mouse overrides keyboard and vice versa.
Remove the 'Press enter to select' button entirely (keep the 'Request access' button). (Keep the 'Request access' button.)

Thanks!

The requested adjustements by @arnoutt are implemented. @MKodde could you verify if all requirements are met.

The IdP selection list works more intuitive now!
One thing that might not yet be as intended is when mouse hovering over the 'Identity providers without access' entries, they are not lit up. Which seems logical to me, but @arnoutt might think otherwise? At the time of taking the screenshot, my cursor was on the location where I drew the red arrow. No additional hover state is applied to the NWO entry.  What do you think Arnout?

@MKodde the 'IdPs without access' list should behave the same as those with access, which means the selection should also light up when hovered. Also, it should be possible to select them with the arrow keys and enter.
Rest looks nice, thanks!

And one little thing in IE11: When returning to the WAYF after cancelling authentication for an IdP, all IdP's are selected in the overview.

@MKodde I've fixed the IE11 behaviour as requested
@arnoutt I've created a ticket in the backlog with your request becasue it's kinda out-of-scope so your request could be implemented later on:
https://www.pivotaltracker.com/story/show/159595638


mkgmap: 4608 -> 4806

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for mkgmap is: "Create maps for Garmin GPS devices from OpenStreetMap (OSM) data"
meta.homepage for mkgmap is: "http://www.mkgmap.org.uk"
meta.changelog for mkgmap is: ""
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

/nix/store/mihlkm1v4s4mcapz76mmx1xhigwxciqb-mkgmap-4806/bin/mkgmap passed the binary check.

1 of 1 passed binary check by having a zero exit code.

1 of 1 passed binary check by having the new version present in output.

found 4806 with grep in /nix/store/mihlkm1v4s4mcapz76mmx1xhigwxciqb-mkgmap-4806

found 4806 in filename of file in /nix/store/mihlkm1v4s4mcapz76mmx1xhigwxciqb-mkgmap-4806

directory tree listing: https://gist.github.com/12678ccc46577aa897eb460aa5613134

du listing: https://gist.github.com/3123d445d12f013795f3d7a69d6b9ae8

Rebuild report (if merged into master) (click to expand)

4 total rebuild path(s)

1 package rebuild(s)

1 x86_64-linux rebuild(s)
1 i686-linux rebuild(s)
1 x86_64-darwin rebuild(s)
1 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
mkgmap

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/mihlkm1v4s4mcapz76mmx1xhigwxciqb-mkgmap-4806 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A mkgmap https://github.com/r-ryantm/nixpkgs/archive/95b152745d8abdc4de19237b061560ea950252cd.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/mihlkm1v4s4mcapz76mmx1xhigwxciqb-mkgmap-4806
ls -la /nix/store/mihlkm1v4s4mcapz76mmx1xhigwxciqb-mkgmap-4806/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

mkgmap

Maintainer pings
cc @sikmir for testing.

Result of nixpkgs-review pr 137485 at 95b15274 run on x86_64-linux 1

1 package built successfully:

mkgmap

3 suggestions:

warning: missing-phase-hooks
buildPhase should probably contain runHook preBuild and runHook postBuild.
Near pkgs/applications/misc/mkgmap/default.nix:54:3:
   |
54 |   buildPhase = "ant";
   |   ^

warning: missing-phase-hooks
checkPhase should probably contain runHook preCheck and runHook postCheck.
Near pkgs/applications/misc/mkgmap/default.nix:58:3:
   |
58 |   checkPhase = "ant test";
   |   ^

warning: missing-phase-hooks
installPhase should probably contain runHook preInstall and runHook postInstall.
Near pkgs/applications/misc/mkgmap/default.nix:60:3:
   |
60 |   installPhase = ''
   |   ^


9016 DEV add new field isFlagged to accounts schema

https://github.com/Shippable/pm/issues/9016

this is done, closing


fix: create abspath dir in case that contents is empty

What type of PR is this?

Uncomment only one /kind <> line, hit enter to put that in a new line, and remove leading whitespace from that line:
/kind bug
/kind documentation
/kind enhancement
/kind good-first-issue
/kind feature
/kind question
/kind design
/sig ai
/sig iot
/sig network
/sig storage

/kind bug
What this PR does / why we need it:
fix bug of issue 2133
Which issue(s) this PR fixes:

Fixes #2133
Special notes for your reviewer:

Does this PR introduce a user-facing change?

NONE

other Note

@vie-serendipity would you like to post a pull request to fix Trivy Scan error. you can check the error Details and find the error reason.

@rambohe-ch ok

/rerun

It looks like the download of the vulnerability database failed due to some network issues. But after I tried again, everything is working fine now.

/LGTM


HOSTEDCP-992: refactor ignition-server reconcilation

https://issues.redhat.com/browse/HOSTEDCP-992
No functional change, just refactor to match CPO coding style of a main component level Reconcile with per-resource reconcile functions within the package.

/override "Red Hat Trusted App Pipeline / hypershift-operator-main-on-pull-request"


Fix python serialization

fixes for #206
Python: if object values are JSON objects with more than 4 keys exception happens

@bsphere thank you for the fix, I cherry picked it from the #207.


Add -i option to index-discovery

This adds the ability to quickly re-index a single item, collection, or community.

@cwilper, This is a fantastic idea.  Do you have a Jira ticket for this feature?

Sure thing, here ya go: https://jira.duraspace.org/browse/DS-3271

+1 via code review. Just needs a quick verification test, but the code looks right to me. I also agree this would be a great improvement

I tested this - the single-item (by handle) reindex showed the expected log lines.
What surprised me, however, is that after I chaged item title directly in the metadatavalue table and refreshed the page, the updated title showed immediately (i.e. without index-discovery -i) in both recent items and in search results. This is good, obviously, but I can't explain why DSpace reaches into the DB when in 5.x it would reach into Solr search.

Tested locally with the following approach.

Add a unique dc.subject to an item
Look at collection facet list, confirm the unique subject term is present
Update the subject term with sql
Run index-discovery -i
Refresh collection page, look for updated facet value (SUCCESS)
Update the subject term with sql
Run index-discovery -i
Refresh collection page, look for updated facet value (NOT UPDATED)

The command appears to successfully update an item.  I imagine that it would also update collection/community metadata.
As a user, when I pass in a collection handle or a community handle, I would expect all descendant objects to be updated.
@cwilper , @helix84 , @tdonohue , what do you think?

Just a minor note to @terrywbrady's question above. Based on my reading of the code, this new -i option will only ever reindex a single object. So, that means that running it against a Collection or Community will just reindex that Collection or Community object (and will NOT recurse and reindex any child objects).
If that is the intended behavior, we might want to clarify that in the usage notes (in the code).

If this is intended only for items, clarify in the code as Tim suggests and in the usage message for that option.  I would also recommend adding a jira enhancement ticket to support communities and collections in the future.
I have a selfish reason for wanting the collection/community support in place, but I understand that will add to the complexity of the feature.  Locally, we have implemented a process to force the re-indexing of a collection or a community.  It is not as elegant as this approach.

I agree with Terry, it would be a worthwhile addition. Meanwhile, it would make sense if the command complained when given a community/collection rather than fail silently.

Another small clarification here... this code actually does work for all types of objects, but it is currently limited by the way the SolrServiceImpl class is implemented.  In that class, there's really only two ways to index/reindex content:

Reindex everything : see updateIndex() methods
OR, reindex a single object (without recursing to child objects): see indexContent() methods

This PR uses the second approach to allow you to trigger a reindex of a single object via the new -i flag.  That means this new -i flag actually does work for Communities or Collections, but it only reindexes the metadata about that Community or Collection itself and doesn't recurse to any child objects.
I agree that this might not be the expected behavior.  The only way I can see to fix this would be to either change the behavior of SolrServiceImpl (a large scale change), or possibly add some basic recursion code into IndexClient which can ensure the expected recursive indexing occurs when -i is used on a Community or Collection.

I agree that recursion is a reasonable expectation for users to have. I'm taking a quick stab at doing it within IndexClient (rather than SolrServiceImpl, which as @tdonohue noted, is a bigger change). If my local test is successful, I'll update the PR.

The above worked for me locally, but should be reviewed again by other folks. Note that, in order to avoid memory exhaustion and slowness as observed with DS-3086, the new Context enableBatchMode and clearCache methods are used.
I actually think it's important to switch to using these for all batch-oriented methods in DSpace, as @tomdesair noted in the comments of DS-3086, but I limited the changes done here to the -i option.

Make it a curation task?  Then you get this-object-and-children for free from the task framework.  If the framework is not yet using batch mode, it probably should.

Doing it as a curation task occurred to me as well, but that's also a larger code change. Currently, the curation task framework isn't using batch mode, and it definitely should (along with discovery re-indexing, in general). See DS-3286, which I just submitted.

I tested this locally and it generally does what I expect.  I found a couple of communities/collections that repeatedly fail with the following exception.  I cannot identify what makes these communities/collections unique.
[dspace@dspace-aux-1 dspace6x-aux-prs]$ /opt/dspace/bin/dspace index-discovery -i 10822/707889
Exception: failed to lazily initialize a collection of role: org.dspace.content.DSpaceObject.handles, could not initialize proxy - no Session
org.hibernate.LazyInitializationException: failed to lazily initialize a collection of role: org.dspace.content.DSpaceObject.handles, could not initialize proxy - no Session
at org.hibernate.collection.internal.AbstractPersistentCollection.throwLazyInitializationException(AbstractPersistentCollection.java:569)
at org.hibernate.collection.internal.AbstractPersistentCollection.withTemporarySessionIfNeeded(AbstractPersistentCollection.java:188)
at org.hibernate.collection.internal.AbstractPersistentCollection.readSize(AbstractPersistentCollection.java:139)
at org.hibernate.collection.internal.PersistentBag.isEmpty(PersistentBag.java:249)
at org.apache.commons.collections.CollectionUtils.isEmpty(CollectionUtils.java:979)
at org.apache.commons.collections.CollectionUtils.isNotEmpty(CollectionUtils.java:992)
at org.dspace.content.DSpaceObject.getHandle(DSpaceObject.java:133)
at org.dspace.discovery.SolrServiceImpl.indexContent(SolrServiceImpl.java:198)
at org.dspace.discovery.SolrServiceImpl.indexContent(SolrServiceImpl.java:2305)
at org.dspace.discovery.IndexClient.indexAll(IndexClient.java:180)
at org.dspace.discovery.IndexClient.main(IndexClient.java:148)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:229)
at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:81)

Terry, thanks for testing. That particular type of error can occur when the Hibernate session cache is cleared (which in this code, happens every so often when processing a list of items), but someone tries to continue accessing a field whose value was previously cached. In this case, I think it's revealing that the .handles member of one of the objects is trying to be accessed, but that object's data has been de-cached from the session.
You can see in the code above where try to I avoid this "stale entity" issue by reloading the community (see the line that populates the reloadedCommunity value). Without doing that, I got a similar error when the .collections member of "community" was accessed on the subsequent line, in cases where cache clearing had occurred (processing a collection with more than a few items).
I think the problem now may be the community.getHandle() call...that member is also cacheable and may be stale as well. A simple solution may be to put that into a communityHandle as early as possible in this code, and just use that so we're not trying to call any methods on a potentially-stale entity.

The above commit should protect from use of a potentially-stale community.handles member, but it may not address the specific problem you're seeing. The stack trace indicates it may actually be the collection.handles member that is problematic. I just don't see how the collection entity could have possibly been de-cached at this point in the code..

@cwilper , I am seeing the same repeatable errors on the same collections/communities.  This makes me suspect that there is some relationship in the database that is triggering this.

@cwilper , the following code seems to fix the issue.
        for (Community subcommunity : community.getSubcommunities()) {
            subcommunity = ContentServiceFactory.getInstance().getCommunityService().find(context, subcommunity.getID());
            count += indexAll(indexingService, itemService, context, subcommunity);
        }
        //final Community reloadedCommunity = (Community) HandleServiceFactory.getInstance().getHandleService().resolveToObject(context, communityHandle);
        for (Collection collection : community.getCollections()) {
            collection = ContentServiceFactory.getInstance().getCollectionService().find(context, collection.getID());

@tdonohue , I think this could be evidence of a larger hibernate issue.

+1, I logged the hibernate issue in https://jira.duraspace.org/browse/DS-3293 since it may be beyond the scope of this enhancement request.  Assuming that issue is accepted as a separate issue, I support merging this code.

I can reproduce the LazyInitializationException with a very simply hierarchy.
Top-Level Community
    - Collection #1
        - Single Item
    - Collection #2

In this setup, if you run this new index-discovery -i command passing in the handle of the Top-Level Community, then a LazyInitializationException will be thrown when it attempts to index Collection #2 (but all other items are indexed successfully).
Not sure yet if this PR's code is to blame, but it's now easily reproducible.

@cwilper , I commented out the following block and the issue goes away.  I imagine that a different method is needed for clearing the cache.
https://github.com/cwilper/DSpace/blob/3c72564c6a9b894014165cdf2e2815fc083646d2/dspace-api/src/main/java/org/dspace/discovery/IndexClient.java#L206-L209
        // Every so often, clear the context cache to preserve
        // memory, and commit the changes to the search index.
        if (context.getCacheSize() > 50) {
            context.clearCache();
            indexingService.commit();
        }

To build off of @terrywbrady's last comment, the problem is specifically with context.clearCache(), which in turn calls DBConnection.clearCache() which flushes Hibernate's session.  This behavior doesn't seem to play well with the recursive method implemented here, and causes the LazyInitializationException.
Removing context.clearCache() also removes the need to reload the Community on this line: https://github.com/cwilper/DSpace/blob/3c72564c6a9b894014165cdf2e2815fc083646d2/dspace-api/src/main/java/org/dspace/discovery/IndexClient.java#L178
So, this PR either needs to avoid using context.clearCache(), or that clearCache() method needs to be fixed/improved to support usage in recursive methods.

The problem is definitely wider in scope and I'm not as concerned about the "-i" capability at this point as I am about the practicality of doing big batch operations (recursively) without running into memory/performance problems or LazyInitializationExceptions. It seems that careful execution of such operations (holding on to the handles as Strings) while recursing down into the hierarchy could avoid this, while still allowing cache-clearing to occur.
As for this PR, my inclination at this point is to concentrate on the broader issue affecting pre-existing functionality first (see DS-3286 ...and maybe revisit this after a working pattern is established. Further comments from me on the broader issue will be in DS-3286's comments.

@cwilper : sounds reasonable. There is definitely a need for a new working pattern for dealing with the Hibernate cache in recursive batch operations, so it makes sense to find a generalizable solution. Thus far, the only usages of  clearCache() have been non-recursive (and the method does work well for such non-recursive batch operations).

Closing in favor of #1495, which implements this feature as well.


Resolved #726

Not for this issue but I can add an improvement for more than two names, like "Johann Chrysostom Wolfgang Amadeus Mozart" as "JM". Do I need a new issue?

Create a new issue but you can reuse this PR.  I won't merge until you give me the go ahead.  but lets track the new issue.

Also need to add title and alt attributes to show full text when mouse over

@melloware
All completed. You can merge it if no problem.


Publish integration test coverage report

Fixes #833

Description

See the issue description.
How Has This Been Tested?

Screenshots, that help to understand the changes(if applicable):
Checklist:

[ ] A test for the new functionality was added.
[ ] All tests run without failure.
[ ] The new code complies with the TiGL style guide.
[ ] New classes have been added to the Python interface.
[ ] API changes were documented properly in tigl.h.

Codecov Report

Merging #834 (1a73f16) into master (3260a54) will increase coverage by 0.19%.
The diff coverage is n/a.

:exclamation: Current head 1a73f16 differs from pull request most recent head 98bec5c. Consider uploading reports for the commit 98bec5c to get more accurate results

@@            Coverage Diff             @@
##           master     #834      +/-   ##
==========================================
+ Coverage   62.19%   62.38%   +0.19%
==========================================
  Files         291      291
  Lines       23494    24477     +983
==========================================
+ Hits        14611    15269     +658
- Misses       8883     9208     +325

Impacted Files
Coverage Δ

src/common/Debugging.h
40.00% <0.00%> (-10.00%)
:arrow_down:

src/rotor/CCPACSRotorBlades.cpp
65.38% <0.00%> (-5.45%)
:arrow_down:

src/rotor/CCPACSRotorBladeAttachments.cpp
64.70% <0.00%> (-4.05%)
:arrow_down:

src/wing/CCPACSWingSectionElement.cpp
17.39% <0.00%> (-3.67%)
:arrow_down:

src/fuselage/CCPACSFuselageSectionElement.cpp
17.39% <0.00%> (-3.67%)
:arrow_down:

src/rotor/CCPACSRotors.cpp
73.91% <0.00%> (-3.36%)
:arrow_down:

src/CCPACSEtaXsiPoint.cpp
16.00% <0.00%> (-3.05%)
:arrow_down:

src/CCPACSPositioning.cpp
65.88% <0.00%> (-3.04%)
:arrow_down:

src/cpacs_other/CCPACSExternalObject.cpp
52.30% <0.00%> (-2.87%)
:arrow_down:

src/CCPACSCurvePoint.cpp
11.76% <0.00%> (-2.53%)
:arrow_down:

... and 210 more

codecov is still a bit misterious to me. According to the comment, we still have a coverage of around 62%. But when we seperated integration tests and unit tests in april, the drop in code coverage did not occur with the related pull request, but only with the pull request after that. So I suppose we can only expect the true coverage to be shown, once the github workflow file is updated on the master branch.

The merge-commit of seperating integration and unit tests: https://codecov.io/gh/DLR-SC/tigl/commit/ea8e57e497174de08a37c52f24dda645abc0f82b/build
The merge-commit after, showing the drop in coverage by 7%: https://codecov.io/gh/DLR-SC/tigl/commit/fb0d5bffa88329882f49ff17c6d79ed1907ff714/

codecov is still a bit misterious to me. According to the comment, we still have a coverage of around 62%. But when we seperated integration tests and unit tests in april, the drop in code coverage did not occur with the related pull request, but only with the pull request after that. So I suppose we can only expect the true coverage to be shown, once the github workflow file is updated on the master branch.

Indeed, strange behavior. But I no reason why adding the integration tests should not increase the coverage. So yes, lets merge this.


How to Load Application Specific Data as well..

I have a gltf file in which a mesh primitive looks like this:
"primitives": [
{
"attributes":{
"BATCHID":"accBatch_0",
"NORMAL":"accNormal_0",
"POSITION":"accPosition_0"
},
"indices":"accIndex_0",
"material":"Material_0",
"mode":4
},
]
where accBatch_0 for batch id refers to an accessor for accessing all the application specific ids of the vertices
however the library fails to load the file

First, a snippet above looks like glTF 1.0. Unity loader supports only 2.0. Moreover, BATCHID is not a valid attribute name: all app-specific attribute names must start with _, i.e. it should be _BATCHID.

how about this???
"primitives": [
{
"attributes": {
"POSITION": 25,
"_BATCHID": 0
},
"indices": 10,
"material": 3,
"mode": 1
}
]
This is from a gltf 2.0 that has a underscore as well. i cant get Unity to load the file... It also uses webquantized attributes. not sure whats causing the failure.. I can provide the complete gltf as well for testing if needed

It's hard to say what's causing the failure without seeing the complete asset (also, I'm not a developer of this particular plugin, so the reason might be somewhat Unity-specific). Anyway, could you please check the asset in the validator to ensure that it's valid glTF 2.0?
Btw, UnityGLTF certainly doesn't support quantized attributes extension.

It isnt a valid GLTF but im expected to get it working anyhow... oh then its probably the quantized attributes. Thanks alot!!!

We don't support loading invalid glTF files. Metadata information should go in the extensions or extras field of your file.


AVI Video not playing at all

Black and stays on spinning circle. Shows length at bottom.
Suspect parent problem (exoplayer) again…
General
Complete name                            : F:\Content\Auxi\BBC wild weather cold - Quinzhee.avi
Format                                   : AVI
Format/Info                              : Audio Video Interleave
File size                                : 12.1 MiB
Duration                                 : 4mn 14s
Overall bit rate                         : 400 Kbps
Writing application                      : Lavf51.12.2
Video
ID                                       : 0
Format                                   : MPEG-4 Visual
Format profile                           : Simple@L1
Format settings, BVOP                    : No
Format settings, QPel                    : No
Format settings, GMC                     : No warppoints
Format settings, Matrix                  : Default (H.263)
Codec ID                                 : FMP4
Duration                                 : 4mn 14s
Bit rate                                 : 261 Kbps
Width                                    : 480 pixels
Height                                   : 272 pixels
Display aspect ratio                     : 2.222
Frame rate                               : 14.985 fps
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Compression mode                         : Lossy
Bits/(Pixel*Frame)                       : 0.134
Stream size                              : 7.93 MiB (65%)
Writing library                          : Lavc51.41.0
Audio
ID                                       : 1
Format                                   : MPEG Audio
Format version                           : Version 1
Format profile                           : Layer 3
Mode                                     : Joint stereo
Mode extension                           : MS Stereo
Codec ID                                 : 55
Codec ID/Hint                            : MP3
Duration                                 : 4mn 14s
Bit rate mode                            : Constant
Bit rate                                 : 128 Kbps
Channel count                            : 2 channels
Sampling rate                            : 44.1 KHz
Compression mode                         : Lossy
Stream size                              : 3.88 MiB (32%)
Alignment                                : Aligned on interleaves
Interleave, duration                     : 26 ms (0.39 video frame)

I'm getting similar issue on my android 9 tv, except player directly shows error and does not play.
i've narrowed down the cause to be exoplayer not supporting mp3.
10-16 02:00:48.175  2446 14831 E SoftMP3 : mp3 decoder returned error 2
10-16 02:00:48.183 14769 14829 E ACodec  : [OMX.google.mp3.decoder] ERROR(0x80001001)
10-16 02:00:48.191 14769 14799 E ExoPlayerImplInternal:   Caused by: com.google.android.exoplayer2.mediacodec.MediaCodecDecoderException: Decoder failed: OMX.google.mp3.decoder

I don't think the author can do much about it - the app relies on exoplayer to do it's job

Format                                   : AVI
Format/Info                              : Audio Video Interleave
File size                                : 353 MiB
Duration                                 : 50 min 3 s
Overall bit rate                         : 984 kb/s
Writing application                      : Nandub v1.0rc2
Writing library                          : Nandub build 1853/release

Video
ID                                       : 0
Format                                   : MPEG-4 Visual
Codec ID                                 : DIV3
Codec ID/Hint                            : DivX 3 Low
Duration                                 : 50 min 3 s
Bit rate                                 : 847 kb/s
Width                                    : 512 pixels
Height                                   : 384 pixels
Display aspect ratio                     : 4:3
Frame rate                               : 25.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Compression mode                         : Lossy
Bits/(Pixel*Frame)                       : 0.172
Stream size                              : 303 MiB (86%)

Audio
ID                                       : 1
Format                                   : MPEG Audio
Format version                           : Version 1
Format profile                           : Layer 3
Format settings                          : Joint stereo / MS Stereo
Codec ID                                 : 55
Codec ID/Hint                            : MP3
Duration                                 : 50 min 3 s
Bit rate mode                            : Constant
Bit rate                                 : 128 kb/s
Channel(s)                               : 2 channels
Sampling rate                            : 48.0 kHz
Compression mode                         : Lossy
Stream size                              : 45.8 MiB (13%)
Alignment                                : Split accross interleaves
Interleave, duration                     : 40  ms (1.00 video frame)
Interleave, preload duration             : 445  ms

@mailinglists35
Format                                   : MPEG-4 Visual
Codec ID                                 : DIV3
Codec ID/Hint                            : DivX 3 Low

Devices usually do not have decoder for this old format so such videos won't play.

divx was quite popular at that time.
I'm surprised it wouldn't be supported.
However for my case if you loon at logcat snippet, it is exisplayer being let down by android mp3 decoder - it doesn't seem to be able to process the audio. no error about the video in my case.

@mailinglists35  Try what I wrote in my second post in #350 if you must see it work