Mount routes

Is there in Slim a feature that let you mount a given controller to an existing application?
Silex supports it.
An example of usage would be:
// define controllers for a forum
$forum = $app['controllers_factory'];
$forum->get('/', function () {
    return 'Forum home page';
});

$app->mount('/forum', $forum);

I'm not sure what you mean, but if you have an invokable class registered with the DIC, you can do this:
$app->get('/forum', 'controllers_factory');

In which case, Slim will do this when dispatching:
$action = $container->get('controllers_factory');
$response = $action($request, $response, $args);

I've had a quick read. No Slim doesn't support that in exactly the same way. You have to be explicit with groups:
$app->group('/blog', function () {
    $this->get('/', 'BlogController:homeAction');
    $this->get('/view', 'BlogController:detailAction');
});

Thanks for the answer. Anyway I think the Silex way would be a nice addition to Slim.

I agree, that SIlex feature is very nice.

It seems like the silex way of doing that is more like calling a new Slim app into the controller.
Something like that I think
$forum = new Slim\App();
$forum->get('/', function ($request, $response, $args) {
    $response->write("Forum home page");
    return $response;
});

$app = new Slim\App();
$app->get('/forum', $forum)

$app->run();

This code will not work but a simple middleware can do the trick.

The base concept which i quite like (But i doubt it would actually use) of creating isolated route categories like $forum and $blog is rather easy to implement. This is a simple implementation that attempts to modify as little as possible.
Step 1) Extend the App class, and add the following code:
public function mount($group, Router $routes) {
	$app = $this;
	return $this->group($group, function() use($routes, $app) {
		foreach ($routes->getRoutes() as $route) {
			$groupedRoute = $app->map($route->getMethods(), $route->getPattern(), $route->getCallable());

			$name = $route->getName();
			if (!empty($name)) {
				$groupedRoute->setName($name);
			}

			$middlewares = $route->getMiddleware();
			if (count($middlewares) > 0) {
				foreach ($middlewares as $middleware) {
					$groupedRoute->add($middleware);
				}
			}

			$arguments = $route->getArguments();
			if (count($arguments) > 0) {
				$groupedRoute->setArguments($arguments);
			}

			$buffering = $route->getOutputBuffering();
			if (!empty($buffering)) {
				$route->setOutputBuffering($buffering);
			}
		}
	});
}

This adds the mount function to your app object, allowing you to call $app->mount("/group", $router); later on. It simply loops over every route in the provided Router, within the prefix provided via the $group parameter. It also tries to bring over all metadata, but i may have missed something and there may be an easier way to do this.
Step 2) Use your new App class rather than Slim. This is as simple as changing the definition of your $app (or whatever you call it!) variable to point to your new App class!
Step 3) Extending the Router, and adding this function to it
private $validMethods = [
	"GET",
	"POST",
	"PUT",
	"PATCH",
	"DELETE",
	"OPTIONS",
	"ANY",
];

public function __call($key, $arguments) {

	$method = strtoupper($key);
	if (!in_array($method, $this->validMethods)) {
		return; // ?? Not sure what to do here?
	}

	if ($method == "ANY") {
		$method = array_splice($this->validMethods, 1, 5);
	}

	return call_user_func_array([$this, "map"], array_merge([$method], $arguments));
}

Rather than manually mapping all supported HTTP Methods into the Router object like Slim did in the main App object, i opted to use a simple magic method that checks to see if it's a valid method (And converts Any into all of them), than calls the Router->Map(); function with an array of Methods, and the Pattern / Handler.
Step 4) Profit. You are technically complete, create a new instance of your Router (Yes, it has to be new, or you'll have your old routes left in there!), set any routes you want on it, and than load it into the application via $app->mount();
@DavidePastore I'm not sure if this will get you want you want, but it should be close!

Hi @KuroThing, thanks for the solution. It would be nice if you could create a PR with this code :+1:  @akrabat what do you think about it?

Ignoring the fact that the provided code doesn't meet PSR-2 Standards that the main Slim project uses and the slight performance overhead of using a magic method rather than actually defining the methods properly, I'm not entirely sure it's worth adding to the main repo.
I say this, because the main repo alread has the Group method, which is literally all this is doing (Running a group callback behind the scenes), to change the way you define groups. I also don't know if there are any side effects from the Router object having Get, Put, Post, etc methods that may be unintended.
If the core maintainers have no problems with the duplicating existing functionality, and the Router having the HTTP Method functions, i'll fix the two issues i mentioned in the first paragraph and send a PR.

I think this has been answered, so closing.


Kmesh eBPF log user space dumping

What would you like to be added:
Kmesh logs governance process information via BPF_LOG. Logs are expected to be dumped to the user space:

Log information should be dumped to user space via ringbuf.
In user space, the log information should be obtained through epoll and formatted for printing.

reference: https://github.com/anakryiko/bpf-ringbuf-examples/blob/main/src/ringbuf-output.bpf.c
Why is this needed:
The current API has the following issues:

Logs are recorded in memory through trace_pipe and can be viewed using bpftool prog tracelog, limited space.
Format parameters such as "%pI4h" are supported latter than v5.13 kernel version,  not yet supported in 5.10 (such as openEuler 22.03 SP3),
Only three parameters are supported.

/assign

Logs are recorded in memory through trace_pipe and can be viewed using bpftool prog tracelog, limited space.

Not an expert on this, does it occupy kernel memory?

Maybe we need to firstly define some simple fixed format，because ebpf prog seems not support var args.
For example:

no args BPF_LOG(ERR, SOCKOPS, "set sockops cb failed!\n")
1 int args or 2 int args BPF_LOG(ERR, KMESH, "set bypass value failed!, err is %d\n", err)  and BPF_LOG(DEBUG, KMESH, "bpf find frontend addr=[%pI4h:%u]\n", &ip, bpf_ntohs(ctx->user_port));, or more int args.
Also can add a flag to represent which kind of log is occured. Just as this implement https://github.com/cilium/cilium/blob/d18d41e6995679624980671c46b25e5a80d40136/bpf/lib/dbg.h#L8

I am not sure whether the format should mix with string args %s, the situation may become complex. Dose anything need to supplement?

Logs are recorded in memory through trace_pipe and can be viewed using bpftool prog tracelog, limited space.

Not an expert on this, does it occupy kernel memory?

This is a debugfs file system, and it will be cleared after the system reboot.

Maybe we need to firstly define some simple fixed format，because ebpf prog seems not support var args. For example:

no args BPF_LOG(ERR, SOCKOPS, "set sockops cb failed!\n")
1 int args or 2 int args BPF_LOG(ERR, KMESH, "set bypass value failed!, err is %d\n", err)  and BPF_LOG(DEBUG, KMESH, "bpf find frontend addr=[%pI4h:%u]\n", &ip, bpf_ntohs(ctx->user_port));, or more int args.
Also can add a flag to represent which kind of log is occured. Just as this implement https://github.com/cilium/cilium/blob/d18d41e6995679624980671c46b25e5a80d40136/bpf/lib/dbg.h#L8

I am not sure whether the format should mix with string args %s, the situation may become complex. Dose anything need to supplement?
The struct possible like this:
struct log_event {
    __u8 type;
    __u8 level;
    char fmt[MAX_MSG_LEN];
    __u32 arg1;
    __u32 arg2;
    __u32 arg3;
};

As describe, the situation may become complexed because of the formatting parameters. IMO, we can use BPF_SNPRINTF to stored format string, and then record into ringbuf map for user-space dumping.
But I didn't test the feasibility, for details about BPF_SNPRINTF usage example, see: https://github.com/torvalds/linux/commit/c2e39c6bdc7eb48459ec1d34d4f27eb82299f4b7

Resolve 5.10 problem.

Define a flag to indicate whether vargs include ipaddr info.
By kernel version, determine use BPF_LOG_K or BPF_LOG_U, these 2 marco be included in  BPF_LOG


Field typed as BaseModel for polymorphism serializes as an empty dict

Initial Checks

[X] I confirm that I'm using Pydantic V2

Description
I would like to use polymorphism in my typing definitions for a model as shown in the example. In practice, I have a dict of arbitrary models that all inherit from BaseModel, but the example is simplified as much as possible to demonstrate the issue.
The expected result is the following:
{"internal":{"field1":1}}
The actual result is the following:
{"internal":{}}
Example Code
from pydantic import BaseModel

class InternalModel(BaseModel):
    field1: int = 1

class MyModel(BaseModel):
    internal: BaseModel

my_model = MyModel(internal=InternalModel())
print(my_model.model_dump_json())

Python, Pydantic & OS Version
pydantic version: 2.8.2
        pydantic-core version: 2.20.1
          pydantic-core build: profile=release pgo=true
                 install path: C:\Users\User\miniconda3\envs\strikoid\Lib\site-packages\pydantic
               python version: 3.9.19 (main, May  6 2024, 20:12:36) [MSC v.1916 64 bit (AMD64)]
                     platform: Windows-10-10.0.22631-SP0
             related packages: typing_extensions-4.12.0
                       commit: unknown

It looks like you need to specify the type of internal as InternalModel
class MyModel(BaseModel):
    internal: InternalModel

I don't want to do that because that doesn't allow for polymorphism (multiple different child classes of BaseModel being used for internal. I think maybe the intention isn't for the types inside a Model to be used for polymorphism, and instead the type you put is the strict type required (not allowing subclasses). But then I would expect that the validation would fail on the constructor call to MyModel, so something is wrong either way.

I'm not sure if BaseModel is limiting this. For root objects, you can try using RootModel
See if you can solve your problem.
class MyModel(BaseModel):
    internal: RootModel

Using RootModel worked. I'm not sure why this behavior is like this though, can you explain?

It seems that you can use SerializeAsAny to dump all fields of a model.
class MyModel(BaseModel):
    internal: SerializeAsAny[BaseModel]

By default model_dump_json only dump the fields of annoated model (i.e. BaseModel, which doesn't have any field), not actual model (InternalModel), I guess.

Using RootModel worked. I'm not sure why this behavior is like this though, can you explain?

I can't explain exactly, RootModel inherits BaseModel and has arbitrary types of fields, so it works. However, this may not be standard usage and does not meet the module authors' expectations.

It seems that you can use SerializeAsAny to dump all fields of a model.
class MyModel(BaseModel):
    internal: SerializeAsAny[BaseModel]

By default model_dump_json only dump the fields of annotated model (i.e. BaseModel, which doesn't have any field), not actual model (InternalModel), I guess.

SerializeAsAny is the correct way to handle your use case (explanation can be found in the linked docs). Thanks @kc0506 for the answer


👷 deploy github actions

Fix #47

We need to add 2 secrets into project to make it works.
It seems I don't have the rights to proceed.

CI Tests on pull_request fails due to the miss of  the PROTOSTAR_VERSION secret variable. I think the PR system mess it up, I assume it will works well after the merge (as it is on the branch).


Add docs for .NET 5.0 BinaryFormatter event source

Note to repo owners: As part of the BinaryFormatter obsoletion strategy, we introduced in .NET 5.0 an EventSource which allows apps to receive notification when objects are being serialized or deserialized using BinaryFormatter. The blurb below is a doc that describes this functionality and demonstrates how to hook it. Not sure where it would go - perhaps within the BinaryFormatter remarks section or as a sibling page to the API docs?

BinaryFormatter event source
Starting with .NET 5.0, BinaryFormatter includes a built-in EventSource that gives developers visibility into when an object serialization or deserialization is occurring. Apps can use EventListener-derived types to listen for these notifications and log them.
This functionality is not a substitute for a SerializationBinder or an ISerializationSurrogate and cannot be used to modify the data being serialized or deserialized. Rather, this eventing system is intended to provide insight into the types being serialized or deserialized. It can also be used to detect unintended calls into the BinaryFormatter infrastructure, such as those originating from third-party library code.
Description of events
BinaryFormatter's event source has the well-known name System.Runtime.Serialization.Formatters.Binary.BinaryFormatterEventSource. As of this writing, listeners may subscribe to 6 events.
SerializationStart event (id = 10)
Raised when BinaryFormatter.Serialize has been called and has started the serialization process. This is paired with the SerializationEnd event. The SerializationStart event can be called recursively if an object calls BinaryFormatter.Serialize within its own serialization routine.
This event does not contain a payload.
SerializationEnd event (id = 11)
Raised when BinaryFormatter.Serialize has completed its work. Each occurrence of SerializationEnd denotes the completion of the last unpaired SerializationStart event.
This event does not contain a payload.
SerializingObject event (id = 12)
Raised when BinaryFormatter.Serialize is in the process of serializing a non-primitive type. The BinaryFormatter infrastructure special-cases certain types (such as string and int) and will not raise this event when these types are encountered. This event will be raised for user-defined types and other types that BinaryFormatter does not natively understand.
This event may be raised zero or more times between SerializationStart and SerializationEnd events.
This event contains a payload with one argument.

typeName (string): The assembly-qualified name (see Type.AssemblyQualifiedName) of the type being serialized.

DeserializationStart event (id = 20)
Raised when BinaryFormatter.Deserialize has been called and has started the deserialization process. This is paired with the DeserializationEnd event. The DeserializationStart event can be called recursively if an object calls BinaryFormatter.Deserialize within its own deserialization routine.
This event does not contain a payload.
DeserializationEnd event (id = 21)
Raised when BinaryFormatter.Deserialize has completed its work. Each occurrence of DeserializationEnd denotes the completion of the last unpaired DeserializationStart event.
This event does not contain a payload.
DeserializingObject event (id = 22)
Raised when BinaryFormatter.Deserialize is in the process of deserializing a non-primitive type. The BinaryFormatter infrastructure special-cases certain types (such as string and int) and will not raise this event when these types are encountered. This event will be raised for user-defined types and other types that BinaryFormatter does not natively understand.
This event may be raised zero or more times between DeserializationStart and DeserializationEnd events.
This event contains a payload with one argument.

typeName (string): The assembly-qualified name (see Type.AssemblyQualifiedName) of the type being deserialized.

[Advanced] Subscribing to a subset of notifications
Listeners who wish to subscribe to only a subset of notifications can choose which keywords to enable.

Serialization = (EventKeywords)1: Raises the SerializationStart, SerializationEnd, and SerializingObject events.
Deserialization = (EventKeywords)2: Raises the DeserializationStart, DeserializationEnd, and DeserializingObject events.

If no keyword filters are provided during EventListener registration, all events will be raised.
See the EventKeywords enum for more information.
Sample code
The following code demonstrates:

creating an EventListener-derived type that writes to System.Console,
subscribing that listener to BinaryFormatter-produced notifications,
serializing and deserializing a simple object graph using BinaryFormatter, and
analyzing the events which have been raised.

using System;
using System.Diagnostics.Tracing;
using System.IO;
using System.Runtime.Serialization.Formatters.Binary;

namespace BinaryFormatterEventSample
{
    class Program
    {
        static EventListener _globalListener = null;

        static void Main(string[] args)
        {
            // First, set up the event listener.
            // Note: We assign it to a static field so that it doesn't get GCed.
            // We also provide a callback that subscribes this listener to all
            // events produced by the well-known BinaryFormatter source.

            _globalListener = new ConsoleEventListener();
            _globalListener.EventSourceCreated += (sender, args) =>
            {
                if (args.EventSource?.Name == "System.Runtime.Serialization.Formatters.Binary.BinaryFormatterEventSource")
                {
                    ((EventListener)sender).EnableEvents(args.EventSource, EventLevel.LogAlways);
                }
            };

            // Next, create the Person object and serialize it.

            Person originalPerson = new Person()
            {
                FirstName = "Logan",
                LastName = "Edwards",
                FavoriteBook = new Book()
                {
                    Title = "A Tale of Two Cities",
                    Author = "Charles Dickens",
                    Price = 10.25m
                }
            };

            byte[] serializedPerson = SerializePerson(originalPerson);

            // Finally, deserialize the Person object.

            Person rehydratedPerson = DeserializePerson(serializedPerson);

            Console.WriteLine($"Rehydrated person {rehydratedPerson.FirstName} {rehydratedPerson.LastName}");
            Console.WriteLine($"Favorite book: {rehydratedPerson.FavoriteBook.Title} by {rehydratedPerson.FavoriteBook.Author}, list price {rehydratedPerson.FavoriteBook.Price}");
        }

        private static byte[] SerializePerson(Person p)
        {
            MemoryStream memStream = new MemoryStream();
            BinaryFormatter formatter = new BinaryFormatter();
#pragma warning disable SYSLIB0011 // BinaryFormatter.Serialize is obsolete
            formatter.Serialize(memStream, p);
#pragma warning restore SYSLIB0011

            return memStream.ToArray();
        }

        private static Person DeserializePerson(byte[] serializedData)
        {
            MemoryStream memStream = new MemoryStream(serializedData);
            BinaryFormatter formatter = new BinaryFormatter();

#pragma warning disable SYSLIB0011 // Danger: BinaryFormatter.Deserialize is insecure for untrusted input
            return (Person)formatter.Deserialize(memStream);
#pragma warning restore SYSLIB0011
        }
    }

    [Serializable]
    public class Person
    {
        public string FirstName;
        public string LastName;
        public Book FavoriteBook;
    }

    [Serializable]
    public class Book
    {
        public string Title;
        public string Author;
        public decimal Price;
    }

    // A sample EventListener that writes data to System.Console.
    public class ConsoleEventListener : EventListener
    {
        protected override void OnEventWritten(EventWrittenEventArgs eventData)
        {
            base.OnEventWritten(eventData);

            Console.WriteLine($"Event {eventData.EventName} (id={eventData.EventId}) received.");
            if (eventData.PayloadNames != null)
            {
                for (int i = 0; i < eventData.PayloadNames.Count; i++)
                {
                    Console.WriteLine($"{eventData.PayloadNames[i]} = {eventData.Payload[i]}");
                }
            }
        }
    }
}

The output of this program will be:
Event SerializationStart (id=10) received.
Event SerializingObject (id=12) received.
typeName = BinaryFormatterEventSample.Person, BinaryFormatterEventSample, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null
Event SerializingObject (id=12) received.
typeName = BinaryFormatterEventSample.Book, BinaryFormatterEventSample, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null
Event SerializationStop (id=11) received.
Event DeserializationStart (id=20) received.
Event DeserializingObject (id=22) received.
typeName = BinaryFormatterEventSample.Person, BinaryFormatterEventSample, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null
Event DeserializingObject (id=22) received.
typeName = BinaryFormatterEventSample.Book, BinaryFormatterEventSample, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null
Event DeserializationStop (id=21) received.
Rehydrated person Logan Edwards
Favorite book: A Tale of Two Cities by Charles Dickens, list price 10.25

In this sample, the console-based EventListener logs that once serialization has started, instances of Person and Book are serialized, and then serialization completes. Similarly, once deserialization has started, instances of Person and Book are deserialized, and then deserialization completes.
The app then prints the values contained with the deserialized Person to demonstrate that the object did in fact serialize and deserialize properly.
Further reading
For more information on using EventListener to receive EventSource-based notifications, see the EventListener class.

I think this could either go in the Remarks section of the API docs, or in the conceptual docs with a link from the API docs. @tdykstra Is this your area?

If we had similar docs for other event sources I would put it with them but I don't see any place like that. My best guess is to put it next to the BinaryFormatter security guide in the TOC.


Add pipeline to publish KEDA Cosmos DB scaler

Added file for 'Publish KEDA Cosmos DB Scaler' pipeline.
Updated filename for 'Publish KEDA' pipeline to have consistent naming.

'Publish KEDA Cosmos DB Scaler' pipeline is scheduled to run at 12:00 UTC on Sundays. It will list all available tags for keda-external-scaler-azure-cosmos-db package on GHCR and replicate them to azurefunctions ACR. It also prunes the untagged manifests of keda-external-scaler-azure-cosmos-db repository on ACR to keep it clean. A manifest loses the tag when a different image with the same tag gets pushed to the registry.
PR information

[x] The title of the PR is clear and informative.
[x] There are a small number of commits, each of which has an informative message. This means that previously merged commits do not appear in the history of the PR. For information on cleaning up the commits in your pull request, see this page.
[x] If applicable, the PR references the bug/issue that it fixes in the description.

Hi @ahmelsayed - Would you be able to the pull request?


Implementation of Nextcloud OAuth authentication

As https://github.com/SocialiteProviders/Providers/pull/632 is already merged, this implementation use it for authentication with Nextcloud.

Looks like repo needs composer update command to make it work?

@artur-shaik Is this done via oAuth2?

@artur-shaik Is this done via oAuth2?

Exactly.

Exactly.

Why don't you name it oAuth2, then it would be more general and users would know that they can include other services as well.
I for example would use it for MailCow ;)

I wish to have generic OAuth provider in this app, too. But authentication implemented with third party library for php called Socialite. So I decide to implement Nextcloud support in that library to easily integrate it in Bookstack.
I think, you can try to substitude your app credentials and see what happen, with this patch.

I will definitely try that out, then all services I have would be accessible via one instance.

I will definitely try that out, then all services I have would be accessible via one instance.

I just realized that this will not work. There are field access values that are unique for Nextcloud.
So, my advise for you is to implement your provider as Socialized plugin and use it on your BookStack instance as I do on my own. Actually you can just copypaste base classes and change it regarding you provider.

Thanks for offering this PR @artur-shaik, But I've come to the decision I don't want to grow out the supported third-party social provides much further, at least without a high level of benefit/request from the user-base. It's not a large amount of work to initially implement but it gets troublesome attempting to support people here when issues eventually arise.
Within the latest release, v21.04, I've added a system where you can perform some level of back-end customization. Details on this system can be seen here: https://github.com/BookStackApp/BookStack/blob/release/dev/docs/logical-theme-system.md
It's still in the testing stages, and I'm looking for feedback on it, but it should be possible to use additional socialite providers via this system, An example of this is shown at the bottom using Reddit which I tested during development of that system.


Props refactor

[x] Move toolbarPosition into toolbarProps
[x] Move miniaturePosition into miniatureProps
[x] Move miniatureBackground into miniatureProps
[x] Move miniatureWidth into miniatureProps
[x] Move miniatureHeight into miniatureProps
[ ] Assign a better name to  disableDoubleClickZoomWithToolAuto prop

Done. Will be released with 3.0


Need way to get microdeposit values

It seems like an API is missing to query for what the micro-deposit values are.
In the short term, I need something because once I initiate a micro-deposit on a depository I need a way to find out what those values are so I can confirm them!  Just for testing.
Longer term though I think we may want to allow customer service folks see those values to determine if a user is entering them correctly or if perhaps there is another problem.
Right now it does not appear there is an API to get these values.

I need something because once I initiate a micro-deposit on a depository I need a way to find out what those values are so I can confirm them!

The source for this data is intended to be when the micro-deposits are posted in the account. This wouldn't be quick however...
If you can initiate micro-deposits in paygate and then call another endpoint to get the values then we haven't confirmed the caller owns the Depository. We've only verified they are auth'd against paygate.
We store the micro-deposits in a database table and can expose them for customer service. There just aren't endpoints for reading that yet.

The source for this data is intended to be when the micro-deposits are posted in the account.

This is harder when the accounts calls/service are disabled. Paygate won't post the transactions anywhere and then you can't verify. I'm not quite sure how to solve this right now..

Wait - so no ACH files are created to create the deposits if the accounts service is not connected?

Ok - I get why you might not want to have an API at the user level that gets the micro-deposits.  If an API gets created to get those values is it safe to assume it would be on the admin port?

ACH files are still created and send to the originating FI. I was talking about how one could query the accounts service and get the micro-deposit amounts.

If an API gets created to get those values is it safe to assume it would be on the admin port?

Yea probably.

This will be in v0.6.0, but it's on the master branch right now.


Footnote make Page and UUID unique together

TODO:

Add validation to check that UUIDs stored in the page reference related footnotes that exist

Current thoughts on validation:
I was thinking something like this on the RichTextBlockWithFootnotes class:
def clean(self, value):
    cleaned_data = super().clean(value)
    page = {} # Not sure how to get the page from here...
    footnote_uuids = []

    def add_uuid(match):
        footnote_uuids.append(match.group(1))
    FIND_FOOTNOTE_TAG.sub(add_uuid, str(value))

    for footnote_uuid in footnote_uuids:
        if not page.footnotes.filter(uuid=footnote_uuid).exists():
            raise ValidationError(f"Invalid footnote reference {footnote_uuid}")
    return cleaned_data

but I couldn't work out how to get the page (I'm not convinced this is possible).
Open to suggestions/ideas

I am splitting the Validation logic out into its own issue to be dealt with separately as I think it needs a little more thought.
See: https://github.com/torchbox/wagtail-footnotes/issues/17


Imed docu branch

csv import

Thanks for the docu =)
This weeks I am focus on nested info (could be a solid base for rollups too)


Add [Coincap] Cryptocurrency badges

Added

Symbol
Rank
Price
Cheange Percent on 24hr

Messages

:book:
:sparkles: Thanks for your contribution to Shields, @alexandresanlim!

  Generated by :no_entry_sign: dangerJS against 14b1f82419edb2ba043c74371080c3672662ff7c


Updated federation docs

Last bit - I have also reviewed amqp-broker-connections.adoc as I was learning/reading it.
No more new additions from me. Just fixes.

I squashed the commits, cleaned up a few things, created a Jira, updated the commit message, and merged.


CLS impacted by element not visible

Hello,
It seems that the metric CLS is impacted by any elements not visible by human.
For instance if an invisible element located behind an overlay (ZIndex usage) shifts then the metric is impacted whereas it is not visible.
This is not consistent with the definition of the metric.
I think that it could be fix into implementation of metrics because visibility is well done when we use property like : opacity visibility or display.

Several bugs related to layout-shift entries being reported for invisible elements have been fixed in Chrome recently. See the metrics CHANGELOG for more details. In summary, elements that are either display:none, visibility:hidden, or opacity:0 should not trigger layout shifts.
In general, it's a good idea to test your site in Chrome Canary before reporting bugs because in some cases the bugs have already been fixed.
As for the issue of z-index and stacking order, here you're correct that the intention of the metric is not being honored, and these layout shifts are reported even though they're not visible. We'd like to eventually find a way to ignore these, but (speaking at least for Chrome) attempting to ignore shifts that are invisible due to stacking order is not possible to do currently without negatively affecting performance. So the advice is: if you have items that you know might shift, hide them with visibility or opacity until you need to show them.


Using CHB-MIT seizure dataset in EDF format

I was wondering how could I use MNE for CHB-MIT seizure dataset classification which does not have stim channel and hence I can not find events.

please share a file or tell us what you typed to try

the CHB-MIT eeg database has no annotation channel so how can I use MNE to find events like seizure.
It gives error that " no STIM channel found

how are stored the annotations? in the edf file?
please give me an example file

Here is the link for the files. These are edf files and seperate annotation file but no annotation channel
https://physionet.org/physiobank/database/chbmit/

I have no problem reading a file with current master:
raw = mne.io.read_raw_edf('chb01/chb01_01.edf', stim_channel='auto')
the annotation of seizures are stored outside of the edf AFAIK see
eg chb01_03.edf.seizures
don't bother about stim channel

thanks but how can i find events for classification purpose

Ho can I find events in a file with no annotation channel

you need to read the seizure file and do the maths to know where it
corresponds to in the file. this is not an MNE problem.

@agramfort , thanks

hi @syedumaramin can you share how did you manage to read the .seizure file? i am a beginner in this EEG im having difficulty to read the .seizure using mne. Thank you so much

Can anyone share the pre-processing code of CHB-MIT dataset. I am new in EEG and facing difficulty to read the files. My email -<EMAIL_ADDRESS>Thank you

@saiful1604008 please use the MNE forum for usage questions. https://mne.discourse.group


Concurrent updates: race can easily result in "silent squash" failures

the openwhisk api does not return a revid upon get. this means that updates against a resource may silently fail -- e.g. both updates from thread T1  and thread T2 will succeed (as indicated by openwhisk), but one or the other updates will not be visible in the steady state.
solution 1: expose the couchdb revid
solution 2: expose it, but call it something else (e.g. semver)
solution 3: more fully reify the existing version field to serve this purpose
along these lines, we should consider whether to only allow updates without a revid when passed a force parameter, and also how to document this lack of thread safety

solution 4:   implement a lock per entity and enforce a sequential consistency model

1-3 are all invasive. 4 is potentially lowest hanging fruit as we already have a lockable entry in the cache. We had considered this before but eschewed it and allowed reads around a write to go straight to the db.


Attribute Error while running on Travis CI

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/_pytest/main.py", line 99, in wrap_session
INTERNALERROR>     config._do_configure()
INTERNALERROR>   File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/_pytest/config.py", line 920, in _do_configure
INTERNALERROR>     self.hook.pytest_configure.call_historic(kwargs=dict(config=self))
INTERNALERROR>   File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/pluggy/__init__.py", line 628, in call_historic
INTERNALERROR>     res = self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/pluggy/__init__.py", line 222, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/pluggy/__init__.py", line 216, in <lambda>
INTERNALERROR>     firstresult=hook.spec_opts.get('firstresult'),
INTERNALERROR>   File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/pluggy/callers.py", line 201, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/pluggy/callers.py", line 76, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/pluggy/callers.py", line 180, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/travis/build/wright-group/WrightTools/.eggs/pytest_mp-0.0.4-py3.6.egg/pytest_mp/plugin.py", line 380, in pytest_configure
INTERNALERROR>     mp_reporter = MPTerminalReporter(standard_reporter, manager)
INTERNALERROR>   File "/home/travis/build/wright-group/WrightTools/.eggs/pytest_mp-0.0.4-py3.6.egg/pytest_mp/terminal.py", line 17, in __init__
INTERNALERROR>     self._tw = self.writer = reporter.writer  # some monkeypatching needed to access existing writer
INTERNALERROR> AttributeError: 'TerminalReporter' object has no attribute 'writer'

This error was generated when running with Python 3.6, but not with Python 3.5.
I did not get this error on my local machine (on python 3.6), only on Travis.
Log located here:
https://travis-ci.org/wright-group/WrightTools/builds/389486834

My current best guess versions of Pytest. I will note that there is a comment in the _pytest/terminal.py that says # self.writer will be deprecated in pytest-3.4, however that is there in the pytest 3.6.1 version, so something is amiss there.
perhaps this library should use _tw rather than writer?

https://github.com/pytest-dev/pytest/commit/cf0cac3b734dd47c8f6cbf3afa759231fbb76bc9#diff-8b669271dfb5b91a3cec82b3b1ce2fa1
This commit somewhat explains what happened, but the fate of this attribute long term is a little iffy.

A slight bit more info, the attribute in question was removed accidentally in pytest 3.3.0, but replaced in pytest 3.3.1. They have changed to use _tw internally, but have not removed the writer alias as of yet.
This incompatibility, combined with the uncertainty of how long writer will be around lead me to think the line here should be something along the lines of:
try:
    self._tw = self.writer = reporter.writer  # some monkeypatching needed to access existing writer
except AttributeError:
    self._tw = self.writer = reporter._tw  # writer attribute replaced in some versions of pytest

Either that or setup.py should explicitly disallow the incompatable version of pytest


Include GUI jobs into Rancher pipeline

ref; https://github.com/mayadata-io/oep-e2e-rancher/pull/42


Matching sub-folders for source and destination

I can't force this plugin to use {base: './my/path/'} in gulp.src,
so i can keep folder structure in destination folder.
I want to compile svg files from folder:
project/src/a|b|c|d(...)/icons/*.svg
to folder:
project/dest/a|b|c|d(...)/icons/sprite.svg
Why i can't keep my file structure in destination folder while generating sprites?

@rgb4u I'm sorry but I can hardly understand your question. If possible please try to rephrase it. Also, reading the docs about output destinations might help you understand how svg-sprite has to be configured.

Sorry for my mistake, i try to set base, cwd for gulp.src without result
Every code compilation works fine, and my files matching
source/destination folders,
only images don't work, sprites are generated, but not in
.project/destination/**/
subfolders :(
On 2016-06-21 16:58, Joschi Kuphal wrote:

@rgb4u https://github.com/rgb4u I'm sorry but I can hardly
understand your question. If possible please try to rephrase it. Also,
reading the docs about output destinations
https://github.com/jkphl/svg-sprite/blob/master/README.md#output-destinations
might help you understand how /svg-sprite/ has to be configured.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/jkphl/svg-sprite/issues/170#issuecomment-227467066,
or mute the thread
https://github.com/notifications/unsubscribe/AJm1S9NEahdod7TSwUgNhW9fBazWzh7Qks5qN_wUgaJpZM4I609O.

Hi
Can you re open this issue ? I have the same problem ! Can't keep the original folder structure even if i type the base option.
gulp.src('assets/img/sprite/**/*.svg', {base: '.'})

Please understand that this is not an "issue" but intended behaviour. svg-sprite is for generating sprites, not for batch-optimizing SVG files. Please use SVGO for that. In order to create a sprite, the file paths need to get translated into a unique ID for each shape. The original file path is then irrelevant for svg-sprite and discarded. The fact that svg-sprite provides an option to keep and output the intermediate SVG files (with the IDs a file names) is nothing else than a convenience function.  As I said, please use SVGO directly for what you want to achieve.


Versioning reset?

We've had gist-embed on cdnjs for a while, and it has 2.x versions present: https://cdnjs.com/libraries/gist-embed
This was updating from an old GitHub repo that no longer exists: https://github.com/blairvanderhoof/gist-embed
I was looking to update the auto-update target to be this repository, but it looks like the versioning has been reset, which means cdnjs won't pull in new versions.

Hi! I’m no longer maintaining that old repository because that account was deleted. I’m now serving this repository with jsdelivr.  Can we keep what’s on cdnjs so peoples websites don’t get affected? We can turn auto update off.

We won't ever remove or change existing assets on cdnjs, but it would be nice to be able to publish new versions for folks. cdnjs has far larger usage than jsdelivr.
I guess we have two options that'd allow that to work: either versions jump a major here so that they'll align with the old versions, or we can publish this updated repo under a new name on cdnjs. Which do you think makes the most sense?

@bvanderhoof Hello! I want to make a Pull Request to a given project. Can you suggest the current repository?


Presentation duration limit

Is there a limit for how long you can share your screen in a Teams meeting?  Or would that limit for sharing your screen be 24 hours?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 905e2c09-ef3e-07cc-0809-6ff171157c47
Version Independent ID: c0d33162-183d-fff8-32e9-a999f391ea51
Content: Limits and specifications for Microsoft Teams - Microsoft Teams
Content Source: Teams/limits-specifications-teams.md
Service: msteams
GitHub Login: @SerdarSoysal
Microsoft Alias: serdars

@erictechtips Thank you for submitting feedback.
I think the best way forward is if you open a service ticket in your tenant so this can get resolved ASAP. Based on the outcome let me know if it is something that can be called out in the docs.
Please follow this link to contact support for business products:https://docs.microsoft.com/office365/admin/contact-support-for-business-products
Please keep us posted here on the resolution so we can feed whatever information you discover into the content.


despf.sh allows invalid CIDR ranges as output

This isn't technically a "bug" in despf.sh, in that it's doing what it's supposed to do: query the DNS and spit out the IP addresses and CIDR ranges.
This is an enhancement request to do some sort of validation checking on the CIDR range before outputting it. For example, when I run despf.sh against outlook.com (or even just against _spf-ssg-b.microsoft.com, which is one of the include: statements that gets evaluated), two of the CIDRs published by Microsoft (<IP_ADDRESS>/30 and <IP_ADDRESS>/26) are invalid.
It would be wonderful if despf.sh ignored invalid CIDRs, and simply didn't output them. Thanks! :)

Perhaps Bill Cole's suggestion of using perl might help, or at least nudge you in the right direction?
http://thread.gmane.org/gmane.mail.postfix.user/253287/focus=253331

I'm working a shell script just for the cleanup. I'm reusing code sent to @stevejenkins . I'll send a pull-request in a few hours.

@jcbf :+1:

@stevejenkins thanks for reporting this issue and sharing the link!

Merged by @jcbf. See ./normalize.sh. No entry in README yet, though.

Re-opening this, hoping that @jcbf can come up with something creative to also address IPv6. LinkedIn.com is currently publishing 2620:109:c006:104::215/32 in their SPF record, which is invalid.

Wait, perhaps I spoke too soon. despf.sh might be creating the invalid IPv6 record itself:
# dig txt linkedin.com | grep spf1
linkedin.com.           3485    IN      TXT     "v=spf1 ip4:<IP_ADDRESS>/25 ip4:<IP_ADDRESS>/24 ip4:<IP_ADDRESS>/24 ip6:2620:109:c00d:104::/64 ip6:2620:109:c006:104::/64 ip6:2620:109:c003:104::/64 ip4:<IP_ADDRESS> ip4:<IP_ADDRESS> mx mx:docusign.net ~all"```

The TXT record only has 3 IPv6 entries. But despf.sh spits out 4, and the additional one is the invalid one (see comment above).
Something in desph.sh is generating the ip6:2620:109:c006:104::215/32 from linkedin.com's SPF record.

And even though this doesn't seem to be an IPv6 validation issue, we should look ahead to IPv6 validation anyway.
Resources:

http://twobit.us/2011/07/validating-ip-addresses/
http://stackoverflow.com/questions/14697403/expand-ipv6-address-in-shell-script

@stevejenkins Fixed. Thanks for catching this. I did not like the hardcoded 32 when I saw it, but did not have time to test thoroughly. Feel free to close this issue if it is fixed now.

Confirmed. That fixes it! :)

@stevejenkins BTW, what does make the CIDR ranges you mentioned in the first post invalid?
$ ipcalc <IP_ADDRESS>/30
Address:   <IP_ADDRESS>       11001111.01000100.10101001.101011 01
Netmask:   <IP_ADDRESS> = 30 11111111.11111111.11111111.111111 00
Wildcard:  <IP_ADDRESS>              00000000.00000000.00000000.000000 11
=>
Network:   <IP_ADDRESS>/30    11001111.01000100.10101001.101011 00
HostMin:   <IP_ADDRESS>       11001111.01000100.10101001.101011 01
HostMax:   <IP_ADDRESS>       11001111.01000100.10101001.101011 10
Broadcast: <IP_ADDRESS>       11001111.01000100.10101001.101011 11
Hosts/Net: 2                     Class C

$ ipcalc <IP_ADDRESS>/26
Address:   <IP_ADDRESS>        01000001.00110111.11101110.10 000000
Netmask:   <IP_ADDRESS> = 26 11111111.11111111.11111111.11 000000
Wildcard:  <IP_ADDRESS>             00000000.00000000.00000000.00 111111
=>
Network:   <IP_ADDRESS>/26     01000001.00110111.11101110.10 000000
HostMin:   <IP_ADDRESS>        01000001.00110111.11101110.10 000001
HostMax:   <IP_ADDRESS>        01000001.00110111.11101110.10 111110
Broadcast: <IP_ADDRESS>        01000001.00110111.11101110.10 111111
Hosts/Net: 62                    Class A

@jcbf I do not see what normalize.sh does. Can you please provide a test input and output?

Using @stevejenkins example domain
 ./despf.sh hotmail.com > /tmp/hotmail.spf
 cat /tmp/hotmail.spf | ./normalize.sh  > /tmp/hotmail.normalized
 diff -u /tmp/hotmail.normalized /tmp/hotmail.spf

should output diff
--- /tmp/hotmail.normalized	2015-12-02 15:53:10.000000000 +0000
+++ /tmp/hotmail.spf	2015-12-02 15:52:31.000000000 +0000
@@ -43,7 +43,7 @@
 ip4:<IP_ADDRESS>/25
-ip4:<IP_ADDRESS>/30
+ip4:<IP_ADDRESS>/30
 ip4:<IP_ADDRESS>/26
@@ -70,7 +70,7 @@
 ip4:<IP_ADDRESS>/26
-ip4:<IP_ADDRESS>/26
+ip4:<IP_ADDRESS>/26
 ip4:<IP_ADDRESS>/28

Sorry, I typo'd the corrected IP address that MS is reporting in my first post. I've edited the original post. Here's what MSFT reports from one of their records:
# dig _spf-ssg-b.microsoft.com txt | grep spf1
_spf-ssg-b.microsoft.com. 494   IN      TXT     "v=spf1 ip4:<IP_ADDRESS>/30 ip4:<IP_ADDRESS>/26 ip4:<IP_ADDRESS>/27 ip4:<IP_ADDRESS>/27 ip4:<IP_ADDRESS>/26 ip4:<IP_ADDRESS>/26 ip4:<IP_ADDRESS>/29 ip4:<IP_ADDRESS>/27 ip4:<IP_ADDRESS>/27 ip4:<IP_ADDRESS>/28 ~all"

Note that <IP_ADDRESS>/30 appears once and <IP_ADDRESS>/26 appears twice in that record. Both are invalid.
When run through ipcalc:
# ipcalc -n <IP_ADDRESS>/30
NETWORK=<IP_ADDRESS>
# ipcalc -n <IP_ADDRESS>/26
NETWORK=<IP_ADDRESS>

If NETWORK != the original IP, then the IP address is invalid. In my Postwhite script, I give the user the option whether to replace the invalid IP with the corrected one, remove it from the results, or leave it in the results.

@jcbf and @stevejenkins, thanks for clarification!!! I have added tests for normalize.sh and added it also to README file. Hats off for cooperation! Thanks!!!
Now I just need to fix the failing added test: https://github.com/jsarenik/spf-tools/pull/46

Sorry for re-opening, but I'd love to see an option for ``normalize.sh(or perhaps a different script altogether such asomitinvalid.sh) that performs the inspection as normalize.sh```, but could omit the invalid IPv4 CIDR from the results.
It's technically possible that normalizing an invalid IPv4 CIDR could change it to a netblock that is not actually owned or controlled by the reporting entity.
That way, despf.sh microsoft.com | normalize.sh and despf.sh microsoft.com | omitinvalid.sh would both produce lists of valid IPs and CIDRs, but the first option would include "fixed" ones and the second one wouldn't.

Omitting those may raise another issue.
In your example, it is somehow expected that ip4/ip6 mechanisms are well defined CIDR but that may not be the case for the a and mx mechanisms.
For instance, assume that www.example.com resolves to <IP_ADDRESS> and you find a SPF record containing "v=spf1 a:www.example.com/28 -all" . In this case despf.sh will expand to ip4:<IP_ADDRESS>/28. It is a valid record and should not be ommited.
I understand your concerns but  normalize.sh is behaving according to RFC and is consistent whit SPF validators.
I could make more checks in normalize.sh to catch invalid IP or networks like
./normalize.sh <<< "ip4:<IP_ADDRESS>0/27"
ip4:<IP_ADDRESS>/27

As per RFC these record should be omitted.

Hi, @jcbf. I agree that all valid IP addresses and ranges should be included.
I'm only talking about what to do with invalid IP CIDR ranges. The default behavior for normalize.sh is to "repair" them. But the repaired CIDR is different than the CIDR published by the host. So what happens if the repaired CIDR actually represents a netblock that is not actually owned by the host? Sure, the CIDR might be technically valid, but it may not represent a netblock that the host wishes to claim it owns. On the contrary, it could lead to whitelisting of untrustworthy hosts.

Hi,
let me see if I got this right.
Quoting RFC 7208, Section 5.  Mechanism Definitions
 If a CIDR prefix length is specified, then only the specified number
   of high-order bits of <ip> and the IP address are compared for
   equality.

What normalize.sh does is zero the low-order bits accordingly with the netmask.
The result will be the first IP address of that block. It is a different record but it is syntactically equivalent and only make sense for applications that are picky regarding CIDR format. In defence to Postfix I may add that CIDR (as of RFC 4632) applies to networks and CIDR tables in Postfix should only use valid networks addresses. In SPF the CIDR notation is used slightly in a different way whereas  a CIDR record referes tan IP address and netmask.
When spammers initially started to use SPF enable domain to send spam we started to observe allot of
v=spf1 +all
v=spf1 <IP_ADDRESS>/0 -all
v=spf1 <IP_ADDRESS>/1 <IP_ADDRESS>/1  -all
etc

I you are concern that something similar to this may happen, you should be careful about the domains you choose to whitelist. normalize.shwill not change the record to give you other netblock than the intended one. On the other side, you got domain owners that can include whatever they want, whether or not they own it or not.
I really need more coffee, so if I'm not really answer for question please provide an example.


missing files

hi, I ran your code glyce/glyce/bin/run_bert_glyce_classifier.py, but there some errors I found.
first: glyce/glyce/dataset_readers/bert_data_utils.py is empty
second: i couldn't find file " glyce.data.data_utils import *" that you want to import
third: "from glyce.models.glyce_bert.glyce_bert_classifier import GlyphBertClassifier " is it should be "from glyce.models.glyce_bert.glyce_bert_classifier import GlyceBertClassifier"?

I will check these issues later today. Reply to you at the latest tomorrow. Thanks a lot.

hi, I have another question. If i want to train the single sentence classification, i just need to change the
data_processor in run_bert_glyce_classifier.py, right?

Yes. You should change 【data_processor】 if you would like to apply Glyce-BERT to a single sentence classification dataset. I will upload the latest code these two days. Thanks for your focus. :-)

I'm sorry to bother you, but there are still some problems I can't solve:
Traceback (most recent call last):
File "run_bert_glyce_classifier.py", line 29, in
from glyce.models.glyce_bert.glyce_bert_classifier import GlyphBertClassifier
File "/home/youmin/syc/glyce-master/glyce/models/glyce_bert/glyce_bert_classifier.py", line 29, in
from glyce.layers.glyce_transformer import GlyphTransformer
File "/home/youmin/syc/glyce-master/glyce/layers/glyce_transformer.py", line 38, in
from glyce.layers.glyph_position_embed import GlyphPositionEmbedder
File "/home/youmin/syc/glyce-master/glyce/layers/glyph_position_embed.py", line 38, in
from glyce.glyph_embedding.models.glyph_char import CharGlyphEmbedding
ModuleNotFoundError: No module named 'glyce.glyph_embedding'

I will check and update all the files tonight. You will be notified of any updates. Thanks for your focus!!!!!

I will check all the files tonight. You will be notified of any updates. Many thanks for your focus!!!!!

I will check tonight. You will be notified of any updates. Many thanks for your focus!!!!!

from glyce.data.data_utils import *  i could not find  dir data  and file  data_utils.ModuleNotFoundError: No module named ''glyce.data.data_utils'

hi,i am sorry to bother you ,but when i run_bert_glyce_tagger.py , I found pinyin_vocab.json to be the same as  wubi_vocab.json. I was a little strange.Maybe there were some mistakes. I hope to reply.

I'm sorry to bother you again. When I ran that code, I found that you took bert's word2id as input, but dict_word ['idx2char'] used the other one. There should be a problem here. Could you check if you have time? If you have time, please reply as soon as possible.

Thanks for your issues. Please pull the latest code. :-)


Huggingface URL parsing

Is your feature request related to a problem? Please describe.
parsing of huggingface url support needed. when given repos url containing more in the path, scanning failed.
Describe the solution you'd like
parsing to determine correct huggingface url will solve the issue.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

Feature available in v1.1 Release


Refactored com.google.fonts/check/metadata/valid_copyright into two checks

com.google.fonts/check/metadata/valid_copyright will not run on upstream
font repos since it needs a METADATA.pb file. This check will now only check
that METADATA.pb files have the correct copyright string.
New check com.google.fonts/check/font_copyright will ensure that a font's
copyright strings are correct.
(follow-up to PR #2383 / Fixes #2210)

Fabulous. Thank you!

:-D

this one will be much harder LOL! https://github.com/googlefonts/fontbakery/pull/2361


Fix prefix error if prefix equals a php command

Fix php error if prefix equals a php command like "chr" for example.
Replaced is_callable with instanceof Closure, which should preserve the desired behavior.
Hopefully better fix for #1339

coverage: 80.208%. remained the same when pulling 14064c67f819ebc1e41beefbe528b4a44e2a2d30 on mmodler:better-prefix-fix into 577bd398727a6fc953658563b1271e498f059098 on predis:v2.x.

@tillkruss Now tests are passing, only code style test failing but has nothing to do with my change?

See #1329.

@vladvildanov Should we move this into v3?

@tillkruss +


Updated endpoints pr

Keeps this PR up to date: https://github.com/boto/botocore/pull/769
I also removed the behavior where you can specify the endpoint prefix of a service instead of the service name and still be able to get the regions.
cc @jamesls @JordonPhillips

:shipit:


MenuToggle and MenuEnumScroller bugs, plus enhancement idea

I found some bugs:

In MenuToggle, the initial value given in the constructor does not seem to work. The toggle will show an off value regardless.
In MenuEnumScroller, you can go out of bounds and it generates exceptions.

enhancements:
1.for the scrollers, have them jump between the first/last values whenever going out of bounds.
2.the numeric scroller's interface(aka get/set property and constructor) shouldn't use 'timesIncremented', instead it should use the value itself.
3. The EnumScroller currently uses a string array and an index to access the values, which isn't very convenient. I think it would be better if it had used a generic list for the elements, and perhaps a Func<T,string> that would display those elements. (And use ToString if no Func has been specified). In addition, instead/in addition to exposing the current index, it should have a CurrentItem property, plus setting the selected item in the constructor, instead of the selected index.

Ah i thought it was just me doing something wrong with that initial value. Tried all sorts of things but returned (OFF) regardless of the boolean.

It returns the correct value, but is not showing it. Correct?

Fixed in #141 :)

If this is resolved, can this issue be closed?

Sure @nevadascout


[Feat] show total of records in table

Feature request
Is your feature request related to a problem? Please describe.
Show a total of records in table.
for example 1300 records in total in the footer
The 1 - 100 of statement doesn't really make sense if you are filtering and sorting since the order and amount on display are no longer relevant, so maybe it's best we still don't include that.
previous iteration of grid

@phamhieu did you implement this somewhere else - i remember seeing a comment somewhere


Feat: Add Option to Specify Other LLM Providers in .env

Formatted .env a bit also.
/closes #10
/claim #10

@nkkko could you review this pr?

@RS-labhub on it

@RS-labhub I'm getting this error:

@RS-labhub I'm getting this error:

Did you install fasthtml?? pip install fasthtml
By the way, I didn't touched the previous packages. So, it must be not from my side!

@RS-labhub I’ve updated main with some changes that affect this PR. Could you please rebase your branch onto the latest main and resolve any conflicts that come up? Once that’s done, we can review the updated changes and move forward.

All good this side!

@RS-labhub your pr doesn't have latest chnages, please rebase.

Check it now @mojafa

I commendtyed out the bit for # LLM_PROVIDER=AzureOpenAI and used my old ENV and it worked. I was expecting an error to tell me ot specify my LLM

The issue here is that the script defaults to AzureOpenAI if the LLM_PROVIDER is not specified in the env, so it bypasses the missing environment variables check even when LLM_PROVIDER is absent.
Making a quick fix for this

@RS-labhub i'm seeing the Error: LLM_PROVIDER is not specified. Please set LLM_PROVIDER in your environment variables in the termnial but code still runs and geenrates devconatiner.

I swtiched from azure to Anthropic. added my key and it doesnt run

@RS-labhub pr fails DCO check, please sognoff your commits

@RS-labhub getting this error on runnninh pip install. I dont think we have that distibition for antrohpic, please test on your end. no rush

Done

fix DCO please sign your commits

@RS-labhub Please update your readme file with changes

Rest everything is fine?? @mojafa

@RS-labhub getting this error on running the app:


implement netCDF4-c default chunksize

[x] Closes Issue #52
[ ] Tests added
[ ] Changes are documented in CHANGELOG.rst

For the script from #52 this prints:
h5netcdf: 19MB
netcdf4: 19MB

as expected.

Some tests are failing but I wanted to get your feedback on some issues first:

Should this change the default only for the legacy API or for the new API, too?
There is some logic that ensures the number of bytes per chunk is below NC_MAX_UINT (4_294_967_295 by default). I assumed we don't need that so I didn't port it. Do you agree?

@dionhaefner Nixe! Let's see how the tests run. Maybe also add a test which checks for correct chunk sizes after file creation?
To your questions:

It should only be default for the legacy API.

Not sure. Is this a hard restriction of the netCDF4 standard?

Another thing with respect to the ported code -> we might have to add the netcdf-c license text somewhere. @shoyer recommended to put the code in a separate file in such occasions. But let's wait for his opinion.

It should only be default for the legacy API.

That sounds confusing to me, but well, it's your library. In this case this should probably be documented somewhere? Right now the README makes it sound like old and new API are functionally equivalent.

Not sure. Is this a hard restriction of the netCDF4 standard?

Looks more like this is a limitation of HDF5:

The maximum number of elements in a chunk is 2^32-1 which is equal to 4,294,967,295.

If I try to create a bigger chunk, h5py throws an exception:
>>> f.create_dataset("foo", shape=(int(5e9),), chunks=(5e9,))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/dion/.pyenv/versions/h5netcdf/lib/python3.8/site-packages/h5py/_hl/group.py", line 149, in create_dataset
    dsid = dataset.make_new_dset(group, shape, dtype, data, name, **kwds)
  File "/home/dion/.pyenv/versions/h5netcdf/lib/python3.8/site-packages/h5py/_hl/dataset.py", line 109, in make_new_dset
    dcpl = filters.fill_dcpl(
  File "/home/dion/.pyenv/versions/h5netcdf/lib/python3.8/site-packages/h5py/_hl/filters.py", line 256, in fill_dcpl
    plist.set_chunk(chunks)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5p.pyx", line 473, in h5py.h5p.PropDCID.set_chunk
ValueError: All chunk dimensions must be less than 2^32 (all chunk dimensions must be less than 2^32)

I'll patch in the code path for big chunks, then.

What does h5py use for chunksize heuristics by default?
Generally we try to make the new API more "h5py-like" and the legacy API more "netCDF4-like" but that isn't a hard rule, and it's nice to be able to achieve the same behavior with both APIs.

What does h5py use for chunksize heuristics by default? Are there cases where users would prefer that?

The heuristic is here:
https://github.com/h5py/h5py/blob/1d569e69fb8d2b1abea196792bb1f8c948180534/h5py/_hl/filters.py#L338-L389
Basically it starts with the dimensions of the dataset (or a fixed 1024 for unlimited axes) and divides each iteratively by 2 until it is close enough to the target chunksize (between 8K and 1M).
IMO the biggest issue is that unlimited dimensions default to a chunk size >> 1. In any NetCDF file I've encountered, unlimited dimensions are something like time where you just extend by 1 at a time.

tl;dr chunking heuristics is quite complex
@dionhaefner Just to summarize: the major pain point with the new API is that the unlimited dimensions can get a default chunk size (using h5py heuristics) much larger than 1 on creation of the dataset. This, in turn, can result in exploding filesizes, as follows:
Array size
(1, 40, 250, 250)
Chunks
netcdf4
(1, 20, 125, 125) -> 2500000 bytes per chunk
1 * (2 chunks * 20) * (2 chunks * 125) * (2 chunks * 125) * 8 byte= 20000000 (complete netcdf4 size)
h5netcdf
(64, 3, 16, 16) ->  393216 bytes per chunk
64 * (14 chunks * 3) * (16 chunks * 16) * (16 chunks * 16) * 8 byte=<PHONE_NUMBER> (complete h5netcdf size)
Citing from Andrew Colletes Book (Python and HDF5):

The “auto-chunker” tries to keep chunks mostly “square” (in N dimensions) and within certain size limits.

Do you even need to specify a chunk size?
It’s best to restrict manual chunk sizing to cases when you know for sure your dataset will be accessed in a way that’s likely to be inefficient with either contiguous storage or an auto-guessed chunk shape. And like all optimizations, you should benchmark!
Try to express the “natural” access pattern your dataset will have
As in our example, if you are storing a bunch of images in a dataset and know that your application will be reading particular 64×64 “tiles,” you could use N×64×64 chunks (or N×128×128) along the image axes.
Don’t make them too small
Keep in mind that HDF5 has to use indexing data to keep track of things; if you use something pathological like a 1-byte chunk size, most of your disk space will be taken up by metadata. A good rule of thumb for most datasets is to keep chunks above 10KiB or so.
Don’t make them too big
The key thing to remember is that when you read any data in a chunk, the entire chunk is read. If you only use a subset of the data, the extra time spent reading from disk is wasted. Keep in mind that chunks bigger than 1 MiB by default will not participate in the fast, in-memory “chunk cache” and will instead be read from disk every time.

I'm not sure what netcdf-c/netcdf4 has in the loop in terms of caching, but hdf5/h5py has a restriction on cache size which can only be adapted at file-open time.
In our example case that would mean that performance might be degraded, since the chunk size is well above the 1MiB limit (2.5 MiB). On the other hand the chunk size for time dimension is definitive too big (64(. We can't just set it to 1, since this would change the size of one chunk to only 1 * 3 * 16 * 16 * 8 byte = 6144 bytes which is below the recommended lower thrshold of 10KiB.
If the conformity with hdf5/h5py is not a hard rule, as @shoyer pointed out, I think we have several options here.

keep everything as is
switch to netcdf4 heuristics for legacy API, keep new API as is
switch to netcdf4 heuristics for both API's
use netcdf4 heuristics for legacy API, reimplement hdf5 heuristics for new API, but special case unlimited dimensions
other

For the script in #52 I get (using this PR) the following chunk sizes:
"test" variable
h5netcdf: (64, 3, 16, 16)
legacy: (1, 20, 125, 125)
netcdf4: (1, 20, 125, 125)
"time" variable
h5netcdf: (1024,)
legacy: (4096,)
netcdf4: (512,)
"z", "y", "x" variables
h5netcdf: None, None, None
legacy: (40,), (250,), (250,)
netcdf4: contiguous, contiguous, contiguous
@dionhaefner It seems that the coordinate variables are treated a bit different with the ported heuristics than original netcdf4.

🤔 I see. I'll have a look at the coordinate variables, perhaps just some integer / float division issue.

In our example case that would mean that performance might be degraded, since the chunk size is well above the 1MiB limit (2.5 MiB).

I'd like to point out that the 1MB cache is not per chunk, but for the whole dataset:

rdcc_nbytes sets the total size (measured in bytes) of the raw data chunk cache for each dataset. The default size is 1 MB. This should be set to the size of each chunk times the number of chunks that are likely to be needed in cache.

So whether your chunks are 300kiB or 2.5MiB doesn't really matter in any reasonable application since even in the best case you can only hold 3 chunks in the cache anyway, and unless you are reading these over and over again it doesn't matter.
But that being said I'd like to add another possibility:

Use the netCDF4 heuristic with an added upper limit of 1MiB like in the h5py heuristic for both new and legacy API (i.e., sort of a hybrid heuristic that respects h5py cache defaults)

In our example case that would mean that performance might be degraded, since the chunk size is well above the 1MiB limit (2.5 MiB).

I'd like to point out that the 1MB cache is not per chunk, but for the whole dataset:

Yes, sorry, you're completely right. I did not read careful enough.

But that being said I'd like to add another possibility:
* Use the netCDF4 heuristic with an added upper limit of 1MiB like in the `h5py` heuristic for both new and legacy API (i.e., sort of a hybrid heuristic that respects `h5py` cache defaults)

That sounds reasonable. Would you take a stab at implementing it? You might enhance the current example and convert it to a test, too.
For the time being I would also suggest to keep current behaviour, the netcdf-heuristics and your combined approach available codewise and decide which combination should be used after a final discussion together with @shoyer.
We might have to think about some FutureWarning/DeprecationCycle anyway.

Actually, netCDF4 increases cache size to 32MB by default:

The cache size for data transfers in HDF5 is 1 MB, by default. This is small for many real applications, so netCDF-4 uses a default cache size of 32 MB.

Hence the target chunk size of 16MB in the netCDF4 heuristic. Are you sure you want to stick with the HDF5 default cache size of 1MB?

Looks like we've opened a can of worms. :grimacing: We might have to open files with rdcc_nbytes=CACHESIZE, to make use of it, right?

Yes that's what I had in mind.
I think 1MB sounds awfully small for modern hardware, and I'm sure the netCDF4 devs benchmarked this properly, so I'd probably increase it. But I understand if you want to stay compliant with HDF5, and I don't think it's going to make a huge difference in practice.

I do not sea a huge problem if we use this for the legacy API, as it should align with netcdf4 anyway. But I'm uncertain how to handle this for the new API. I'd wait for @shoyer's insight, which way this should be implemented in a future proof maybe parameterizable way.

I did some quick and dirty benchmarking with different options, and as it turns out, the h5py default settings actually perform best:
{'compression': None, 'chunks': True, 'rdcc_nbytes': 1048576}
write: 0.686s
size: 200MB
chunksize: (2, 5, 32, 32)
read: 0.104s
---
{'compression': None, 'chunks': True, 'rdcc_nbytes': 33554432}
write: 0.712s
size: 200MB
chunksize: (2, 5, 32, 32)
read: 0.117s
---
{'compression': None, 'chunks': (4, 20, 125, 125), 'rdcc_nbytes': 1048576}
write: 0.763s
size: 228MB
chunksize: (4, 20, 125, 125)
read: 0.202s
---
{'compression': None, 'chunks': (4, 20, 125, 125), 'rdcc_nbytes': 33554432}
write: 0.768s
size: 228MB
chunksize: (4, 20, 125, 125)
read: 0.120s
---
{'compression': 'gzip', 'chunks': True, 'rdcc_nbytes': 1048576}
write: 7.375s
size: 180MB
chunksize: (2, 5, 32, 32)
read: 0.920s
---
{'compression': 'gzip', 'chunks': True, 'rdcc_nbytes': 33554432}
write: 7.494s
size: 180MB
chunksize: (2, 5, 32, 32)
read: 0.977s
---
{'compression': 'gzip', 'chunks': (4, 20, 125, 125), 'rdcc_nbytes': 1048576}
write: 9.399s
size: 179MB
chunksize: (4, 20, 125, 125)
read: 1.100s
---
{'compression': 'gzip', 'chunks': (4, 20, 125, 125), 'rdcc_nbytes': 33554432}
write: 9.204s
size: 179MB
chunksize: (4, 20, 125, 125)
read: 1.035s

Now this is for the trivial case of a file with dimensions (10, 40, 250, 250) that is written and read in one go (no slicing involved), but still...
Given that we might just use the h5py heuristic, add a special case for unlimited dimensions and call it a day?

Given that we could just use the h5py heuristic, add a special case for unlimited dimensions and call it a day?

Yes, from the implementation work, this would be the easiest.
To be as versatile as possible we could override the handling of the chunks kwarg in create_dataset like that:
chunks = True # default, same as chunks = "h5py"
chunks = "h5py" # use default h5py/hdf5 heuristics
chunks = "netcdf4" # use netcdf4 heuristics
chunks = "h5netcdf" # use default h5py/hdf5 heuristics with special casing unlimited dimensions

If our aim is to be very much in line with netcdf-c/netcdf4-python we could set the default for legacy API to chunks = "netcdf4".
That way we keep the current behaviour (no deprecations etc.) and enable the user to decide which heuristics to use.

Or even ask the h5py folks whether they could consider changing the default?

I think we should discuss your findings with the h5py people. Maybe they have already a suggestion or issue about that?

This is a pretty neat API!
I have opened h5py/h5py#2029, let's see what they say :)

Looks like they are hesitant to merge this.. Should we go ahead and implement it in h5netcdf, with your proposed API?

@dionhaefner
Yes, I think we can add this here. If this will eventually be used enough, this might get enough interest upstream. I completely understand h5py-devs, as use-cases are widespread and the number people requesting this particular feature low.
chunks = True # default, same as chunks = "h5py"
chunks = "h5py" # use default h5py/hdf5 heuristics
# chunks = "netcdf4" # use netcdf4 heuristics
chunks = "h5netcdf" # use default h5py/hdf5 heuristics with special casing unlimited dimensions

I would not implement the netcdf4 heuristics for now.

How about issuing a warning if chunks=None or chunks=True and unlimited dimensions are present? E.g.
>>> f.create_dataset("foo", shape=(None, 1, 2, 3))
UserWarning: Using h5py's default chunking with unlimited dimensions can lead
to larger file sizes and degraded performance. Consider using `chunks="h5netcdf"`
or explicit chunk sizes. To silence this warning, pass `chunks="h5py"`.

Could we equip this warning with both of the chunk sizes? The user might ideally see, which one to choose for the setup.
I'm still a bit hesitant, since this will clutter the logs downstream and code changes are needed everywhere. As h5netcdf is supposed to reflect netcdf4-python behavior anyway we could instead state that h5netcdf will be the default in later versions (1.0 ?).

I actually think changing the defaults (after a depreciation cycle) is the best way forward, so if you're open to that I'm happy :)

Yes, I do not see an easier way to move on, if we want to make people aware of that. Having that warning there forever is a no-go.

Example warning:

FutureWarning: Using h5py's default chunking with unlimited dimensions can lead to increased file sizes and degraded performance (using chunks: (2, 3, 3, 256)). Consider passing chunks="h5netcdf" (would give chunks: (10, 10, 10, 1); default in h5netcdf >= 1.0), or set chunk sizes explicitly. To silence this warning, pass chunks="h5py".

Is there anything else you would like covered by tests regarding chunking or unlimited dimensions?

@dionhaefner I appreciate all your efforts. Rethinking the whole issue there might be a solution to our problem on the h5py-side (https://github.com/h5py/h5py/pull/2036#issuecomment-1016137566). I'm hoping that this can lead us to prevent the reimplementation of the autochunker code here.

IMO this would be a worse solution, since it wouldn't work in cases where you actually want unlimited chunks > 1. The heuristic should work for all of these shapes:
shape = (None,)  # want chunks (1024,)
shape = (None, 1, 1)  # want chunks (1024, 1, 1)
shape = (None, None)  # want chunks (32, 32)
shape = (None, 128, 128)  # want chunks (1, 128, 128)

The proposed h5netcdf heuristic handles all of these, and more.
Anyhow, my bandwidth to work on this is exhausted. I'll unsubscribe for now, feel free to ping me if you want to take this up again or need my input.

That's why I was wondering if that would be feasible at all. Obviously it's not.
@dionhaefner There are two things to do to get this in finally. Adding the h5py license into licences-folder (partly reimplementation, but would be good anyway) and fixing the linter error. I can do this, if your schedule doesn't allow it. Please let me know. Thanks.

That should do it :)


Update Powershell command

Get-AzureRMVMImagePublisher --> Get-AzureRmVMImagePublisher
Get-AzureRMVMImageOffer --> Get-AzureRmVMImageOffer

@options : Thanks for your contribution to the Azure documentation! The author, @thraka, has been notified to review your proposed change.

Hello @options
Thank you for your change suggestion. Can you please let me know why you want this change?


fix: load tsconfig with extends

Checklist

[x] npm test passes
[x] tests and/or benchmarks are included
[ ] documentation is changed or added
[ ] commit message follows commit guidelines

Affected core subsystem(s)

Description of change

Codecov Report

Merging #82 (d3a9484) into master (c50e5ce) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##            master       #82   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            1         1
  Lines           24        24
  Branches         5         5
=========================================
  Hits            24        24

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c50e5ce...d3a9484. Read the comment docs.


"watch" always listen for an unchanged array !

Version
2.6.11
Reproduction link
https://codepen.io/gh-xiaoxiao/pen/ExgyewZ
Steps to reproduce
Enter text1 or text2, and then view the print information
What is expected?
Print "form.text1" or "form.text2" only
What is actually happening?
"form.arr" is also printed out

It's because you are emitting a copy of the whole object and setting the whole form, and form.arr is triggering because form changed to a new object whereas the other ones are primitives and only triggering if the actual value changes

Remember to use the forum or the Discord chat to ask questions!


Link to KnoxDevs Discord

Should there be a link to join the KnoxDevs Discord server on the homepage? There is currently only a link to join the Slack group on the KnoxDevs website.

@wigging I think that's a great idea.

@gregors Ha, I didn't realize you were on the KnoxDevs GitHub. You're everywhere! I'll submit a Pull Request with the Discord link. Although, you could do this yourself if you want.

🦗


make csi sequences available as &str or String

All the CSI sequences are implemented as structs that implement Display.
This is good when you directly write them in a formatter.
However, I'd like to build Strings that contain CSI sequences.
For now, my only option is to use format! and do things like
let mut line: String::from("some text, ");
line.push_str(&format!("{}", termion::style::Invert));
line.push_str("some selected text");
line.push_str(&format!("{}", termion::style::Reset));

Instread, I'd like to be able to do something like:
let mut line: String::from("some text, ");
line.push_str(termion::style::Invert::as_str());
line.push_str("some selected text");
line.push_str(termion::style::Reset::as_str()));

For dynamic sequences, like RGB colors, we cannot have &str of course, but we could have String.
 String implements core::fmt::Write so I can just do something like termion::style::Invert::fmt(&mut line) in the example above.


Arbitrary values are not working

Arbitrary values are not working:
~text-[2rem]/[4rem]
~py-[2rem]/[4rem]

Hi, could you share your config/setup? I'm not able to reproduce this locally. It does appear that they're broken on Tailwind Play, if that's what you're seeing: https://play.tailwindcss.com/TtzXK6R9I1 (Tailwind Play seems to override the content.extract option); I can add a note about it 🫤

Tailwind Play & Play CDN users: I've opened an issue for this


Format will work very slow on big files

VSCode Version: Code 1.8.1 (ee428b0eead68bf0fb99ab5fdc4439be227b6281, 2016-12-19T14:49:23.350Z)
OS Version: Windows_NT ia32 6.3.9600
Extensions:

Extension
Author
Version

jsrefactor
cmstead
0.5.3

jshint
dbaeumer
0.10.15

vscode-stylesheet-formatter
dbalage
0.2.0

githistory
donjayamanne
0.1.4

jquerysnippets
donjayamanne
0.0.1

JSDocTagComplete
HookyQR
0.0.2

prettify-json
mohsen1
0.0.3

jscs
ms-vscode
0.1.9

debugger-for-chrome
msjsdiag
2.4.2

stylelint
shinnn
0.21.2

addDocComments
stevencl
0.0.8

sort-lines
Tyriar
1.1.0

gitblame
waderyan
1.3.0

bootstrap-3-snippets
wcwhitehead
0.0.9

Steps to Reproduce:

Create or have an unformatted 10+k .js file
Hit the format combination
No format will be executed, not even selected code (>500 lines) format.

There are limits to what VS Code can do. Since you say 10k+ files I assume you have a really large file in which case you hit the case in which we stopping offering fancy features like format et al. That's because we know if will be slow and blocking.

not even selected code format? for 500 lines?

I belive the best way is to warn the user and let the user decide if it's ok to wait and format or not...

not even selected code format? for 500 lines?

The design is that the file doesn't even exist for extensions. That is because most extension don't check for the size of the document they operate on and just start crunching data on load. That's why we are so conservative with very large files.
Sorry that I don't have a better answer but that's the current design and there is no easy way to make a small chance that enabled your (valid) scenario.


As an admin user, I want to logout of the CEPI application using the Logout option in the Navigation menu items

Test script for this story is done but waiting for user credentials that has access to login in dev.


docs(readme): add contributor covenant badge

Self explanatory.

:tada: This PR is included in version 2.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 2.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


docs(website): Publish January 2024 product newsletter

Publish the 2024 product newsletter (email version already sent)

Have links coming inbound to this, so I'll publish it now and address feedback in a later PR.


Typings dont work for redux version 4

ERROR in [at-loader] ./node-modules/@types/redux/index.d.ts:120:18
TS2428: All declarations of Dispatch must have identical type parameters
ERROR in [at-loader] ./node-modules/@types/redux/index.d.ts:166:13, 302:42, 302:53 ......so on
TS2314: Generic type Dispatch<A, D, S> requires 3 type argument(s)

They do. We specifically test them too. I'd recommend looking through the test suite for example usages here: https://github.com/reduxjs/redux/tree/master/test/typescript
We're also working on docs for TS usage in #3201.

@vidhyeshpatil Looks like you have multiple redux typings in your node_modules. Specifically redux now bundles their own types so you don't need @types/redux anymore

@vidhyeshpatil Looks like you have multiple redux typings in your node_modules. Specifically redux now bundles their own types so you don't need @types/redux anymore

@eps1lon - I use react 15.6.2 version with redux 3.6.2 & react-redux 5.0.7.
In the current version, Reducer only take 1 arguments. I need to type safe the reducer also.
Can you help me with some example of simple counter, best practice to use TS with react, redux.

@vidhyeshpatil See my comment above. Our test suite and the WIP docs will have what you're looking for.


Forbid the window to pop out of the screen when moving or scaling it

close with https://github.com/kalisio/kdk/issues/581


feat: add dataset_preview schema

Reviewers
Functional:
@Bento007
Readability:

Changes

add
remove
modify

Definition of Done (from ticket)
QA steps (optional)
Known Issues

outdated


[T7][W15-C2]

@AlvianPrasetya this PR now includes v0.1 code as well. Thanks for your comments on v0.0!


Unable to Upgrade Cluster Configuration at All: Start-ServiceFabricClusterConfigurationUpgrade incorrectly identifies an incompatible authentication type change

I've been learning how to stand up a cluster with X509 authentication, and encountered an issue with the Start-ServiceFabricClusterConfigurationUpgrade incorrectly identifying an incompatible authentication type change. The reality is, I am not changing the authentication type, but merely trying to update the ApplicationPorts.PortRange. I will go over my reproduction steps here:

Create a valid cluster following the template .\Microsoft.Azure.ServiceFabric.WindowsServer.6.4.654.9590\ClusterConfig.X509.MultiMachine.json that is provided with the on-premise install package.
Deploy a small Service Fabric application to validate that the install is stable and correct. In my case I am able to use a client certificate to access my remote cluster and deploy my application. The application successfully deploys.
I created a Powershell script to automate the process of pulling down the existing configuration, modifying the ports and the version, and then invoking the  Start-ServiceFabricClusterConfigurationUpgrade cmdlet.

# This code assumes we are executing out of the
# Microsoft.Azure.ServiceFabric.WindowsServer.6.4.654.9590
# installation package folder

# Load the previous configuration from the cluster into an in-memory JSON object
# that we can modify.
$previousConfig = (Join-Path $env:TEMP ([string][System.Guid]::NewGuid() + ".json"))
Get-ServiceFabricClusterConfiguration | Out-File $previousConfig
$scratchJson = Get-Content $previousConfig | ConvertFrom-Json

# Modify the JSON as needed. In this case I am just modifying the end port.
$scratchJson.Properties.NodeTypes[0].ApplicationPorts.EndPort = 22048 # This is the end port I wanted to use
$scratchJson.ClusterConfigurationVersion = "1.0.1"

# Export the modified in-memory JSON object to a file
$newConfig = (Join-Path $env:TEMP ([string][System.Guid]::NewGuid() + ".json"))
$scratchJson | ConvertTo-Json -Depth 32 | Out-File $newConfig

# Run the TestConfiguration cmdlet in the installer folder. I am seeing the following:
#
# Trace folder already exists. Traces will be written to existing trace folder: C:\Users\Administrator\Downloads\Microsoft.Azure.ServiceFabric.WindowsServer.6.4.654.9590\DeploymentTraces
# Running Best Practices Analyzer...
# Running Best Practices Upgrade Analyzer. Please make sure you are validating against the current cluster json config in your cluster. Cluster configuration in the system may change after you add or remove node which means the original cluster json config can't represent the cluster state anymore. You can get the latest configuration by Get-ServiceFabricClusterConfiguration in Powershell.
# Best Practices Analyzer completed successfully.
# LocalAdminPrivilege        : True
# IsJsonValid                : True
# IsCabValid                 :
# RequiredPortsOpen          : True
# RemoteRegistryAvailable    : True
# FirewallAvailable          : True
# RpcCheckPassed             : True
# NoDomainController         : True
# NoConflictingInstallations : True
# FabricInstallable          :
# DataDrivesAvailable        : True
# DrivesEnoughAvailableSpace : True
# IsAllOrNoneIOTDevice       : True
# DotNetExeInPath            : True
# Passed                     : True
#
.\TestConfiguration.ps1 -ClusterConfigFilePath $newConfig -OldClusterConfigFilePath $previousConfig -Verbose

Start-ServiceFabricClusterConfigurationUpgrade -ClusterConfigPath $newConfig

The result of this script is the following exception:
Start-ServiceFabricClusterConfigurationUpgrade : System.Runtime.InteropServices.COMException (-2147017627)
ValidationException: Authentication type cannot be changed from unsecured to Windows.
At line:1 char:1
+ Start-ServiceFabricClusterConfigurationUpgrade -ClusterConfigPath $ne ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (Microsoft.Servi...usterConnection:ClusterConnection) [Start-ServiceFa
   ...gurationUpgrade], FabricException
    + FullyQualifiedErrorId : StartClusterConfigurationUpgradeErrorId,Microsoft.ServiceFabric.Powershell.StartClusterC
   onfigurationUpgrade

At no point is my cluster unsecured, and the new config is not set to use Windows authentication, so I'm assuming this is a bug. Any help would be appreciated.

I also receive the above mentioned error in an intermittent way, sometimes I get it and sometimes I dont, below steps to reproduce it:
Connect to cluster securely using X509 Certificate
$connectArgs = @{  ConnectionEndpoint = 'XXXXXXXXXXXXXXXXXX:19000';                      X509Credential = $True;                      StoreLocation = "CurrentUser";                      StoreName = "My";                      ServerCommonName = "XXXXXXXXXXXXXXXXXX";                      FindType = 'FindByThumbprint';                      FindValue = "XXXXXXXXXXXXXXXXXX";                     ServerCertThumbprint = "XXXXXXXXXXXXXXXXXX" } Connect-ServiceFabricCluster @connectArgs
Get current cluster configuration, modify needed lines and make sure you modify config file version
Get-ServiceFabricClusterConfiguration -UseApiVersion -ApiVersion 10-2017 > ClusterConfigNew.json
Test Modified file versus old one
Test-ServiceFabricConfiguration  -FabricRuntimePackagePath '.\MicrosoftAzureServiceFabric.6.4.644.9590.cab' -ClusterConfigurationFilePath '.\ClusterConfigNew.json' -OldClusterConfigurationFilePath '.\ClusterConfigOld.json'

Running Best Practices Analyzer...
Running Best Practices Upgrade Analyze
your cluster. Cluster configuration in
er json config can't represent the clu
erConfiguration in Powershell.
Best Practices Analyzer completed succ

LocalAdminPrivilege        : True
IsJsonValid                : True
IsCabValid                 : True
RequiredPortsOpen          : True
RemoteRegistryAvailable    : True
FirewallAvailable          : True
RpcCheckPassed             : True
NoDomainController         : True
NoConflictingInstallations : True
FabricInstallable          :
DataDrivesAvailable        : True
DrivesEnoughAvailableSpace : True
IsAllOrNoneIOTDevice       : True
DotNetExeInPath            : True
Passed                     : True
Start-ServiceFabricClusterConfigurationUpgrade -ClusterConfigPath .\ClusterConfigNew.json

Start-ServiceFabricClusterConfigurationUpgrade : System.Runtime.InteropServices.COMException (-2147017627)
ValidationException: A configuration upgrade with identical configuration version is not allowed.
At line:1 char:1

Start-ServiceFabricClusterConfigurationUpgrade -ClusterConfigPath .\C ...

  + CategoryInfo          : InvalidOperation: (Microsoft.Servi...usterConnection:ClusterConnection) [Start-ServiceFa
 ...gurationUpgrade], FabricException
  + FullyQualifiedErrorId : StartClusterConfigurationUpgradeErrorId,Microsoft.ServiceFabric.Powershell.StartClusterC
 onfigurationUpgrade

I am also experiencing this with the same repro steps as @ojaimi . In my case, the only elements changed in  my cluster config were ClusterConfigurationVersion and FabricClusterAutoupgradeEnabled (from true to false). I don't feel comfortable sharing my full clusterConfig, but if there's a section I can provide let me know.

I updated the title to hopefully reflect the severity of this issue. We are completely unable to upgrade the cluster configuration due to this issue.

This is utter insanity that I can't upgrade my on-premise Service Fabric cluster that is using X509 authentication, and the issue hasn't even been commented on by anyone in over 3 months. How am I supposed to use Service Fabric in production if I can't perform basic cluster configuration upgrades when the time comes? I've also posted the issue here, thinking I posted in the wrong location, but still, no response!

I will close this for now because its clear that this repository is Linux focused (for now), and keep the conversation going here, but my hopes are pretty low since nobody has even acknowledged the issue on the Microsoft side of things.

I've re-opened this issue as we are still seeing this issue on version 7.0.464.9590 of the Service Fabric runtime. The root of the issue has to do with Get-ServiceFabricClusterConfiguration returning a superfluous element that, when supplied to Start-ServiceFabricClusterConfigurationUpgrade, cause the exception:

Authentication type cannot be changed from unsecured to Windows.

The superfluous element in question is called WindowsIdentities and can be seen in the following snippet (trimmed down for readability):
{
  "Properties": {
    "GoalStateExpirationReminderInDays": 0,
    "Security": {
      "$id": "1",
      "CertificateInformation": {
        "$id": "2",
        "ClusterCertificate": ...
        "ServerCertificate": ...
        "ReverseProxyCertificate": ...
        "ClientCertificateThumbprints": ...
      },
      "WindowsIdentities": {
        "$id": "3"
      },
      "ClusterCredentialType": "X509",
      "ServerCredentialType": "X509"
    },
    ...
    ...
}

As you can see, our desire is to have a pure X509 secured cluster with no Windows Identities, yet this element is being returned (empty no less). The existence of this element causes Start-ServiceFabricClusterConfigurationUpgrade to fail.
The workaround is to remove the WindowsIdentities prior to upgrading. This is troublesome since we are fond of automating a lot of our upgrade workflows, so we will have to build this workaround into our automation tooling.

As mentioned in a few other thread on the same issue including https://github.com/microsoft/service-fabric-issues/issues/1512, the issue has been fixed as of 7.0 CU4 (7.0.470.9590).


Fixup package.xml generation

Output name should be package.xml, not package.xml.template.

Fixed in #1


[Feature][Log]Add timezone information in log output

Purpose of the pull request
Added timezone pattern for logback and log4j date string.
This PR will close #9822
Brief change log

dolphinscheduler-alert/dolphinscheduler-alert-server/src/main/resources/logback-spring.xml
dolphinscheduler-api/src/main/resources/logback-spring.xml
dolphinscheduler-data-quality/src/main/resources/log4j.properties
dolphinscheduler-master/src/main/resources/logback-spring.xml
dolphinscheduler-standalone-server/src/main/resources/logback-spring.xml
dolphinscheduler-worker/src/main/resources/logback-spring.xml

Verify this pull request
This pull request is code cleanup without any test coverage.

Codecov Report

Merging #9913 (756ab4f) into dev (15a15de) will decrease coverage by 0.00%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##                dev    #9913      +/-   ##
============================================
- Coverage     40.32%   40.31%   -0.01%
+ Complexity     4532     4531       -1
============================================
  Files           835      835
  Lines         33747    33747
  Branches       3726     3726
============================================
- Hits          13608    13606       -2
- Misses        18852    18853       +1
- Partials       1287     1288       +1

Impacted Files
Coverage Δ

...dolphinscheduler/remote/future/ResponseFuture.java
81.96% <0.00%> (-1.64%)
:arrow_down:

...r/plugin/task/sqoop/parameter/SqoopParameters.java
53.33% <0.00%> (-1.34%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 15a15de...756ab4f. Read the comment docs.


Issue - Voice sentiment analysis

Could you rename the directory to voice-sentiment-analysis

Thanks a lot for contributing