Interpolation of polygons

I understand that by enabling dragging, I can move a polygon around by shifting it in X and Y directions. Similarly, is there a feature in CVAT that would also allow me to rotate the polygon?

@fywu85 , it is not possible. What is the reason for the request? Could you please describe your use case?

Say if I want to label an object in a video that rotates only in the image plane from frame to frame, e.g., a vehicle making a left turn, being able to rotate the object's polygon would be huge time saver. For example, one of my use cases is as follows:

Without rotation, I will need to readjust every point in the polygon in the previous frame to make it refit to the left turning vehicles in the current frame. With rotation, this operation will be almost instant.

@fywu85 , internally we have a prototype for interpolation of polygons between frames (as for bounding boxes). In the case it will work perfect. But it will take time to release it. Let me rename the issue. I'm not sure that the feature will be available in next release.

Thank you for the info. That sounds great! In your knowledge, what would be the current best approach to label this? If I do not care about the precision of the boundary of the object, can I rotate or interpolate a bounding box to get a reasonable location label of a turning vehicle? Or, can I potentially use the semi-automatic segmentation feature here?

@fywu85 , to do that efficiently need to wait interpolation of polygons. You can find technical resources on your side and we will give you ideas and a sample of code how it can be implemented inside CVAT.

@nmanovic Thank you! That will be really helpful. Would you mind sharing with us your ideas on how to approach implementing this interpolation + rotation feature? If we succeed, we will be happy to contribute a PR (:

@fywu85
Hi, as Nikita said, we have a prototype for interpolation of a poly shapes (not only polygons, by the way).
It can be found in cvat-core module which is expected to be a part of the tool in the nearest future (current production version CVAT mostly doesn't work with this module).

Our JavaScript client-side implementation can be found here
It is based on stable marriage problem algorithm which is used to finding a matching of points between two shapes.
It doesn't work perfectly yet, but we suppose after some testing and adjustment it would be much better.
One more comment is that algorithm isn't implemented on the server part at all, so it is also need to write the same code in Python. But it isn't expected to be the most difficult thing here.

Glad to see your comments and ideas about how it can be improved.

I'm also very interested in this feature.
Thanks for your work on this soft !

Hello! We are also really interested in this feature. Is there a rough estimation when this functionality will be available?
Best,
Ekin

We will try hard to implement it in this month.
But there aren't guarantees


New Project - students list

@eden-amar this file should have the basic HTML structure (head, body..).

Ok
I did “touch” to the file so I didn’t open the VS.
I’m on it.
בתאריך יום ב׳, 23 בינו׳ 2023 ב-22:06 מאת David Meir-Levy <
@.***>:

@eden-amar https://github.com/eden-amar this file should have the basic
HTML structure (head, body..).
—
Reply to this email directly, view it on GitHub
https://github.com/d-stuff/netcraft-11-2022/pull/16#issuecomment-1400908381,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A3SHXME3TTOLJ2CDMLJ2AW3WT3QEXANCNFSM6AAAAAAUEHNYYY
.
You are receiving this because you were mentioned.Message ID:
@.***>

@eden-amar this file should have the basic HTML structure (head, body..).
@davidmeirlevy  I fixed it

Make sure you choose "squash and merge" for this PR. ✌


Clipboard Plugin shows Unmet Dep

Not sure when this started.

Obviously this means it no longer functions, its objects do not appear, etc.

I guess it also means this entire page is wrong/useless: https://qsapp.com/manual/Clipboard and Shelf/

The clipboard plug-in requires the Shelf plug-in. I suspect you have that disabled. Quicksilver will download and install plug-ins to meet such dependencies, but it will not forcibly re-enable something you’ve disabled.
The manual could be wrong. I’m working my way through updating the entire thing, but haven’t gotten that far. Feel free to update that page and submit a pull request.

As stated by Skurfer, the Clipboard plugin requires the 'Shelf' plugin be enabled. However I agree, the error message could be more helpful. I have made this more visible in the Quicksilver interface now:


[add-resource] New Theme: Casper2

I would like to add my new theme to the Grav Repository.
Here are the project details: https://github.com/diomed/casper2

Added to GPM


FormatException

Hi guys, any idea about this issue:
FormatException: Unexpected character (at character 1)
Too many requests
I dont know if is an API timeout, if it is which will be the better solution?

Seems like you hit a quota and got rate limited. But maybe we need to think about some library level handling of such a situation.
What it means for you right now is, that you need to wait until you can make a request again. In the Developer Dashboard you should be able to see your current volume.

@addreeh You could show the user a message, that the app is currently not usable and that he/she should try it again later. I suggest to check the code for unnecessary requests (sometimes they are done in some for-loops) and look for alternatives (e.g. instead of requesting all albums for each artist, you could collect each albumId for each artist and do a getSeveralAlbums request). That way you would reduce requests.

while (true) {
      try {
        var tracks = await spotifyApi.playlists
            .getTracksByPlaylistId(playlistObject.id)
            .getPage(limit, offset);
...
}

So one thing, that looks suspicious is that you fetch all playlist items. If - let's say - a playlist has 300 tracks, but only - let's say - 15 are displayed, then it is indeed a waste of requests. As far as I understand your code, you want to display tracks from a playlist in a list (?). You could reduce requests by only GETting one Page at a time. This could be done with ListViews that allow to fetch more data once the end of the list is reached. Unfortunately I forgot the proper name of those lists, but with a quich search you would be able to find it.

_getFullArtist() async {
    List<spotify.Artist> artists = [];
    for (var artist in playlistArtists) {
      try {
        var newArtist = await spotifyApi.artists.get(artist.id);
        artists.add(newArtist);
      } catch (e) { ... }
    return artists;
  }

You could use spotify.artists.list() here instead of requesting each artist individually. So in your case it would look like this:

_getFullArtist() async {
    // converting list of artists into list of artist id's
    var artistIds = playlistArtists.map((artist) => artist.id);
    // bulk retrieve artist information saving you the for-loop
    return await spotifyApi.artists.list(artistIds);
  }

Hope it helps :)

Seems like you hit a quota and got rate limited. But maybe we need to think about some library level handling of such a situation.

Maybe tracking the number of requests?

@addreeh You can check out the wiki-page, where all implemented endpoints are listed.

I tried your app out, and retrieving each track and having a statistic like that, is also the only way I would come up with. But I think, you could optimize some of your for loops, for example
int offset = 0;
    int limit = 100;

    while (true) {
      try {
        var tracks = await spotifyApi.playlists
            .getTracksByPlaylistId(playlistObject?.id ?? '')
            .getPage(limit, offset);
        if (tracks.items != null && tracks.items!.isNotEmpty) {
          try {
            songs.addAll(tracks!.items); //<-- just use addAll instead of another for loop.
            offset += limit;
          } catch (e) {
            offset = songs.length;
          }

And you also could play around with the dart collection api. It has some methods that could make you remove some of your for loops.

@addreeh can this issue be closed?


Restrict backport action to vX.X.x branches

Restrict backport github action to only trigger on backports to X.X.x branches i.e. labels:
backport 5.4.x or backport 5.3.x this conforms to the new release policy at Sourcegraph: https://www.notion.so/sourcegraph/Release-Process-7fb8aff26cbe4c3786ca25396b4986d0

If this is merged I'll update: https://www.notion.so/sourcegraph/Sourcegraph-backporting-tool-211d005248bb4b588b4fe8401381e71c#858adef279914a1d8ba4d809980419d9
and add instructions and a link in: https://www.notion.so/sourcegraph/Release-Process-7fb8aff26cbe4c3786ca25396b4986d0


Many ng-scroll on the page with different datasources

Hello,
Thanks for a great package.
I'm struggling with making many ng-scoll items on the page, they are actually in ng-repeat and have different datasources
Datasources are the problem, how to make dynamic datasets and wrap in datasources? How can I pass params from angular view from rendering to the datasource to provide data?

so I need something like this - https://plnkr.co/edit/JjAiw3zvG4uIWGNjLUU7?preview
but with dynamic datasource for each ng-scroll

@villian If you need 2 different Datasources for two different views, you may just define 2 ds objects:

...

$scope.datasource = { get: /* implementation of ds1 */ };
$scope.datasource2 = { get: /* implementation of ds2 */  };

But if you need to have multiple datasources under a single umbrella, the following approach might be useful. Basically, if you can guarantee indexes consistency across the datasources, you may provide the replacement of the datasource.get method body in runtime:
let myDynamicDatasource = (i, c, s) => s([]); // stub ds.get

$scope.datasource = {
  get: (index, count, success) =>
    myDynamicDatasource(index, count, success)
};

// data is new array of items we want to loop through with ui-scroll
$scope.onDatasourceChanged = (data) =>
  myDynamicDatasource = (index, count, success) =>
    success(data.slice(index, index + count));

But if each ds has its own indexes flow not correlating with others, then I would advise to clean up the internal state via Adapter.reload API:
$scope.onDatasourceChanged = (data, indexToReload) => {
  myDynamicDatasource = (index, count, success) =>
    success(data.slice(index, index + count));
  $scope.adapter.reload(indexToReload);
}

Thanks, for the response! This is what we have:

I have to pass productId into datasource somehow and datasource should be unique for
do we need an array with all items and then manage it via indexes? any idea?

@villian Well I think I got the issue. Basically, datasource is an object with "get" field which should exist on $scope during render, and if you know the products array length, you may arrange appropriate number of datasources pushed in a single map-object with keys corresponded to the list of product ids:

Below is the draft implementation of the datasource-product map.
const getDataByProduct = (product, index, count, callback) => {
  // the logic of this method might be dynamic/async
  // anyway here you have all the data to provide appropriate potion of data
  const data = $scope.items[product.id] || [];
  callback(data.slice(index, index + count));
};

const createDatasource = (product) => ({
  get: (index, count, success) =>
    getDataByProduct(product, index, count, success)
});

$scope.datasources = $scope.products.reduce((acc, product) => ({
  ...acc,
  [product.id]: createDatasource(product)
}), {});


Correct typo: namr → name

I see that doc/luasnip.txt is autogenerated from DOC.md (though I'm not sure how), so I didn't patch that file.

TY!


Update README.md (add link to "Dive Into Deep Learning")

Added a link to the ebook and a short description of it.

Thanks! I’ll address the Travis issue (happens often), then I’ll merge it

No problem! Glad I could help!


High severity vulnerability of dot-prop package

Getting this after npm install and npm audit:

More info link from screenshot -  https://npmjs.com/advisories/1213. Something similiar is happened to others recently - https://github.com/gatsbyjs/gatsby/issues/26128. It vould be really cool to try this package, so hope you guys can fix that problem

There's already a fix in #73. Please merge?

dependencies updated


L.Control.Layers doesn't work with dark theme from MapBox

Leaflet's Layers Control doesn't work with MapBox's dark theme.
Control's background color should be dark and the icon white but the background color stays white.
Browser: Google Chrome 33.0.1750.154 m
OS: Windows 7 Professional Service Pack 1
Leaflet: 0.7.2
Demo: http://jsfiddle.net/cebence/53nJr/ (based on https://www.mapbox.com/mapbox.js/example/v1.0.0/layers-control/)
One part of the issue is in Leaflet, the other in MapBox.
Leaflet issue: https://github.com/Leaflet/Leaflet/issues/2554
MapBox issue: https://github.com/mapbox/mapbox.js/issues/716
Example:

Seems fixed by https://github.com/Leaflet/Leaflet/commit/f51f783b3a480a9e243dbf910783e59a5bf46730 to me.


How to use transfer learning using 5k iteration model?

I have limited GPU resources and I can not train the model for so long.
I kept it for one whole day for training and I was only able to reach to 300 something iterations(in between 2nd epoch).
Can anyone please tell me how much time it will take to train the complete model and is it possible to do transfer learning using 5k iteration model?
Also if anyone is able to train the complete model and got the good results, can you please share the model? I kind off require it for my academic project.
Thanks in advance :)

@Nishanksingla It depends on what your goals are. Speech recognition using DNNs is very much a research area that is compute and data limited.
The state of Speech A.I. is not like that of Image A.I. If you want a tiny word error rate (like <5%) on a wide range of audio (phone calls, radio shows, youtube vids, recorded lectures), then that doesn't exist yet. If you want to have some fun and see the tech working pretty well (but not perfectly) and be part of the progress as things get better, then you can do that with even limited resources. The best way it by contributing to open source deep learning projects like this one. There still is a lot of work to do.

Hi Scott
Thank you for the reply.
My goals are simple, just want to train this model on LibriSpeech dataset to see how well it can perform on my recorded voice.
I am interested in contributing to this project. Please let me know how can I contribute and from where I can get the information about it

Which GPU do you have?
P.S. I'm not at Baidu so my advice is just coming from a fellow person interested in Speech A.I. :)


Don't display completed-but-repeatable quests as "Completed"

(Required) What is your Quests version / build number (do not say "latest")?
b169
(Required) Please describe the problem and how to replicate it. Any console errors?
We've had a previous discussion about this. Currently, a completed repeatable quest will result in no particle effects above NPCs and a listing as "Completed" on the NPC's quest display, with no indication that the quest is repeatable.
Desired behavior is a configuration option (in main plugin config or individual quest options) to display the quest as-if-new or otherwise indicate that it is repeatable both in chat options and with particle effects so that players do not inadvertently miss out on available repeatable quests.
(Optional) Can you provide an example quest from quests.yml using Pastebin.com?
  Daily Fishing:
    name: Demonstrate your fishing skills (Daily)
    redo-delay: 43200
    npc-giver-id: 11
    requirements:
      quests:
      - Teach the newbie to fish
      fail-requirement-message: '<gray>Fishing Newbie<white>: You need to teach me
        how to fish first!'
    ask-message: '<gray>Fishing Newbie<white>: Could you show me a little more of
      your fishing skills?'
    finish-message: '<gray>Fishing Newbie<white>: Awesome! Maybe someday I''ll be
      as good as you are!'
    stages:
      ordered:
        '1':
          fish-to-catch: 16
    rewards:
      money: 50
      quest-points: 1

This was brought up once before in the Blackvein repo (can't find the original). As I recall, I came to the conclusion that the "Completed" message is to indicate that the quest has been completed at least once. I do not recall if there were plans to change this.
However, I agree that some action should be taken to visually inform users that a quest is redo-able. Your config option is one idea, but I'd like leave this open for a while pending further ideas.

As long as a repeatable-but-completed quest is visually distinct from a not-repeatable-and-completed quest, that's fine with me. I think the VFX should default to enabled for available repeatable quests, or there should at least be an option.

Maybe its an option to only show it visually once to indicate that the user has indicated it alteast once.

I've given it some thought and agree that @SXRWahrheit 's proposal is best, which boils down to:

Add a config option to show particle effects for redoable quests
Display a sort of "(Redoable)" tag in, at least, the NPC GUI

How to execute these is a bit tricky. Obviously, not every admin is going to want a NPC effect for quests that are redoable (in other words, they're happy with the current setup). That said, we can't just disable the show-npc-effects option since that would disable effects for new quests, too. We could add a separate show-redoable-npc-effects, but then admins might enable that while disabling the base show-npc-effects, which may cause confusion for players across servers.
All that said, I think it's time to redesign config.yml. We're already saving the config in a manner which allows comments, so I'm thinking something like this. Please check out line 36 and let me know what y'all think.
For the "(Redoable)" tag... it should be noted that the "(Completed)" message currently only shows for quests which are given by a NPC whom does not have a GUI display. It does not show in GUIs or when considering quests through /quests list (although I guess showing that in the latter might defeat the purpose of /quests stats). So we ought to consider the implications of adding the tags elsewhere if it might make one of those commands rather useless. Thoughts?

Line 36 etc. looks great.
I'm not familiar with the GUI display that you're talking about. Right now, players right click on an NPC and get a list of quests. Is there an alternative?
I put daily/weekly/etc in the names of all of the quests on my server, so I don't really need the addition of a (redoable) tag - if anything I would like the option to just exclude the (Completed) tag on redoable quests. Adding a redoable tag would be fine, and it's always possible to change the display functionality if someone comes up with a better idea, of course.

@SXRWahrheit Yes, check out option 9 when creating/editing a quest :)
It sounds like the plan is to replace (Completed) with (Redoable) when applicable. Still gotta determine the fate of /quests list vs /quests stats, but I'll add "help wanted" until I start work on this myself.

The word "redoable" itself feels weird, by the way. Any suggested replacements would be welcomed.

Agreed. That word was used internally before I started work on the plugin. How about "repeatable"?

Repeatable is much better.

Updated config per discussion. The new options don't do anything in code yet. I was wrong about being able to add comments to it, so I merely added a header to direct users to the Wiki.

A new "repeatable" string has been added to the language file and will show at the quest accept prompt for NPCs without a GUI. To clarify, the "(Completed)" tag shown in the image below will now display "(Repeatable)" instead.

I have not deleted the "completed" string from the lang file as it is still used the moment a user completes a quest in-game.

That sounds great!

@SXRWahrheit If a NPC has both repeatable AND uncompleted quests, which particle effect should show?

Priority goes to uncompleted, IMO.

I plan to use the same particle for both (maybe that'll change in the future), but I imagine "you haven't done this yet" is more important than "you can do this again."

The repeat effect has been implemented and priority goes to uncompleted quests ("new-effect" in config). I changed the repeat-effect in config from "witch" to "angry_villager", made it so the capitalization of effect names no longer matters, and cut down on error spam if an invalid effect is used in config.
The only things left are to add the "(Repeatable)" tag to a few places where there is no tag at all, and to determine the fate of /quests list vs _/quests stats.

In reference to changing this setup^, I've given it more thought and am now planing to revert replacing (Completed) with (Repeatable). As I mentioned in my first reply to this issue, the mere presence of a tag means it has been completed at least once, which I feel "Completed" makes clear. While the fact that the quest appears in the quest prompt means it's repeatable, the word "repeatable" doesn't expressely imply to unfamiliar players that they've done the quest at least once already.
On the flip side, @SXRWahrheit has made a good point that having completed quests grouped in the same list as uncompleted quests isn't visually distinct, in which case "Repeatable" makes more sense. Since SXR also mentioned, "I would like the option to just exclude the (Completed) tag" my new plan is to simply keep the tag configurable in the language file, so that it can be changed from "(Completed)" to "(Repeatable)", or removed entirely, per admin's preference.
Sorry for the wall of text but I felt it best to explain why I'm changing "Repeatable" back to "Completed" now rather than later.

Fixed a bug where repeatable quests would be listed multiple times in /quests stats. Also completed unfinished Blackvein code which tracks the number of times the quests have been taken and shows that with a trailing number. That said, I've decided to leave stats and list as separate commands for now, although they could certainly be combined one day.
With that, I believe all tasks for this issue have been completed. Thanks!


IllegalStateException "Required class information is missing" when indexing a local class within an inner class

Recently I ran into a problem where I could not start a Junit based integration test including Hibernate where the Jandex library threw this exception:
Exception in thread "main" java.lang.IllegalStateException: Required class information is missing
	at org.jboss.jandex.Indexer.searchNestedType(Indexer.java:1255)
	at org.jboss.jandex.Indexer.searchTypePath(Indexer.java:1169)
	at org.jboss.jandex.Indexer.updateTypeTarget(Indexer.java:1139)
	at org.jboss.jandex.Indexer.updateTypeTargets(Indexer.java:849)
	at org.jboss.jandex.Indexer.index(Indexer.java:1980)

After some debugging and testing I assume that this seems to be related with local classes within an inner class having an annotated field with the type of another local class. The annotation has to have the TYPE_USE target.
I can reproduce the problem with following code (using jandex 2.4.1.Final):
import static java.lang.annotation.ElementType.FIELD;
import static java.lang.annotation.ElementType.TYPE_USE;
import static java.lang.annotation.RetentionPolicy.RUNTIME;

import java.io.IOException;
import java.io.InputStream;
import java.lang.annotation.Documented;
import java.lang.annotation.Retention;
import java.lang.annotation.Target;

import org.jboss.jandex.Indexer;

public class ReproduceMissingClassInfo {

    public static void main(String[] args) throws IOException {

        dumpSystemProperties();

        //reproducing code
        InputStream input = ReproduceMissingClassInfo.class.getResourceAsStream("/ReproduceMissingClassInfo$InnerClass$1LocalClass.class");

        new Indexer().index(input);
    }

    class InnerClass {

        void doSomeThing() {

            class LocalNestedClass {
            }

            class LocalClass {

                @SomeAnnotation
                private LocalNestedClass nested;

            }
        }
    }

    @Target({FIELD, TYPE_USE})
    @Retention(RUNTIME)
    @Documented
    @interface SomeAnnotation {

    }

    private static void dumpSystemProperties() {

        System.out.println("SYSTEM PROPERTIES:\n");
        System.getProperties().entrySet().stream()
                .map(entry -> String.format("%s = %s", entry.getKey(), entry.getValue()).replaceAll(System.getProperty("user.name"), "xxxx").replaceAll("xxxx/.*target", "xxxx/.../target"))
                .forEach(System.out::println);
        System.out.println();
    }
}

On my environment running this produces following output:
SYSTEM PROPERTIES:

sun.desktop = windows
awt.toolkit = sun.awt.windows.WToolkit
java.specification.version = 11
sun.cpu.isalist = amd64
sun.jnu.encoding = Cp1252
java.class.path = E:\devel\workspace-idea\reproduce-jandex-exception\target\classes;C:\Users\xxxx\.m2\repository\org\jboss\jandex\2.4.1.Final\jandex-2.4.1.Final.jar
java.vm.vendor = AdoptOpenJDK
sun.arch.data.model = 64
user.variant =
java.vendor.url = https://adoptopenjdk.net/
user.timezone =
os.name = Windows 10
java.vm.specification.version = 11
sun.java.launcher = SUN_STANDARD
user.country = DE
sun.boot.library.path = C:\Program Files\AdoptOpenJDK\jdk-<IP_ADDRESS>-hotspot\bin
sun.java.command = ReproduceMissingClassInfo
jdk.debug = release
sun.cpu.endian = little
user.home = C:\Users\xxxx
user.language = de
java.specification.vendor = Oracle Corporation
java.version.date = 2019-04-16
java.home = C:\Program Files\AdoptOpenJDK\jdk-<IP_ADDRESS>-hotspot
file.separator = \
java.vm.compressedOopsMode = Zero based
line.separator =

java.specification.name = Java Platform API Specification
java.vm.specification.vendor = Oracle Corporation
java.awt.graphicsenv = sun.awt.Win32GraphicsEnvironment
user.script =
sun.management.compiler = HotSpot 64-Bit Tiered Compilers
java.runtime.version = 11.0.3+7
user.name = xxxx
path.separator = ;
os.version = 10.0
java.runtime.name = OpenJDK Runtime Environment
file.encoding = UTF-8
java.vm.name = OpenJDK 64-Bit Server VM
java.vendor.version = AdoptOpenJDK
java.vendor.url.bug = https://github.com/AdoptOpenJDK/openjdk-build/issues
java.io.tmpdir = C:\Users\MARCE_~1\AppData\Local\Temp\
java.version = 11.0.3
user.dir = E:\devel\workspace-idea\reproduce-jandex-exception
os.arch = amd64
java.vm.specification.name = Java Virtual Machine Specification
java.awt.printerjob = sun.awt.windows.WPrinterJob
sun.os.patch.level =
java.library.path = C:\Program Files\AdoptOpenJDK\jdk-<IP_ADDRESS>-hotspot\bin;C:\WINDOWS\Sun\Java\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\Program Files\AdoptOpenJDK\jdk-<IP_ADDRESS>-hotspot\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\ATI Technologies\ATI.ACE\Core-Static;C:\Program Files (x86)\AMD\ATI Technologies\ATI.ACE\Core-Static;C:\Program Files\OpenVPN\bin;C:\Program Files (x86)\GtkSharp\2.12\bin;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;C:\Program Files\Git\cmd;C:\Program Files\Go\bin;C:\Users\xxxx\scoop\apps\nodejs\current\bin;C:\Users\xxxx\scoop\apps\nodejs\current;C:\Users\xxxx\scoop\shims;C:\Program Files (x86)\Git\bin;E:\Tools\gource-0.39.win32;E:\Tools\apache-maven-3.0.4\bin;C:\Users\xxxx\AppData\Local\Microsoft\WindowsApps;E:\Tools\micronaut\micronaut-1.2.1\bin;E:\Tools\gradle\gradle-5.6.2\bin;C:\Users\xxxx\AppData\Local\atom\bin;C:\ProgramData\xxxx\atom\bin;C:\Users\xxxx\AppData\Local\Programs\Microsoft VS Code\bin;C:\Program Files (x86)\FAHClient;C:\Users\xxxx\AppData\Local\Microsoft\WindowsApps;C:\Users\xxxx\go\bin;.
java.vendor = AdoptOpenJDK
java.vm.info = mixed mode
java.vm.version = 11.0.3+7
sun.io.unicode.encoding = UnicodeLittle
java.class.version = 55.0

Exception in thread "main" java.lang.IllegalStateException: Required class information is missing
	at org.jboss.jandex.Indexer.searchNestedType(Indexer.java:1255)
	at org.jboss.jandex.Indexer.searchTypePath(Indexer.java:1169)
	at org.jboss.jandex.Indexer.updateTypeTarget(Indexer.java:1139)
	at org.jboss.jandex.Indexer.updateTypeTargets(Indexer.java:849)
	at org.jboss.jandex.Indexer.index(Indexer.java:1980)
	at ReproduceMissingClassInfo.main(ReproduceMissingClassInfo.java:23)

Process finished with exit code 1

The same issue also occured with OpenJDK 17 (compile and runtime).

Looking at the relevant code in Jandex, I found #92 / #111 and I think this might be a similar case.
I changed the reproducer to this:
public class Reproducer {
    @Target(ElementType.TYPE_USE)
    @Retention(RetentionPolicy.RUNTIME)
    @interface SomeAnnotation {
    }

    class InnerClass {
        class InnerInnerClass {}

        Object doSomething() {
            class LocalClass {}

            class AnotherLocalClass {
                @SomeAnnotation
                LocalClass local;

                @SomeAnnotation
                InnerInnerClass inner;
            }

            return new AnotherLocalClass();
        }
    }

    public static void main(String[] args) throws IOException {
        Index.of(new Reproducer().new InnerClass().doSomething().getClass());
    }
}

When I compile this class using javac from OpenJDK 17.0.1 and then decompile the Reproducer\$InnerClass\$1AnotherLocalClass.class file using javap -v, I get the following output for the 2 fields:
  org.jboss.jandex.test.Reproducer$InnerClass$1LocalClass local;
    descriptor: Lorg/jboss/jandex/test/Reproducer$InnerClass$1LocalClass;
    flags: (0x0000)
    RuntimeVisibleTypeAnnotations:
      0: #16(): FIELD, location=[INNER_TYPE, INNER_TYPE]
        org.jboss.jandex.test.Reproducer$SomeAnnotation

  org.jboss.jandex.test.Reproducer$InnerClass$InnerInnerClass inner;
    descriptor: Lorg/jboss/jandex/test/Reproducer$InnerClass$InnerInnerClass;
    flags: (0x0000)
    RuntimeVisibleTypeAnnotations:
      0: #16(): FIELD, location=[INNER_TYPE, INNER_TYPE]
        org.jboss.jandex.test.Reproducer$SomeAnnotation

This kinda sorta makes sense for the inner field, whose type is actually Reproducer.InnerClass.InnerInnerClass (and each of those type usages may be annotated separately). I think it doesn't make sense for the local field, because you can't really write its type as Reproducer.InnerClass.EmptyLocalClass or anything like that. So I'd guess that the type annotation's type path should be empty.
The JVM specification, chapter 4.7.6. The InnerClasses Attribute, says in the description of outer_class_info_index:

If C is not a member of a class or an interface - that is, if C is a top-level class or interface (JLS §7.6) or a local class (JLS §14.3) or an anonymous class (JLS §15.9.5) - then the value of the outer_class_info_index item must be zero.

The JLS, chapter 14.3. Local Class and Interface Declarations, says:

Like an anonymous class (§15.9.5), a local class or interface is not a member of any package, class, or interface (§7.1, §8.5).

Further, the JLS, chapter 8. Classes, says:

A nested class is any class whose declaration occurs within the body of another class or interface declaration. A nested class may be a member class (§8.5, §9.5), a local class (§14.3), or an anonymous class (§15.9.5).

This all leads me to believe that my assessment above is correct: LocalClass is not a member class and type of the local field can't be written like as Reproducer.InnerClass.EmptyLocalClass or anything similar.
Need to dig more into the specifications though, these are just preliminary results.

The JVM specification, chapter 4.7.6. The InnerClasses Attribute, says in the description of outer_class_info_index:

If C is not a member of a class or an interface - that is, if C is a top-level class or interface (JLS §7.6) or a local class (JLS §14.3) or an anonymous class (JLS §15.9.5) - then the value of the outer_class_info_index item must be zero.

Keep in mind that in the anon or local case you also have the enclosing method, which we do analyze. I haven't looked at the code, but it could just be we aren't tracking non-inner types for this search. I might have either misinterpreted this being allowed, or it could just be an oversight. One thing that is confusing is the names that javap prints aren't necessarily reflective of the true definition (e.g.  It's not really INNER but "Nested" in terms of the spec). I have also ecountered invalid data in this tabl (e.g. type annotations with bridge methods) e, so its good to double check the correctness aspect and potentially ignore.

Just as an experiment, I tried to compile the class with ECJ 4.22 and it emits the same type path. I'll continue digging.

Ah and using the workaround from #111 for this situation seems to produce expected results. So that's a possible way to go.

I spent some time digging around (mostly in JLS and JVMS, but I also looked at the original JSR 308 specification) and I think my original assessment is wrong. Assuming both specifications are absolutely precise, that is.
JLS, chapter 8. Classes, says:

A nested class may be a member class (§8.5, §9.5), a local class (§14.3), or an anonymous class (§15.9.5).

That is, a local class is still a nested class (even though it is not a member class).
(Aside: funnily enough, per chapter 8.1.3. Inner Classes and Enclosing Instances, local classes are also inner classes. That doesn't matter here, because the JVMS virtually never refers to inner types outside of the specification of the InnerClasses attribute.)
JVMS, chapter <IP_ADDRESS>. The type_path structure, says:

If a nested type T1.T2 is used in a declaration or expression, then an annotation may appear on the name of the innermost member type and any enclosing type for which a type annotation is admissible (JLS §9.7.4).

This shows that they make a difference between nested types and member types which is consistent with how the JLS differentiates between these terms.
(Aside: at the same time, it also leaves the case of non-member nested types a little undefined...)
Further, it says:

The value of the path_length item gives the number of entries in the path array:

If the value of path_length is 0, and the type being annotated is a nested type, then the annotation applies to the outermost part of the type for which a type annotation is admissible.
If the value of path_length is 0, and the type being annotated is not a nested type, then the annotation appears directly on the type itself.
If the value of path_length is non-zero, then each entry in the path array represents an iterative, left-to-right step towards the precise location of the annotation in an array type, nested type, or parameterized type.

[...]

Value
Interpretation

...
...

1
Annotation is deeper in a nested type

...
...

That is, all descriptions of the type path in JVMS use the term "nested type". While the enclosing classes of a local class can't be denoted in the local class name, and hence can't also be annotated, they are still technically there.
I'll see what Jandex can do about it.

I'm looking into this again and I found that there's one more bug: Jandex doesn't recognize a type annotation on the outer-most type in a nested type path. For example, this piece of code
public class Reproducer {
    @Target(ElementType.TYPE_USE)
    @Retention(RetentionPolicy.RUNTIME)
    public @interface TypeAnn {
        String value();
    }

    class Inner {
        class InnerInner {
            @TypeAnn("reproducer") Reproducer.@TypeAnn("inner") Inner.@TypeAnn("inner inner") InnerInner field;
        }
    }

    public static void main(String[] args) throws IOException {
        Index index = Index.of(Inner.InnerInner.class);
        ClassInfo clazz = index.getKnownClasses().iterator().next();
        System.out.println(clazz.field("field").type());
    }
}

prints
@TypeAnn(value = "inner") org.jboss.jandex.Reproducer$Inner.@TypeAnn(value = "inner inner") @TypeAnn(value = "reproducer") InnerInner

The @TypeAnn("reproducer") annotation is associated with the InnerInner type, instead of the Reproducer type.
There's a comprehensive test for type annotations that includes nested type paths like this, but interestingly, it doesn't cover the case of an annotated outer-most type.

Done in #205.


Investigate using snow lib

Try out this lib for noise instead https://github.com/mcginty/snow

yeah i misread the repo, it's not perlin noise. definitely not useful

yeah i misread the repo, it's not perlin noise. definitely not useful


Handle mapbox-gl-style-spec recently merged into mapbox-gl-js' repo

mapbox-gl-style-spec repo has recently been merged with mapbox-gl-js's one. The last update of the style-specs are in this repo, so #108 still not working.
Hope my explanations are clear and will make you find a way to correct this ;)

Yeah we need to handle this big merge..

If someone else wants to jump on this welcome!!!

I have a working branch of this  https://github.com/maputnik/editor/compare/master...orangemug:fix/issue-110-update-mapbox-style-spec I still need to verify a few things before I raise a PR.
Note this still doesn't fix #108 because we'll need to make alterations to the ColorField component to make that work correctly.

I have a working branch of this master...orangemug:fix/issue-110-update-mapbox-style-spec I still need to verify a few things before I raise a PR.
Note this still doesn't fix #108 because we'll need to make alterations to the ColorField component to make that work correctly.

I'll do this in 2 parts updating to the new style spec initially. @lukasmartinelli can you merge https://github.com/lukasmartinelli/mapbox-gl-inspect/pull/6 so I can update the current branch and make a PR

I'll do this in 2 parts updating to the new style spec initially. @lukasmartinelli can you merge lukasmartinelli/mapbox-gl-inspect#6 so I can update the current branch and make a PR

Done!

PR #124

PR #124 merged


Show how many MS the node is behind

In the old dashboard if you hovered over the LSMI/LMI metric you could see the difference. This was really helpful to see if the node is catching up.

This will be visible in the extended peer page which is not yet implemented

This will be visible in the extended peer page which is not yet implemented

Peer metrics page added https://github.com/iotaledger/node-dashboard/commit/54b73bc4a31fa0fbf418fb37c7f3d92557fc72ab

Peer metrics page added https://github.com/iotaledger/node-dashboard/commit/54b73bc4a31fa0fbf418fb37c7f3d92557fc72ab


cannot install webview_flutter, conflict with flutter_html

pubspec.yaml

flutter_html: ^2.2.1
webview_flutter: ^3.0.0

ERROR LOG

Because flutter_html 2.2.1 depends on webview_flutter ^2.0.4 and no versions of flutter_html match >2.2.1 <3.0.0, flutter_html ^2.2.1 requires webview_flutter ^2.0.4.
So, because otlink depends on both flutter_html ^2.2.1 and webview_flutter ^3.0.0, version solving failed.
pub get failed (1; So, because otlink depends on both flutter_html ^2.2.1 and webview_flutter ^3.0.0, version solving failed.)

HI @eggysudianto, Thanks for filing the issue. From the error I can understand that

flutter_html 2.2.1 depends on webview_flutter ^2.0.4
And your application depends on webview_flutter: ^3.0.0

So, Inorder to solve this conflict your application must depend on either webview_flutter ^2.0.4 or flutter_html should upgrade its webview plugin to webview_flutter: ^3.0.0.
Because flutter_html is a third-party plugin we can really track this issue here. I would suggest you to please file the issue with the maintainer.
Closing as this isn't an issue with flutter.


⚠ ClamAV has degraded performance

In aed6b94, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 203 ms

Resolved: ClamAV performance has improved in 3915738.


uv tool upgrade - problem with private indexes due to short living credentials

I am using uv tool install to install some CLI tools from private index (CodeArtifact).
When new version is released, uv tool upgrade  generally fails. I have a mechanism that refreshes CodeArtifact index URL with new credentials and stores them in ~/.config/uv/uv.toml, but when tools are installed, index used is stored in uv-receipt.toml as far as I can tell and upgrade is trying to use that again. It usually fails, because those credentials are good only for couple hours.
Workaround is uv tool uninstall <tool> && uv tool install <tool>.
Should upgrade use globally defined index if stored one fails? Maybe only if domain is the same (but ignore credentials)? Not sure what the best approach is here, but it would be good for upgrade to work.
Happy to provide any additional details if needed.
❯ uv --version
uv 0.4.26 (1b9b9d56b 2024-10-23)

Thanks for the report! I'm not sure what the best solution here is, seems tricky to know which credentials we should prefer.

Maybe do what pipx does, which is to use pip config at the moment of command invocation (I haven't read the code, but behaviour seems like that, since updating pip.conf has that effect on pipx as well).
In this case, its not even a different tool (like pip and pipx), its all uv and uv add (for example) works with values from config file while uv tool only sometimes uses same values. I am not sure what is the purpose of storing index in receipt.toml, so I am not sure if suggestion make sense at all.

I am not sure what is the purpose of storing index in receipt.toml, so I am not sure if suggestion make sense at all.

We store the index in the receipt so that if you do something like uv tool install --index-url ... a subsequent uv tool upgrade operation will use the correct index.

Ah, ok. In that case, maybe it can be stored only if one from config file was overridden via CLI flag? Not sure, but it seems wrong to store credentials there anyway, especially ones that have short lifespan.

Unfortunately it's a pain to distinguish between values from the CLI and config file (due to the way we've implemented it), but yeah I think that would make sense.

Indexes are actually the one thing where we can differentiate between CLI and elsewhere, we track the origin.

Ah great, I knew you added that for something :)

(N.B. that I didn't read the full discussion here before commenting, just your last comment.)


shared dependencies, depending on a release (feature request)

Some resources are meant to be running only once on a cluster, but many charts may be requiring them. Those includes for example cert-manager resources, issuers, ingress controllers, ...
Instead of manually installing those resources in releases before installing more applicative resources, it would be nice to be able to directly install an applicative chart, and have helm itself create the releases for the shared resources, unless it already exists.

@mildred Could this be solved by chart dependencies? https://github.com/helm/helm/blob/master/docs/charts.md#chart-dependencies

Or you mean multiple releases may be relying on something such as redis for example? and you dont want to install it twice? You may use something like Helmfile to install baseline services prior to your custom releases https://github.com/roboll/helmfile

I believe this is a bit different. Dependencies are meant to be
instantiated each time they are mentioned, here, I am thinking of
dependencies that are instanciated once for each release.

I'm going to close this ticket due to inactivity, but please re-open if this still needs to be addressed. Thanks!


fix include feature

include feature will not work properly when sourcemap is generated
(i.e. chunk.files is an array). fix this issue by flattening output
array.

I signed it!

@tbgse is this the PR you were referring to in #23?

Yes

Hi, since #16 already provides filesBlacklist to exclude source map files by default, I think it's safe to include all chunk.files, instead of using just the first file, as described in #23

This will be in the next release going out in the coming days. Thanks for the fix, @laysent!


Color customization

Hello @mayuur, nice work here! :+1:
Could you add color customization, please?

As of now, you can set the background color when notifier gets allocated. Anyways, I am already providing an update to this. will add this one as well!
Thanks! :)

Great! Thank you very much :+1:


Multiple Ark Survival Ascended Servers

I have Ark running all maps from a single install.
I have the Main instance set to do auto backup and update, I have UsePID set to $false.
The main instance wont shutdown any of the others.
Then when it tried to update or start it wont because there are other instances already running.
Server Log.txt

This is an on going issue.
Some words of wisdom would be amazing.

I found the issue, the script does not close all instances of an executable, it finds the first one.
It will take some work to fix, I have to add an option to force close all instances if needed.
The issue is that you only want to do that for updates when you run multiple servers from the same install.

After some work I think it is better to avoid running multiple instances from the same folder.
Making this work would involve a lot of work as the script would need to maintain a state of every running instances and lock every scheduled task during an update as well as maintaining backups and proper procedures for each instances.
All of this while keeping compatibility for other games.


New plugin: typescript-graphql-request

A new plugin, for NodeJS developers, that generates SKD for graphql-request based on schema and operations.
Usage example:
schema: https://countries.trevorblades.com/
documents: 'src/graphql/**/*.graphql'
generates:
  src/sdk.ts:
    plugins:
      - typescript
      - typescript-operations
      - typescript-graphql-request

query continents {
  continents {
    name
    countries {
      ...CountryFields
    }
  }
}
fragment CountryFields on Country {
  name
  currency
}

And then to use the SDK:
import { GraphQLClient } from 'graphql-request';
import { getSdk } from './sdk'; // THIS FILE IS GENERATED

async function main() {
  const client = new GraphQLClient('https://countries.trevorblades.com/');
  const sdk = getSdk(client);
  const { continents } = await sdk.continents(); // This is fully typed, based on the query

  console.log(`GraphQL data:`, continents);
}

main().catch(e => {
  console.error(e);
});

An example repo could be found here:
https://github.com/dotansimha/graphql-codegen-graphql-request-example

This is awesome!

This looks very promising! <3

This is great! Thanks so much!

That's exactly what I was looking for! Thx.

how would it work with swr? swr.now.sh
`import { request } from 'graphql-request'
const API = 'https://api.graph.cool/simple/v1/movies'
const fetcher = query => request(API, query)
function App () {
const { data, error } = useSWR(
{       Movie(title: "Inception") {         releaseDate         actors {           name         }       }     },
fetcher
)
// ...
}`

@Manubi It seems like you need more than just this plugin, because you need to generate the useSWR with typings, right?
I think it could be done only with a new plugin

Yeha I'd be interested in this

@ColeTownsend @Manubi feel free to open a new issue for that if it's still relevant. It seems like a new plugin (or at least configuration for the existing one) is needed to support this use case.

for future google wanderers, I just stumbled across this: https://github.com/croutonn/graphql-codegen-plugin-typescript-swr re: swr


Addition of databases endpoint.

Overview
Based on the discussion in issue #22, #44, and #46, this is a suggestion to extend the OPTiMaDe API specification with a databases endpoint.
This will bump the version up from v0.9.7-develop to v0.9.8-develop.
databases endpoint
The new databases endpoint provides a way for one OPTiMaDe API to point to a number of related "databases"  by the same provider (also served via OPTiMaDe APIs).
It is understood that all databases endpoints for all OPTiMaDe database APIs by the same provider return the same list of related databases.
The makeup of the endpoint can be seen in the diff, where an example is also provided.
API test database
With the addition of the databases endpoint, a concept of a test database has also been formulated and included here. The choice to not split it in another PR is because it is tightly linked with the databases endpoint.
From this PR:

The provider MUST supply one OPTiMaDe API test database. This database MAY be empty.

Note: The database need not be an actual database, but must be available on a separate OPTiMaDe base url.
In the future, the consortium should provide a few structures (e.g. Silicon, BaTiO2, a MOF, and more) that can be uploaded to an actual test database. This should/will be discussed in #44. The OPTiMaDe API implemented by the provider may then be tested by the consortium or the providers themselves to ensure they are compliant with the declared version of the OPTiMaDe API.
Hence, the specifications should in the future be changed from “This database MAY be empty” to something like “This database MUST contain at least the structures provided at <some_url>”. If we agree on this, I suggest we add this statement as a comment to #44.
No “global” empty database
As an added bonus, the requirement of a test database will solve the issue of having a “global” (but empty) database that is referenced by the YAML file in Appendix 1.
Instead the YAML file in Appendix 1 will reference the databases endpoint of the test database.
Minor changes
Updated base URL info-endpoint to include available_endpoints.
This item was included in the JSON example, but not in the official description.

I've been thinking more about a point @giovannipizzi and I discussed in the morning last meeting.
My use case is: I'm providing a general OPTIMaDe implementation in my Python library httk. My implementation will allow users to run httk in a "server mode" where they just point at a httk-generated database file and the entries within that database file are served using the OPTIMaDe API.
Before the present PR, this setup was straightforward. I ask httk to expose the API at the base URL of, e.g., http://<IP_ADDRESS>/, and the user then points their OPTIMaDe client to that URL, and it retrieves data as. e.g.,: http://<IP_ADDRESS>/v0.9/structures/..., etc.
If we add this PR, the URL structure of my implementation will be more awkward. I need to provide more than one Base URL, and set it up so that my index Base URL is, e.g., http://<IP_ADDRESS>/index, and
  http://<IP_ADDRESS>/index/v0.9/databases

points at:
  http://<IP_ADDRESS>/db1/v0.9/structures

Putting the version numbers in the URLs here is optional, but I'm including it to clarify where the base URLs are. The user now needs to be told to point their OPTIMaDe client to: http://<IP_ADDRESS>/index/, which I'm sure will confuse them (i.e., they will try to point it at http://<IP_ADDRESS>/).
I don't think this is a super important issue, as I can work around it. But I think it makes conceptual sense to keep it possible to serve OPTIMaDe out of a single Base URL.
Could it be OK to specifically allow (but not require) databases to exist inside the index URL endpoint, but specifically only under, e.g., /data?
Then I can put http://<IP_ADDRESS>/as my index Base URL. This means that the databases list is at:
  http://<IP_ADDRESS>/v0.9/databases

Which points to:
  http://<IP_ADDRESS>/v0.9/data/db1

Which provides:
  http://<IP_ADDRESS>/v0.9/data/db1/info
  http://<IP_ADDRESS>/v0.9/data/db1/structures

etc. The difference here is that db1 is formally within the index Base URL, meaning I don't need to handle the client asking for a different API version number. I.e., this is not valid:
  http://<IP_ADDRESS>/index/v0.9/data/db1/v0.9/info

Ok, I have created a pull request that implements my comments above, it is in: CasperWA/OPTiMaDe#1.
What this proposal does:

Support for "embedded databases" which makes it possible to serve databases under
the same API URL that hosts the index database.
What was previously /databases is now called /index
This is for two reasons, 'index' (the proper noun, i.e., ~ a directory of things) is
IMHO a better name if we want to be able to link base URLs that aren't strictly databases
under this prefix. And, '/databases' also seems the best name to use for embedded databases
under the same base URL.
I changed the formatting a of the response format from index/siblings quite a bit, as the proposal seemed non-json-api:ish for a listing of entries under a prefix (compare with the output structures/). But maybe I've misunderstood something. I'm not very strongly opposed to revert this, but then we probably should discuss why we disagree on this.
Some other minor fixes, such as clarifying the options in how to organize data, pointing out what to do at a 404, and make sure to say that info and index endpoints need not accept query parameters (since they can be statically hosted.)

Replaced by #63


Move image carousel "Images" heading/controls to the top of the carousel

coverage: 94.504%. remained the same
when pulling 86bd6e5412a45844f9312df063a596e53864f0c5 on nimmo-carousel-heading
into b4343fb7cbe7b5a1e9d146e6180eee9c961bead1 on main.


Release 1.6.27

Summary and Scope
Overview:

CANU versioning management and the build pipeline has been entirely reworked.  End-users should experience no change, but behind the scenes this helps tremendously.
A unnecessary Control Plane ACL, previously removed for other CSM versions was removed for CSM 1.2.
CAN port configurations are now removed from switch ports if the CHN is enabled.
canu validate shcd-cabling and canu validate paddle-cabling have been fixed to better handle incoming LLDP data.  This required enhancement of the CANU internal network model.
Output of many commands has been cleaned up and better next-step guidance added when errors are found.

Details:

Fix the CANU model to allow "cable swapping" while using LLDP data.
Update generated docs from template changes.
Fix sls_utils so that it works correctly when using a SLS.json file.
Shutdown UAN CAN port when CHN is enabled.
Remove Control Plane ACL for CSM 1.2 previously missed.
Update Arista BGP templates.
Add feature flag for CHN BGP control plane.
Fix ordering of switch configuration when using custom switch configs or --reorder.
Node types and model lookups:

Fix login node Shasta name for v1 architecture.
Added gpu and kvm node definitions to v1 architecture.
Add gpu node definitions to v2 TDS architecture.

Messaging and Error cleanup:

Added logging to canu generate switch/network config to clean up output.
Provided better messaging about next-step handling of missing configs in generated files.
Provide better guidance during SHCD validation if port re-use/change is attempted.

Wait for #237 , it's coming in hot/shortly!


Subscribed events tab links broken

The action ("submitted", "commented on", "accepted") on the event stream is no longer visible and clicking on the event no longer redirects to the event.

Fixed. The suggested change logic was looking for 'revisions' instead of 'changes', probably due to a global search/replace that went a bit too far.

Yikes, this is likely my bad.  Sorry.

No prob, ultimately the client should include unit and end-to-end tests that would catch issues like this.


Repair Descriptions with newlines and emit a warning

Closes #1390
Pull Request Checklist

[ ] Changes have tests
[x] News fragment added

Nit: maybe it’d also make sense to str.rstrip() broken descriptions? My guess is that most descriptions with newlines have one at the end. If careless usage of multiline strings (see e.g. PyLops/pyproximal@90d37d0eba62bd3de6a74a021de04a2af4473c2d) is frequent enough, it might also make sense to handle newlines and spaces at the beginning as well, and so str.strip() the descriptions instead.

Nit: maybe it’d also make sense to str.rstrip() broken descriptions? My guess is that most descriptions with newlines have one at the end. If careless usage of multiline strings (see e.g. PyLops/pyproximal@90d37d0eba62bd3de6a74a021de04a2af4473c2d) is frequent enough, it might also make sense to handle newlines and spaces at the beginning as well, and so str.strip() the descriptions instead.

Good point, although I do want to be cautious and not overcorrect here. If somehow a space is already present at the beginning or end of the description, I wouldn't want to alter that as part of correcting for newlines, even if that is the correct thing to do. On the other hand, converting a trailing or leading newline to a space seems janky at best.
I think for now I'll stick with this approach, as it still will correct the underlying defect (that metadata gets corrupted) and provide a path forward. I'll defer more sophisticated validation to subsequent efforts (which probably should happen earlier than when writing values out and probably coordinated across multiple forms of input such as setup.py or setup.cfg).

Good point, although I do want to be cautious and not overcorrect here. If somehow a space is already present at the beginning or end of the description, I wouldn't want to alter that as part of correcting for newlines, even if that is the correct thing to do. On the other hand, converting a trailing or leading newline to a space seems janky at best.
I think for now I'll stick with this approach, as it still will correct the underlying defect (that metadata gets corrupted) and provide a path forward. I'll defer more sophisticated validation to subsequent efforts (which probably should happen earlier than when writing values out and probably coordinated across multiple forms of input such as setup.py or setup.cfg).


UnicodeDecodeError: 'gbk' codec can't decode byte 0x94

Hi. Thanks for the nice package.
with open("README.md", "r") as fh:     long_description = fh.read()
This line in setup.py is causing some problem when doing pip install  jsbuilder  in certain systems (non-utf8 Windows for example):
File "C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-install-mtlho8cf\jsbuilder_c32551d6c9a34bb488d8b58fca88b832\setup.py", line 4, in <module>
        long_description = fh.read()
    UnicodeDecodeError: 'gbk' codec can't decode byte 0x94 in position 14: illegal multibyte sequence

A small patch may fix the problem:
with open("README.md", "r", encoding='utf8') as fh:

Merged, thanks!


Add ledger data to rekor entry

Cosign does not give any option to add it to rekor entry. However was able to add it to the signature file as an annotation


On signoutRedirect() token is still

Why is it that after signoutRedirect() is completed my bearer token is still valid.
Is there something that I missed in the documentation, such that I can destroy my token?

Why is it that after signoutRedirect() is completed my bearer token is still valid.

If it's a JWT, then it is unaffected by user session

I see, makes sense. I think we can close this issue as well.


Add mondatory state keys to useStoreon

Storeon React useStoreon has arguments for a reason. Without it all your components will re-render on any state changes.

There are no reasons to provide technics from React to Solid. We use mandatory state keys to React useStoreon to avoid re-render due to React's Virtual DOM and specific works with hooks. In this case, Solid is more like Vue or Svelte, there is no Virtual DOM, and it uses a reactive graph to detect changes and re-render (https://github.com/ryansolid/solid/blob/master/documentation/comparison.md#react)
Here is a little comparison of how those frameworks render own component
React: https://codesandbox.io/s/magical-poitras-w7l18?file=/src/App.js:0-307
Solid: https://codesandbox.io/s/solid-counter-hn2cq?file=/index.tsx

As I can see there are three layers in Svelte, React, or Solid.js:

Global state
Controller state
DOM

Svelte and Solid.js improved the connection between the Controller state and the DOM layers. But the problem between the Global state and Controller state is still actual.
For instance, here is Solid.js controller:
export default function Counter() {
  const [state, dispatch] = useStoreon()

  return (

      {state.count}
      <button onClick={() => dispatch('inc')}>inc</button>

  )
}

Every time when you will change any state, this code will be executed:
const [state, dispatch] = useStoreon()

In real apps, with 100+ components with a lot of hooks inside, these calls will slow down performance even outside of DOM operations.

This function will executed only once, on initial render.
const [state, dispatch] = useStoreon()

The state is a proxied object of Storeon state and due to that Solid detect changes on the store directly, and if the property used somewhere, it will be reactively updated.
const App = () => {
  const [state, dispatch] = useStoreon();

  const timer = setInterval(() => dispatch("inc"), 1000);
  onCleanup(() => clearInterval(timer));

  createEffect(() => {
    console.log(state.foo);
  });

  return
{state.count}
;
};

function inc(store) {
  store.on("@init", () => ({ count: 0, foo: "baz" }));
  store.on("inc", state => ({ count: state.count + 1 }));
}
const store = createStoreon([inc]);

The log function inside createEffect will be called only once. If we change count, there will no reaction to that, because we don't use it, but if we change foo, our log function will be called again.
For reference: https://codesandbox.io/s/solid-counter-hn2cq?file=/index.jsx:0-773

What is I have code like:
const App = () => {
  const [state, dispatch] = useStoreon();

  const users = state.admins.lenght + state.citizens.length;

  return
{users}
;
};

When state.admins.lenght + state.citizens.length will be called on state.admins changes?

No, exactly this code will not work, because when you add two of these properties (admins.length, citizens.length) you lose reactive reference of parent object state.
But if you put computation into tempalte, like
const App = () => {
  const [state, dispatch] = useStoreon();

  return
{state.admins.lenght + state.citizens.length}
;
};

or use createMemo function from solid-js
const App = () => {
  const [state, dispatch] = useStoreon();

  const users = createMemo(() => state.admins.lenght + state.citizens.length);

  return
{users}
;
};

or even like this, with component state
const App = () => {
  const [innerState, setInnerState] = createState({ users: 0 });
  const [state, dispatch] = useStoreon();

  createEffect(() => {
    const users = state.admins.lenght + state.citizens.length;
    setInnerState({ users })
  })

  return
{innerState.users}
;
};

it will work)

Got it.

Move the repo to storeon org and release it under @storeon/solidjs name. Ping me after each step, I will fix rights.

If you want to promote Solid.js framework, feel free to write a tweet about @storeon/solidjs with a quick description of what is Storeon (Redux-like state manager in X bytes) and Solid.js. It will be nice to have component’s source code and cleaned compiled code images (to show benefits of Solid.js in image).
I will report this tweet, since I like any Svelte-like project.

@ai Package is moved
https://www.npmjs.com/package/@storeon/solidjs
Please rename this repo to solidjs


Bug: can't use Auth config in different namespace than App

PHP Version
8.1.16
CodeIgniter4 Version
4.3.6
Shield Version
dev-develop
Which operating systems have you tested for this bug?
macOS
Which server did you use?
fpm-fcgi
Database
MariaDB 10.2
Did you customize Shield?
No.
What happened?
After merge of https://github.com/codeigniter4/shield/pull/748 we cant use Auth config in different namespace than App.
Steps to Reproduce
We are storing Auth as a module in ROOTPATH/modules/Auth folder. There is Config folder with Auth.php config. This worked before merge https://github.com/codeigniter4/shield/pull/748, after merge our App is broken.
We are using App/Config/Autoload.php to load module like:
        'Auth'           => ROOTPATH . 'modules/Auth/',

Expected Output
We would like to store Auth as module in our App.
Anything else?
Workaround is to move Auth.php to App/Config.

We are storing Auth as a module in ROOTPATH/modules/Auth folder.

@jozefrebjak do you mean that you installed the shield without composer?
To be honest, I don't understand exactly what you mean.
Can you explain the reproduction step by step?

@datamweb  Initialise new CodeIgniter4 project
composer create-project codeigniter4/appstarter shield-auth-config-poc && cd appstarter shield-auth-config-poc

Change minimum-stability in your project
composer config minimum-stability dev
composer config prefer-stable true

Install latest dev-develop shield
composer require codeigniter4/shield:dev-develop

Create .env with content:
database.default.database = ../writable/database.db
database.default.DBDriver = SQLite3

Migrate
php spark migrate -all

Update security, we are using Registrar.php in modules/Auth/Config folder with content:
<?php

namespace Auth\Config;

class Registrar
{
    public static function Security(): array
    {
        return [
            'csrfProtection' => 'session',
        ];
    }
}

Create Auth module folders
mkdir modules && cd modules && mkdir Auth && cd Auth && mkdir Config

Back to root
cd ...

Now manually copy Auth.php to modules/Auth/Config/
cp vendor/codeigniter4/shield/src/Config/Auth.php modules/Auth/Config/

Update Auth.php
namespace CodeIgniter\Shield\Config; -> namespace Auth\Config;
use CodeIgniter\Config\BaseConfig; -> use CodeIgniter\Shield\Config\Auth as ShieldAuth;
class Auth extends BaseConfig-> class Auth extends ShieldAuth
Then in app/Config/Autoload.php add:
    public $psr4 = [
        APP_NAMESPACE => APPPATH, // For custom app namespace
        'Config'      => APPPATH . 'Config',
+       'Auth'        => ROOTPATH . '/modules/Auth',
    ];

Then add Routes.php with content:
<?php

namespace Auth\Config;

use Config\Services;

$routes = Services::routes();

service('auth')->routes($routes);

to modules/Auth/Config folder.
Now we can run:
php spark serve

and go to http://localhost:8080/login and it works, but when we change someting in /modules/Auth/Config/Auth.php it's ignored.
For example we want to use our view for login, so we can create folder Views in modules/Auth/Views and copy default login and change
public array $views = [
        - 'login'                       => '\CodeIgniter\Shield\Views\login',
        + 'login'                       => '\Auth\Views\login',
        'register'                    => '\CodeIgniter\Shield\Views\register',
        'layout'                      => '\CodeIgniter\Shield\Views\layout',
        'action_email_2fa'            => '\CodeIgniter\Shield\Views\email_2fa_show',
        'action_email_2fa_verify'     => '\CodeIgniter\Shield\Views\email_2fa_verify',
        'action_email_2fa_email'      => '\CodeIgniter\Shield\Views\Email\email_2fa_email',
        'action_email_activate_show'  => '\CodeIgniter\Shield\Views\email_activate_show',
        'action_email_activate_email' => '\CodeIgniter\Shield\Views\Email\email_activate_email',
        'magic-link-login'            => '\CodeIgniter\Shield\Views\magic_link_form',
        'magic-link-message'          => '\CodeIgniter\Shield\Views\magic_link_message',
        'magic-link-email'            => '\CodeIgniter\Shield\Views\Email\magic_link_email',
    ];

it's not work. Whole Auth config in our module is ignored.

I believe that issue is with change  of logic like config('AuthRoutes') become config(AuthRoutes::class) and when is config(AuthRoutes::class) used that it cant find config in different namespaces than App.
Here is repo to test it.

@kenjis can you take a look ? I think it's a bug in framework or config(SOMETHING::class) should find configs only in App namespace ? It's correct ? Or it's a bug? I have the same issue in codeigniter4-assets library where it's the same in line.

I thought config('App') and config(App::class) are exactly the same, but it seems not.
I will take a look tomorrow.

@codeigniter4/core-team
Should the following code load the Auth\Config\Auth?
config(\CodeIgniter\Shield\Config\Auth::class)

Looking at the class loading part of Factories it seems supplying the fully qualified class name of the config will satisfy immediately the class_exists check. If we supply Auth then Factories will load first Config\Auth if available since preferApp is true.

Right. We use the generic 'Auth' to signal to Factories that we want any Config class with that name, and it will look in Config namespace first and then work through third-party modules to the framework. All third-party modules have the same priority so supplying the FQCN skips the lookup.
The only workaround I can think of (I use this in my Handlers package I think) is to locate all matching Config classes and then give priority to one that extends another (e.g. Auth\Modules\Auth extends CodeIgniter\Shield\Config\Auth would use the modules version).

Created PR to revert: #762

I have the same issue in codeigniter4-assets library where it's the same in line.

It is a bug in codeigniter4-assets.

I am using codeigniter4/shield as Auth modules.
I have /modules/Auth/Config/AuthGroups with some of dummy permission. this is the core config for my all base auth group configuration.
namespace Modules\Auth\Config;

// other stuffs
use CodeIgniter\Shield\Config\AuthGroups as ShieldAuthGroups;

class AuthGroups extends ShieldAuthGroups
{
    // other stuffs
    public array $permissions = [
        'admin.access'        => 'Can access the sites admin area',
        'admin.settings'      => 'Can access the main site settings',
        'groups.settings'     => 'Can access the groups settings',
        'groups.edit'         => 'Can edit existing user groups',
        'users.list'          => 'Can view a list of users in the system',
        'users.manage-admins' => 'Can manage other admins',
        'users.view'          => 'Can view user details',
        'users.create'        => 'Can create new non-admin users',
        'users.edit'          => 'Can edit existing non-admin users',
        'users.delete'        => 'Can delete existing non-admin users',
        'users.settings'      => 'Can manage User settings in admin area',
        'beta.access'         => 'Can access beta-level features',
        'site.viewOffline'    => 'Can view the site even when in "offline" mode',
        'widgets.settings'    => 'Can view the settings for site Widgets',
        'consent.settings'    => 'Can view the settings for the Consent module',
        'recycler.view'       => 'Can view the Recycler area',
        'logs.view'           => 'Can view the logs',
        'logs.manage'         => 'Can manage the logs',
        'me.edit'             => 'Can edit user\'s own settings',
        'me.security'         => 'Can change user\'s own password',
    ];
    // other stuffs
}

i have another config class called /modules/Languages/Config/AuthGroups. this is the auth group config where i only defined the language modules related auth group & permissions.
namespace Modules\Languages\Config;

// other stuffs
use Modules\Auth\Config\AuthGroups as ConfigAuthGroups;

class AuthGroups extends ConfigAuthGroups
{
    protected array $permissions = [
        'languages.access'       => 'Can access the Languages Files Management Module',
        'languages.view'         => 'Can view all language files',
        'languages.view_single'  => 'Can view individual language files',
        'languages.create'       => 'Can create new language files',
        'languages.edit'         => 'Can edit existing language files',
        'languages.edit_content' => 'Can edit translations within a language file',
        'languages.edit_meta'    => 'Can edit metadata of a language file (e.g., name, locale)',
        'languages.delete'       => 'Can delete language files',
        'languages.manage'       => 'Can manage all aspects of language files (create, edit, delete)',
        'languages.upload'       => 'Can upload language files',
        'languages.download'     => 'Can download language files',
        'languages.import'       => 'Can import translations from an external source',
        'languages.export'       => 'Can export translations to an external source',
        'languages.approve'      => 'Can approve or reject changes made to a language file (for workflows)',

    ];
}

when i request a list of permissions using setting('AuthGroups.permissions'), i expected all the permissions array but i get default permission array defined in AuthGroup.
public array $permissions = [
    'admin.access'        => 'Can access the sites admin area',
    'admin.settings'      => 'Can access the main site settings',
    'users.manage-admins' => 'Can manage other admins',
    'users.create'        => 'Can create new non-admin users',
    'users.edit'          => 'Can edit existing non-admin users',
    'users.delete'        => 'Can delete existing non-admin users',
    'beta.access'         => 'Can access beta-level features',
];

If i am doing something wrong, please correct me.


Docs: Metadata must be base64 encoded

When setting metadata the value must be base64 encoded. We describe in a couple of places that the values are encoded when pulling data, but it's nowhere mentioned that the data must be encoded when setting metadata on a user or organization. The encoding is a bit unexpected and it's easy to miss it.
Acceptance criteria

[ ] Document the encoding on our API docs

Context
https://discord.com/channels/927474939156643850/1053234838859169902/1054722672505929768

In included that in the new API docs

I will close this, as we have included it in the new api documentation


Need 3 chunks for chunkmemset_3 and chunkmemset_6.

Fixes read/write past allocated buffer when using NEON.

@nmoinvaz Review

I'm not sure if I know enough about that ARM code to review it. Perhaps @xuchunmei000 can verify if it works.

@nmoinvaz The guideline is still that two long-time contributors must review any pull request ;)

With the patch, still decompress failed. following is the reproduce steps and valgrind result.
[root@localhost zlib-ng]# test=
[root@localhost zlib-ng]# echo $test

[root@localhost zlib-ng]# for i in $(seq 1 60000); do test=abc$test; done
[root@localhost zlib-ng]# echo $test > test.log
[root@localhost zlib-ng]# ll test.log
-rw-r--r-- 1 root root 180001 Sep 14 21:18 test.log
[root@localhost zlib-ng]# ./minigzip ./test.log
[root@localhost zlib-ng]# valgrind ./minigzip -d ./test.log.gz
==8037== Memcheck, a memory error detector
==8037== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==8037== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==8037== Command: ./minigzip -d ./test.log.gz
==8037==
==8037== Invalid read of size 16
==8037==    at 0x40CFE8: chunkmemset_neon (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x409197: zng_inflate (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x4023EB: gz_decomp (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x4027B7: gz_read (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x4028A7: zng_gzread (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x40147B: gz_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x4016D7: file_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x4010CF: main (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==  Address 0x4a06468 is 49,144 bytes inside a block of size 49,152 alloc'd
==8037==    at 0x4866EB0: malloc (vg_replace_malloc.c:309)
==8037==    by 0x401447: gz_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x4016D7: file_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x4010CF: main (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==
==8037== Invalid write of size 8
==8037==    at 0x40CFF0: chunkmemset_neon (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x409197: zng_inflate (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x4023EB: gz_decomp (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x4027B7: gz_read (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x4028A7: zng_gzread (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x40147B: gz_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x4016D7: file_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x4010CF: main (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==  Address 0x4a0646b is 49,147 bytes inside a block of size 49,152 alloc'd
==8037==    at 0x4866EB0: malloc (vg_replace_malloc.c:309)
==8037==    by 0x401447: gz_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x4016D7: file_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x4010CF: main (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==
==8037== Invalid read of size 16
==8037==    at 0x40D01C: chunkmemset_neon (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x409197: zng_inflate (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x4023EB: gz_decomp (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x4027B7: gz_read (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x4028A7: zng_gzread (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x40147B: gz_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x4016D7: file_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x4010CF: main (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==  Address 0x4a06468 is 49,144 bytes inside a block of size 49,152 alloc'd
==8037==    at 0x4866EB0: malloc (vg_replace_malloc.c:309)
==8037==    by 0x401447: gz_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x4016D7: file_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x4010CF: main (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==
==8037== Invalid write of size 8
==8037==    at 0x40D02C: chunkmemset_neon (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x409197: zng_inflate (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x4023EB: gz_decomp (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x4027B7: gz_read (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x4028A7: zng_gzread (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x40147B: gz_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x4016D7: file_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x4010CF: main (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==  Address 0x4a0646e is 49,150 bytes inside a block of size 49,152 alloc'd
==8037==    at 0x4866EB0: malloc (vg_replace_malloc.c:309)
==8037==    by 0x401447: gz_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x4016D7: file_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==    by 0x4010CF: main (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==8037==
==8037==
==8037== HEAP SUMMARY:
==8037==     in use at exit: 0 bytes in 0 blocks
==8037==   total heap usage: 9 allocs, 9 frees, 118,502 bytes allocated
==8037==
==8037== All heap blocks were freed -- no leaks are possible
==8037==
==8037== For lists of detected and suppressed errors, rerun with: -s
==8037== ERROR SUMMARY: 6 errors from 4 contexts (suppressed: 0 from 0)

I didn't add padding to the end of the allocated buffer yet. That is the next step.

I didn't add padding to the end of the allocated buffer yet. That is the next step.

use the latest patch, is ok.
But I have a question, does it mean that we should add padding to allocated buffer in the application code which uses zlib ?

@xuchunmei000 do you have a working test case that reproduces this issue against zlib-ng?
@mtl1979 do you have a test case that reproduces the issue?

@xuchunmei000 I think it is only required when decompressing.

@nmoinvaz We should only need maximum of 16 bytes of padding because it never tries to read more than single vector width at a time. That is because of the extra len checks I added. In other architectures it always use single vector register so there is no issue either.

@xuchunmei000 I think it is only required when decompressing.

there are too many applications which use zlib, modify every application is impossible and complicated.
any possible to process padding in zlib-ng internal ?

@xuchunmei000 do you have a working test case that reproduces this issue against zlib-ng?

I post the reproduce steps at https://github.com/zlib-ng/zlib-ng/pull/759#issuecomment-692404211, do you mean this?

Ah I didn't see all the steps in the output. Nice!

@xuchunmei000 I've seen many applications use less than 16 bytes for output buffer, so it would be very hard for it to be handled inside zlib-ng... For compressing, it can be handled inside zlib-ng though...

@xuchunmei000 I've seen many applications use less than 16 bytes for output buffer, so it would be very hard for it to be handled inside zlib-ng... For compressing, it can be handled inside zlib-ng though...

ok, got it.

This doesn't seem very compatible with stock zlib, it has no requirements of buffers being 16bytes bigger..

@Dead2 I know, but there is no easy alternative except removing the whole chunkcopy/chunkset code... The current code always reads or writes at least 8 bytes, which means in some cases it can read or write beyond the specified buffer size. Buffer size doesn't get passed to the internal functions, so there is no easy way to check that bounds are honored...

@mtl1979 are you sure it is the case? Does this code prevent it?
https://github.com/zlib-ng/zlib-ng/blob/1f10aa4e29d4be6f2a3c63458e737a5b16b2d6e5/inffast.c#L271
Also in other places it uses chunkcopy_safe (above that line) so it should be safe there.

@nmoinvaz Obviously bounds check fails somewhere, but as I don't have real hardware with AAarch64 processor, I can't debug it... I must trust the output of valgrind.

@xuchunmei000 I changed the code to honor boundaries... It shouldn't need the padding for inflate() anymore... Can you retest?

with the latest patch:
[root@localhost zlib-ng]# echo $test > test.log
[root@localhost zlib-ng]# ll test.log
-rw-r--r-- 1 root root 180001 Sep 16 21:14 test.log
[root@localhost zlib-ng]# ./minigzip ./test.log
[root@localhost zlib-ng]# valgrind ./minigzip -d ./test.log.gz
==7717== Memcheck, a memory error detector
==7717== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==7717== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==7717== Command: ./minigzip -d ./test.log.gz
==7717==
==7717== Invalid read of size 16
==7717==    at 0x40D050: chunkmemset_neon (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x409197: zng_inflate (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x4023EB: gz_decomp (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x4027B7: gz_read (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x4028A7: zng_gzread (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x40147B: gz_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x4016D7: file_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x4010CF: main (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==  Address 0x4a06468 is 49,144 bytes inside a block of size 49,152 alloc'd
==7717==    at 0x4866EB0: malloc (vg_replace_malloc.c:309)
==7717==    by 0x401447: gz_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x4016D7: file_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x4010CF: main (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==
==7717== Invalid write of size 8
==7717==    at 0x40D058: chunkmemset_neon (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x409197: zng_inflate (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x4023EB: gz_decomp (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x4027B7: gz_read (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x4028A7: zng_gzread (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x40147B: gz_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x4016D7: file_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x4010CF: main (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==  Address 0x4a0646b is 49,147 bytes inside a block of size 49,152 alloc'd
==7717==    at 0x4866EB0: malloc (vg_replace_malloc.c:309)
==7717==    by 0x401447: gz_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x4016D7: file_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x4010CF: main (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==
==7717== Invalid read of size 16
==7717==    at 0x40D084: chunkmemset_neon (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x409197: zng_inflate (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x4023EB: gz_decomp (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x4027B7: gz_read (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x4028A7: zng_gzread (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x40147B: gz_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x4016D7: file_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x4010CF: main (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==  Address 0x4a06468 is 49,144 bytes inside a block of size 49,152 alloc'd
==7717==    at 0x4866EB0: malloc (vg_replace_malloc.c:309)
==7717==    by 0x401447: gz_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x4016D7: file_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x4010CF: main (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==
==7717== Invalid write of size 8
==7717==    at 0x40D090: chunkmemset_neon (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x409197: zng_inflate (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x4023EB: gz_decomp (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x4027B7: gz_read (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x4028A7: zng_gzread (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x40147B: gz_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x4016D7: file_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x4010CF: main (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==  Address 0x4a0646e is 49,150 bytes inside a block of size 49,152 alloc'd
==7717==    at 0x4866EB0: malloc (vg_replace_malloc.c:309)
==7717==    by 0x401447: gz_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x4016D7: file_uncompress (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==    by 0x4010CF: main (in /root/rpmbuild/BUILD/zlib-ng/minigzip)
==7717==
==7717==
==7717== HEAP SUMMARY:
==7717==     in use at exit: 0 bytes in 0 blocks
==7717==   total heap usage: 9 allocs, 9 frees, 118,502 bytes allocated
==7717==
==7717== All heap blocks were freed -- no leaks are possible
==7717==
==7717== For lists of detected and suppressed errors, rerun with: -s
==7717== ERROR SUMMARY: 6 errors from 4 contexts (suppressed: 0 from 0)

@xuchunmei000 I'm removing whole unrolling and use safe copy for all distances not equal to chunk size... That should get rid of the invalid 16 byte read at least...

@mtl1979 can you rename the file to test/GH-751/test.txt and then add a unit test for it in cmake?
Should be something like this..
    add_test(NAME GH-751
        COMMAND ${CMAKE_COMMAND}
        "-DTARGET=${MINIGZIP_COMMAND}"
        -DINPUT=${CMAKE_CURRENT_SOURCE_DIR}/test/GH-751/test.txt
        -P ${CMAKE_CURRENT_SOURCE_DIR}/cmake/test-compress.cmake)

@nmoivaz Done... Added it to test/Makefile.in too for configure builds...

@nmoinvaz Only really working solution is to remove chunkmemset_3() and chunkmemset_6() as those will always cause crash due to requiring three times the bit width...

I'm okay with that solution if @Dead2 is. It will make the NEON implementation simpler. I recommend still leaving in the tests.

@nmoinvaz I didn't see them used anywhere else and in inflate() they are useless... I still need to check if the CHUNKUNROLL/CHUNKCOPY is needed after adding the CHUNKCOPY_SAFE version...

Is this going to be followed by a PR to remove chunkmemset_3/6 for other arches also, since they get removed from chunkset_tpl.h too?
Is this a problem on the other arches too? Just wondering why this PR removes the support from chunkmemset_tpl.h and not just from the ARM arch, as I don't see any mention of that being intentional.

@Dead2 Neither 3, 5, 6 or 7 fits evenly in vector registers so require more registers than other distances. With distances larger than 8 we can use same fallback if chunk size is larger than 8 bytes. If chunk size is 8 bytes, it will still unroll first for distances larger than 8. Basically it will use largest possible but smaller register width than distance automatically. I tested the fix on all known architectures and it avoids buffer overflow.


User avatars seem to change color often

Gitea version (or commit ref): 1.1.1+2-g2854c8a
Git version: 2.12.2
Operating system: Windows Server 2012
Database (use [x]):

[x] MSSQL

Can you reproduce the bug at https://try.gitea.io:

[x] Not relevant because avatars use Gravatar.

Log gist: Not relevant

User avatars seem to change color often. This is for auto-generated user avatars. I am in offline-mode and not using Gravatar.
Shapes in avatars remain the same. It seems I notice this almost daily. Is this a feature?
I would prefer color to stay consistent.

It is bugging me too.
I looked to find the cause, and this here seems to be the culprit: https://github.com/go-gitea/gitea/blob/master/models/user.go#L863
This looks pretty hacky.  I am not knowledgeable enough to move the block elsewhere. Can a contributor chime in on this?

@razzintown @dubeg have you tested v1.1.3 we release last days. I haven't reproduce the issue on that version.

I downloaded 1.1.3 just to test it out. The bug still exists.
Steps to repro:

Install Gitea.
Register your first user.
Login as your first user. Notice what your avatar looks like.
Logout.
Login again. Notice that your avatar looks different. The colors are most likely changed from the last time you logged in.

If at step 5, your avatar looks the same, then you lucked out. Try the steps again and you'll problably notice a differently colored avatar.
By the way, if I comment out or simply add a check before user.go#L863, the bug goes away. I'm pretty sure the fix could be to move the following block of code...
if !u.IsOrganization() {
	if len(u.AvatarEmail) == 0 {
		u.AvatarEmail = u.Email
	}
	u.Avatar = base.HashEmail(u.AvatarEmail)
}

... somewhere else. I don't think this code should be in the updateUser function. That function is called very often for many different reasons. There's no need to update the Avatar property everytime this method is called.
Instead, this block of code should be ran once at user creation, and perhaps also after avatar settings are updated. That said, there should be a check before the Avatar property is updated with the hash of the email: this should only be done if the Gravatar service is being used, I imagine. Alternatively, the files of the randomly-generated avatars could be named after the hash of the user's email address.

@dubeg OK. I reproduced that.


Refactor data flow and terminal library

The data flow design was slightly broken, a global Cli_state.state was passed around and mutated, by both user input, the rendering loop, and network input.  Error handling was done at the beginning of the rendering loop.  Also, by using (outdated) Lambda_term/Zed/Camomile libraries, the width of unicode characters (can be 2 columns) lead to UI problems (motivated by 茶 as status line).
Now, there is a rewrite: the main loop renders the state, receives a state -> state Lwt.t function from a Lwt_mvar, and continues (unless quit was executed).
Terminal input handling is done in a separate task (to have SMP question support, where both a question and a shared secret needs to be entered, another Lwt_mvar is used (plus a global bool ref, bad bad bad).
Commands, navigation, also network input puts something into the state -> state Lwt.t mvar.  Thus, there's a single (immutable!) state (NB: the Contact.contacts still uses a Hashtbl, but should be straightforward to use a Map.t now).  To comply with this design, the network interaction needed slight improvement.  Error handling is now done while executing the action, and while rendering (instead of having every potentially failing network send wrapped in a try_lwt).  Once a disconnect happens, a reconnect timer is started (via Lwt.async from the main loop)!
The rendering now uses the notty library, which exposes combinators for rendering text.  The rendering code is hugely improved (and much more readable), additionally, there is no need anymore to compute width and height manually.
There might be some regression (such as certain key bindings do not work, C-, C-, mark/yank/paste, undo/redo), also smart line wrapping (find whitespace within the last 10 chars) is not implemented yet.

There is no longer any need for force_redraw and thus no hex characters in the lower left corner.  The resulting binary is self-sufficient (no need for camomile data files) and roughly 13MB in size (on an amd64).  After decrypting a message, the resulting data is validated to be UTF-8.
New features include that suggestions for commands are now rendered in lightgray (type / to see an example); the configuration dialog has been rewritten and does not exit the program anymore if invalid data is entered.

I think this is good now... will see what reports I get in ;)


Add a checkbox for displaying local modules

A fix for Issue #3014
I have even added a separate checkbox for displaying only Libraries/APIs, however it does not do anything yet.

Can one of the admins please verify this patch?

@GooeyHub: ok to test
@TheFlash98 hey there and thank you :-)
For persisting the state of the config item have you looked at some of the other player settings that persist? I don't recall if there are any checkboxes but there are other things like player name that persist. Should be the same functionality for a checkbox, just a different widget

Hooray Jenkins reported success with all tests good!

I have saved the state of the checkbox in the config. There was a merge conflict which I took care of. Do have a look and let me know! 🙂

Hooray Jenkins reported success with all tests good!

Hooray Jenkins reported success with all tests good!

Hooray Jenkins reported success with all tests good!

Hooray Jenkins reported success with all tests good!

Hooray Jenkins reported success with all tests good!

Do let me know if there is any else 🙂

Something weird happens when you keep the checkbox ticked, close the game and then restart it.
Unticking the checkbox now doesn't load the modules that aren't locally present.
Have a look at the updateModuleInformation, there might be an issue with the metaDownloader that receives all the modules after some 2-3 seconds and then changes the sortedModules list.

@Cervator and @nihal111 I had a look at updateModuleInformation, couldn't figure out the problem. Any more help?

I don't know much about it but maybe @vampcat or @eviltak can help with anything related to config :-)

Hooray Jenkins reported success with all tests good!

Spotted the updated commit, validated that the checked/unchecked state of the checkbox sticks now :-)
The other checkbox doesn't persist and as noted doesn't do anything yet either so I went ahead and disabled it so it'll show but not be something you can interact with yet.
Thanks!


Halt bad commits pushing noisy branch creates.

Codecov Report
Base: 23.67% // Head: 23.67% // No change to project coverage :thumbsup:

Coverage data is based on head (5af0a07) compared to base (debb3aa).
Patch has no changes to coverable lines.

Additional details and impacted files
@@           Coverage Diff           @@
##           stable     #456   +/-   ##
=======================================
  Coverage   23.67%   23.67%
=======================================
  Files          63       63
  Lines        5754     5754
  Branches      312      312
=======================================
  Hits         1362     1362
  Misses       4343     4343
  Partials       49       49

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Kivy touch events not working on Raspberry PI with Adafruit PiTFT

Hi!
I have an Adafruit 2.8in LCD with a resistive touchscreen [1], connected to a Raspberry Pi and working with Kivy. I have had to use fbcp [2] to get this setup to work with Kivy, but that is fine. While I have seen issues with idle apps using 30-40% CPU (as per htop), the primary blocker in my setup is that the touchscreen errors out with Kivy.
I can create a Button in a Kivy App and press it, Kivy does see the touch, but errors out:
[INFO              ] [HIDMotionEvent] using <stmpe-ts>
 Exception in thread Thread-1:
 Traceback (most recent call last):
   File "/usr/lib/python2.7/threading.py", line 552, in __bootstrap_inner
     self.run()
   File "/usr/lib/python2.7/threading.py", line 505, in run
     self.__target(*self.__args, **self.__kwargs)
   File "/home/pi/kivy/kivy/input/providers/hidinput.py", line 580, in _thread_run
     process_as_mouse_or_keyboard(*infos)
   File "/home/pi/kivy/kivy/input/providers/hidinput.py", line 456, in process_as_mouse_or_keyboard
   l = keyboard_keys[ev_code][-1 if 'shift' in Window._modifiers else 0]
 KeyError: 330

Here are a few additional notes:

After this, I am unable to get another button press to register in the log, but the app is still running and updating the screen.
The above errors were seen with kKivy installed from git master from a few days ago, I will update to latest master and retry.
I can debug, build, etc, specific branches, patches, etc.. anything to help this move along.

[1] https://www.adafruit.com/prohttps://www.adafruit.com/product/1601duct/1601
[2] https://github.com/tasanakorn/rpi-fbcp

Sorry, I know this thread is closed. @tito, would you make tutorial-like post for other who began work with PiTFT + Kivy + Raspberry, because for example I can't fully understand what you've done to work this touchscreen. It would be good because all who have issues will be clarified by your post and will not disturb you. Thanks

@tito I am facing issue even displaying anything on the touchscreen. The screen is calibrated (I don't use X either). Event test works (ts_test & evtest). Please let me know how to display the UI on touch display instead of HDMI. (I have tried the usual methods FRAMEBUFFER variable & DISPLAY variable)