Dolphin Bean Garbage Collection

The model in Dolphin Platform is defined as a hierarchy of dolphin beans. When a bean is removed from the hierarchy the representation in the removing layer (PM) isn't removed. A developer need to call bean manager.remove(..) to remove the PM. A GC algorithm can found this beans and remove them automatically at the end of each server cycle.

see #63


Contribution discussion: dynamically change the log level on the server

I think I saw a node logging module that had this feature - change the log level at runtime, for given methods even, to delve into problematic areas. pince implements namespacing and dynamic log level change on the client, but not on the server.

Could you edit the Meteor.settings object at runtime?  Does it check that only once when things spin up, or each time a log message is created?


Fix Mining Issues

Fix ore message when you successfully got an ore.
Fix issue #247 and currently redid how graham had it originally

def find_pickaxe
  level = mob.skill_set.get_skill(Skill::MINING).current_level
  inventory, equipment = mob.inventory, mob.equipment

  indices = PICKAXE_IDS.select do |id|
    PICKAXES[id].level <= level && (inventory.contains(id) || equipment.contains(id))
  end

  PICKAXES[indices.max]
end

Still pretty hacky but that will do for now

current_level should be changed to maximum_level since you can't boost your level to use better pickaxes

@cubeee true but what about when u get stat reduced on mining does it affect ur pickaxes?

@WizardJesse1 No, it just lowers your chances of getting ore

o cool okay thanks

With this branch open (and assuming you haven't made any further commits to it, or any changes to other files), run:
git reset --hard HEAD~1
git fetch origin master
git rebase origin/master


Quickfilter on invisible column

Hi
I display text field (looked up) in grid, but would like to filter on ID of the Enterprise.
If there a way to put QuickFilter on nondisplayed field ?
(I want to get rid of EnterpriseId field from grid)
[QuickFilter(CssClass = "hidden-xs"), LookupEditor(typeof(Lookups.UsersEnterpriseLookup))]
        [Width(5)]
        public Int32 EnterpriseId { get; set; }
        public String EnterpriseUsername { get; set; }

found it
[QuickFilter(CssClass = "hidden-xs"), LookupEditor(typeof(Lookups.UsersEnterpriseLookup))]
        [Visible(false)]
        public Int32 EnterpriseId { get; set; }


test: update tests to handle upstream libxml2

What problem is this PR intended to solve?
upstream integration testing with libxml2, see gnome/libxml2@4fefba4c

Upstream CI is https://github.com/sparklemotion/nokogiri/actions/runs/9130983439


i18n(zh-cn): Update microsoft-azure.mdx

Description (required)
Update microsoft-azure.mdx

Related issues & labels (optional)

#7955

Lunaria Status Overview
🌕 This pull request will trigger status changes.

Learn more
By default, every PR changing files present in the Lunaria configuration's files property will be considered and trigger status changes accordingly.
You can change this by adding one of the keywords present in the ignoreKeywords property in your Lunaria configuration file in the PR's title (ignoring all files) or by including a tracker directive in the merged commit's description.

Tracked Files

Locale
File
Note

zh-cn
guides/deploy/microsoft-azure.mdx
Localization changed, will be marked as complete.

Warnings reference

Icon
Description

🔄
The source for this localization has been updated since the creation of this pull request, make sure all changes in the source have been applied.


Issues with fitView when asynchronously loading elements and container height from an API - NextJS (codesandbox provided)

Hi,
I'm having trouble trying to get fitView called after new elements are loaded asynchronously in a Next JS site I'm working on.
I've looked through all the fitView-based threads on here and tried all the solutions but they're not working for me. I followed this thread and tried the different fixes in there, including the one linked at the bottom by @chrtze (here) but it didn't solve the issue for me.
Just to explain my set-up. I have an API that serves react-flow elements as json to the getStaticProps() function of two pages in a NextJS app. These pages pass the elements, along with the container height to the component that uses ReactFlow as props. I use useEffect to set the elements and height and have tried using the setup in @chrtze 's answer to set the ReactFlow state onLoad() with a second useEffect that checks that the elements, ReactFlow instance and height have been set and then calls fitView() on the instance. Each page has a link to each other underneath the ReactFlow container, which triggers the re-render.
However, this isn't working, the charts don't resize and clip out of the container. I have added a button that calls fitView() on click to make sure fitView() is working correctly, which it is. It just seems that, even with using the useEffect() trick, fitView() still isn't being called after the new height is set and the new elements are loaded.
I've also added console.log() comments to see in what order each function is called (onLoad and the useEffects) and it seems they are called in a different order on first load (when you refresh the page manually) and when you click the link, triggering the re-render, so I guess this is where the issue might be.
I would be very grateful if somebody could take a look and can see what I am doing wrong. The main difference between my example and @chrtze 's is that I'm using NextJS and also changing the height of the container  sits inside, so maybe that's part of the issue also?
Here is the codesandbox: https://codesandbox.io/s/sleepy-silence-1pdcw
Thanks to anyone who has a look at it, it's greatly appreciated!
P.S. Please ignore the custom node and edge visual bugs, I haven't got around to fixing them yet because this issue took my focus.

I know this may not be the "straightforward" answer you're looking for, but it might help.
I had this same problem, although instead of fitting the view I needed to set it to a specific placement and zoom.
Anyways, what I did was create a loading component that visually covered the ReactFlow container. I then passed this loading state to the container of ReactFlow, which would turn off the loading screen once the RF instance was loaded. It looks roughly like this:
const FlowContainer = () => {
    // state here
    return (
         <>
             <FlowLoading loading={loading} />
	    <FlowPanel setLoading={setLoading} />
        </>
    )
}

Then, in FlowPanel:
const onLoad = reactFlowInstance => {
	setInstance(reactFlowInstance);

	setTimeout(() => {
                // notice that I don't use the state instance
		if (Boolean(reactFlowInstance)) {
			reactFlowInstance.fitView();
		}
		setLoading(false);
	}, 20);
};

This timeout should help. Essentially, we're allowing the Flow to render without us seeing it right away. Once it's done rendering (given to us by onLoad), we can safely remove the loading screen and fit the view.
My setup is identical to yours: NextJS with API-served elements. I'm not sure if this will work for you, but let me know if it does.
I hope this helps.

I have just encountered the same bug in a different project. We will look into it for the next release as well.

@xplato Thanks for this, I think that'll be the way I end up going. I've got it working (relatively well) with the setTimeout trick so far so thanks for that! The graphs I'm working with are relatively small so the change before/after the fitView isn't perceptible but I'm worried that when I start incorporating larger graphs, they will flash for a split second as not fitting before fitView is called and so I'll probably then need to go your route and overlay a "loading" pane. Thanks again for the help!

@tompbsmith I don't know how complex your project is, but could you try the next major version npm i --save react-flow-renderer@next (there are some breaking changes that are listed here #1555). That version comes with a fitViewOnInit prop that should solve your problem.

Please re-open if the issue still persists.


[5.3] Add $when parameter in binding

Sometimes, you have clashes with name binding.
If you have the following code
Router::get('/news/{category}', ...);
Router::get('/faqs/{category}', ...);

You will not be able to bind category to load your news category or your faq category.
This PR adds a third parameter to Router::bind and Router::model. This $when parameter allows you to add a regex which will match your route.
You will be able to specify the regex matching your route
Router::bind('category', function($value) {return ...}, 'news/.*');
Router::model('category', Faq\Category::class, 'faqs/.*');

The PR includes test coverage.

No plans to add this to core.


Way to describe CNV gain of greater than a dup (ie ISCN x4)

International System for Human Cytogenomic Nomenclature (ISCN) has "x1", "x3", "x4" etc
x1 is a HGVS del
x3 is a HGVS dup
x4 on different chromosomes could be "homozygous for a dup" but if it is x1 on one chromosome and x3 on the other there is no easy way to describe it in HGVS (if it is very large)
There are 1093 entries for x4 in ClinVar - example - https://www.ncbi.nlm.nih.gov/clinvar/variation/687667/
GRCh37/hg19 7q11.21(chr7:63903253-64426719)x4
There is repeated sequences but that appears to be for when the repeat is small, ie not "del/dup" sized
Suggestion for new nomenclature
You could potentially argue for just adding x4 on the end (inkeeping with ISCN) however if you wanted to style it like a mashup of repeat sequences + del/dup perhaps you could describe:
GRCh37/hg19 7q11.21(chr7:63903253-64426719)x4
in HGVS as:
NC_000007.13:g.63903253_64426719cnv[4]
So these are equivalent:
NC_000007.13:g.63903253_64426719del = NC_000007.13:g.63903253_64426719cnv[1]
NC_000007.13:g.63903253_64426719dup = NC_000007.13:g.63903253_64426719cnv[3]
(edit: I changed "mul" to "cnv". I had originally also considered "cop" but disliked it)

A thing to consider here is that the HGVS nomenclature reserved "dup" for tandemly duplicated sequence, unlike VCF which does not apply this restriction.
I like the idea of
NC_000007.13:g.63903253_64426719del = NC_000007.13:g.63903253_64426719cnv[1]
NC_000007.13:g.63903253_64426719dup = NC_000007.13:g.63903253_64426719cnv[3]
``

but this format suggests we know the exact number of copies of the duplicated region, and may break if we do not. So how about something like

NC_000007.13:g.63903253_64426719del = NC_000007.13:g.63903253_64426719cnl[1]
NC_000007.13:g.63903253_64426719dup = NC_000007.13:g.63903253_64426719cng[3]

where cnl states an overall loss of copies and cng an overall gain of copies and the base format can be

NC_000007.13:g.63903253_64426719del = NC_000007.13:g.63903253_64426719cnl
NC_000007.13:g.63903253_64426719dup = NC_000007.13:g.63903253_64426719cng

if the exact number of copies is not known.

Instead of thr number being copies, you could store the change, ie:
100_200cnv[-1] == :g.100_200del
:g.100_200cnv[+1] == :g.100_200dup

NC_000007.13:g.63903253_64426719dup = NC_000007.13:g.63903253_64426719cnv[3]
[...]
1:g.100_200cnv[+1] == 1:g.100_200dup

A duplication is not equivalent to a copy number gain, i.e., every duplication can be written down as a CNV (losing information about the location of the duplicated region), but not the other way around.
With regards to absolute or relative counts, perhaps we can also use a couple of examples from chrY to see what feels more natural.

Agreed, the cnv description must be distinct from the dup description.

regarding this discussion, we already have [2] to indicate two copies of a sequence which is distinct from dup for a duplication. Why we need "cnv" is not clear to me.

@jtdendunnen. Please provide the links to the varnomen page if possible and an example of use.
My argument would be that HGVS reserved the dup syntax for a tandem duplication so we should not use that. Also, we ideally need a way to state a copy number change and whether the copy number is increased or decreased. It could be we already have rules for this that could be used but the relevant place in the website and a couple of examples would be useful.

@jtdendunnen  I think you mean repeated sequences e.g. NC_000003.12:g.63912687AGC[13]
As far as I can tell from the spec they must have a sequence, which means they are good for small repeats but not for large CNVs

when I have sequenced the break point I should now where an additional copy is located and how to describe it. When I did not sequence the break point and when I am not sure whether the extra copy is in tandem (so a duplication) or not (so an insertion) I describe the variant like RefSeq:g.(A_B)_(C_D)[2] where A/D gives the max. and B/C the min. extent of the extra copy (described as [2]).

Yeah, I agree that we should distinguish between repeating in place (+1 extra copy = dup, +2 being a tripling of a sequence) vs being an insertion
So maybe 'cnv' isn't a good descriptor term as that can be untethered from location (eg a x3 could be dup plus an ins elsewhere) and really just means amplification
So maybe 'rpt'?

I don't know if it's a good idea to allow HGVS to describe copying to a new or unknown location...
Though if you can, it would allow you to describe Trisomy 21 in HGVS  as:
NC_000021.9:g.1_46709983cnv[+1]

This also would bring HGVS in line with VCF and VRS which can describe cnv increase or decrease agnostic of knowing the new location.

Trisomy 21 in HGVS is described as NC_000021.9:g.pter_qtersup  (sup for supernumary chromosome)

I think part of this conversation is based on the thought that the variant description is for the entire organism. E.g;

So these are equivalent:
NC_000007.13:g.63903253_64426719del = NC_000007.13:g.63903253_64426719cnv[1] NC_000007.13:g.63903253_64426719dup = NC_000007.13:g.63903253_64426719cnv[3]

This is not correct; an HGVS variant description is based on one molecule, one allele. I.e., the proposed NC_000007.13:g.63903253_64426719cnv[3] would state that there are four copies of this region in the individual. Three copies on one allele, additionally to the normal copy on the other allele. I.e., NC_000007.13:g.63903253_64426719cnv[3] == NC_000007.13:g.[63903253_64426719[3]];[63903253_64426719[1]].
See here for more information:
https://hgvs-nomenclature.org/en/latest/recommendations/DNA/alleles/
So any variant description coming out of this discussion should always be allele-based, as HGVS is allele-based.

The proposed g.100_200cnv[-1] would equal a simple deletion and should therefore be written as g.100_200del. The other allele, if present, is assumed normal if not mentioned.
g.100_200cnv[-2] is not possible as those positions are only present once in that reference sequence. This should, therefore, be written as g.[100_200del];[100_200del].
g.100_200cnv[+1] or g.100_200[+1] makes just as much sense as g.100_200cnv[2] or g.100_200[2] in my opinion.
g.100_200cnv[+2] could be g.100_200cnv[3] (g.100_200[3]), or g.[100_200cnv[2]];[100_200cnv[2]] (g.[100_200[2]];[100_200[2]]), or in theory, even a +3 combined with a deletion. I don't remember from the top of my head whether there is a rule that indicates you're not sure which event took place.
Note that, although the syntax using [2] is currently in use, it's not endorsed by the HVNC, and therefore, isn't official HGVS nomenclature.

HGVS can't currently describe copying to a new or unknown location...

The insertion syntax can be used to describe a copy to a new location, e.g.,
NC_000021.9:g.10000000_10000001ins20000000_20000030
or when the inserted sequence originates from a different chromosome:
NC_000021.9:g.10000000_10000001insNC_000022.7:g.20000000_20000030
Copying to an unknown location cannot be expressed as far as I know.

HGVS does have a format for an insertion at another position in the genome, see http://varnomen.hgvs.org/recommendations/uncertain/. Correct is NC_000021.9:g.10000000_10000001ins[NC_000022.7:g.20000000_20000030]

Copying to an unknown location cannot be expressed as far as I know.

The nomenclature guidelines provide this example:

g.?_?ins[NC_000023.10:g.(12345_23456)_(34567_45678)]: the insertion of a sequence from the X-chromosome (NC_000023.10), maximally involving nucleotides 12345_45678 but certainly nucleotides 23456_34567, at an unknown position (g.?_?) in the genome

Two more pages indicate similar examples. However, the major issue here is that a reference sequence is required, meaning at least the chromosome in which the insertion took place should be provided. If the chromosome in which the insertion took place is unknown, no reference sequence identifier can be added to the variant description, and it'll be impossible to generate a valid HGVS-compliant variant description.


I can't see FakeLogonScreen.exe and FakeLogonScreenToFile.exe

Can you link me this file? thanks

Please check the Releases page where you can download the fakelogonscreen_trunk.zip file which contains both executables. Note that antivirus programs might trigger when scanning the zip and automatically remove both executables from it.


Update to fluentd 1.10.4

Signed-off-by: Jonas-Taha El Sesiy<EMAIL_ADDRESS>

@repeatedly The usual :)

Thanks!


test-branch/execute-job/comment-from-wrong-user-on-pr/e7400243-a2f0-4e01-9dda-cb10f2ed4152

PR for testing 442337df7c9eb190ea99fcba74b138a4fbbbc237

/canonical/self-hosted-runners/run-workflows 442337df7c9eb190ea99fcba74b138a4fbbbc237


Download of test modell not working - Access Denied

Hi Hiroki,
when downloading the modell, I get the following error.
Jan
AccessDenied
<Message>Access denied.</Message>

Anonymous caller does not have storage.objects.get access to chakki/datasets/public/ner/models_en.zip.

</Error>

In anago ver 1.0.7, you can download the model from the following link:

https://storage.googleapis.com/chakki/datasets/public/ner/conll2003_en.zip

Thanks


Viewers: Add customization setting for axis name size

Adding a setting to customize the axis name size. This feature would enhance usability and could be implemented by right-clicking on the axis text for all viewers with selectors.

This issue has been mirrored in Jira: https://reddata.atlassian.net/browse/GROK-17129


Clean out the GitHub Repo

From what I remember @RubenVerborgh did this task last year...

?

Oh my mistake, I think it was @justinwb 😓

I think @RubenVerborgh proved it was doable.
But the task needs some inventory.

Finalizing this in https://github.com/solid/team/issues/22


Turn off client side metrics sink since the master will be removed

What changes are proposed in this pull request?
Turn off client side metrics sink since the master will be removed.
Why are the changes needed?
Turn off client side metrics sink by default since the master will be removed.
Does this PR introduce any user facing changes?
NA

alluxio-bot, merge this please


Deleting a space doesn't delete associated pipelines

From @fbricon on May 2, 2017 15:22
After deleting a space, I still have a zombie pipeline still trying to build. It's possible the pipeline state is in a borked state that prevents it from being deleted, I have no idea, but in that case the deletion process should be more robust.
Copied from original issue: openshiftio/openshift.io#132

I guess this is a UX question; should deleting a space delete all Pipelines & Apps in that space? Or move them to some other space? Maybe the space deletion wizard needs to present the user with that choice?
Delete space cheese

Delete 3 pipelines too? Y/N
Delete 2 applications in Stage and Run Y/N?

etc?

Or the simple solution - don't allow a Space to be deleted until you delete all the Pipelines + Apps?

@fabric8io/uxd @catrobson please review for design of deleting spaces.
@jstrachan do we have an API to remove pipelines and apps?

Yeah, we've all the APIs we need in the openshift REST API.
Once we get the UX figured out I can implement it pretty easily. I might add a basic option for now with a couple of checkboxes & we can iterate with the UX

sounds good, we need this and can adjust when the designs come in

Adding @qodfathr to the thread to get some input on expectations when deleting a space.  There are many things that a space contains that could either be migrated to another space or deleted upon deleting a space.  Is there an expectation around this from the PM standpoint on what we should be supporting?

related to #1252 #1197 #1251

There are a lot of considerations to be made here, and many equally valid and competing use-cases. We'll need to do some UX research, but, based upon even some back of the napkin use-casing, I feel that we essentially need to offer the user a list of everything which is associated with the Space (pipelines, repos, etc.), each with a checkbox. The checkbox controls "delete this thing, too."
Simple use case A:
I make a new space "foo", use a quickstart, and now have both a "foo" repo and a "foo" pipeline. Then I decide it was a mistake to make it. I likely want to delete everything with one operation.
Simple use case B:
I make a new space "foo" and import an existing "foo" repo. A pipeline is created for me. The I decide it was a mistake to create it. I likely want to delete the space and perhaps the pipeline, but definitely not the repo. (Because it was pre-existing.)
Maybe the logic should be a set of checkboxes, with the checkboxes defaulted to checked (meaning "delete this, too") for those items which were created by OSIO? In other words, in use case A, everything would be defaulted to checked -- Space, Pipeline and Repo. But in use case B, only Space and Pipeline would be checked by default.

Issue was originally logged here:  https://github.com/fabric8-ui/fabric8-ui/issues/1251

Closing 1251 as a dup of this issue.

This issue was moved to openshiftio/openshift.io#915


⚠ Adacta-fintech.com has degraded performance

In 535633d, Adacta-fintech.com (https://www.adacta-fintech.com) experienced degraded performance:

HTTP code: 200
Response time: 1704 ms

Resolved: Adacta-fintech.com performance has improved in 49b8ff4.


Arch linux having trouble

There is something going very wrong with arch linux. Maybe there is something wrong with the kernal? Here is a picture after you boot it up:

Thanks for the report. I can reproduce this, will be fixed in the next release in a couple of days.

Fixed. Let me know if the problem occurs again.


Enabling legacyMode and pingInterval together leads to reproducible crash

Creating a redis client with legacyMode: true AND a non-zero pingInterval leads to a 100% reproducible crash.
Demonstration:
'use strict';

const redis = require('redis');

(async () => {
    const clientOptions = {
        url: 'redis://:foobar@localhost:6379/',
        legacyMode: true,
        pingInterval: 3000
    };

    const redisClient = redis.createClient(clientOptions);

    redisClient.on('error', error => {
        console.log(`redisClient: error ${error.stack}`);
    });

    await redisClient.connect();

    while (true) {
        console.log(new Date().toISOString());
        await new Promise(resolve => {
            setTimeout(resolve, 1000);
        });
    }
})();

Output:
2023-01-20T01:27:38.215Z
2023-01-20T01:27:39.216Z
2023-01-20T01:27:40.218Z
.../node_modules/@redis/client/dist/lib/client/index.js:434
            .then(reply => this.emit('ping-interval', reply))
            ^

TypeError: Cannot read properties of undefined (reading 'then')
    at Timeout._onTimeout (.../node_modules/@redis/client/dist/lib/client/index.js:434:13)
    at listOnTimeout (node:internal/timers:564:17)
    at process.processTimers (node:internal/timers:507:7)

Node.js v18.13.0

https://github.com/redis/node-redis/blob/master/packages/client/lib/client/index.ts#L365-L368 seems to assume .ping() always returns a promise.
Environment:

Node.js Version: 18.13.0
Redis Server Version: 7.0.7
Node Redis Version: 4.5.1
Platform: Mac OS 13.1 (Intel)

@benjie I manually applied your patch to my local install and confirmed it fixes the issue.
Thanks for the quick response!

No worries, I just happened to be hit by the same issue as you just 12 hours later, by pure coincidence!

@keithmo @benjie thanks for sharing and even fixing the issue! :)
wanna open a PR or should I copy your fix? ;)


Update credential-guard-manage.md

Customer pointed out that while they were following steps outlined in the article, they ran into some errors. Validation revealed that step 3 in the feature installation procedure is outdated as of Windows 10 1607. Added a note about that

@DulceMontemayor The PR that introduces a clarifying note to an article Windows Defender Credential Guard that is created by a MS contributor is available for review. Thanks.

@DulceMontemayor Following up for updates.
cc @Dansimp

@Dansimp The PR that has been approved by the document author can be merged. Thanks.

@JohanFreelancer9 : Just a minor thought, would it be useful to highlight the keywords in the note in bold, or is it enough that they are capitalized already?
example:

In Windows 10, version 1607 and later, the Isolated User Mode feature has been integrated into the core operating system. Running the command in step 3 above is therefore no longer required.

(For all I know, the focus may be clear enough already. I just thought it would be appropriate to ask.)

@illfated Good question, but I would be very happy to leave it as it is, as the note is purely informative.

That is as good an answer as any. I concur with your assessment.

Still under escalation.

@Dansimp The PR that has been approved by the document author can be merged. Thanks.

@MSFTandrelom We'd like to merge your contributions. Would you change the PR merge target from master to public? Thanks.


cannot get through the error "Do not change code above the line" in the task checklist.

Challenge Storing Values with the Equal Operator has an issue.
User Agent is: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.111 Safari/537.36.
Please describe how to reproduce this issue, and include links to screenshots if possible.
My code:
// Setup
var a=7;
var b = a;

// Only change code below this line

I'd suggest not changing code above the // Only change code below this line line in order to pass the Do not change code about the line test. Thanks and happy coding!

Hi everyone,
I've tried both ways still won't let me pass challenge... maybe something wrong?

I just noticed that you must only add code and not change it.
//Only change code below this line
var a = 7;
var b = a;

// Setup
var a;
var b = 2;
// Only change code below this line
a = 7;
b = a;


Add kustomize to go image

superseded by https://github.com/replicatedhq/kotsadm/commit/2f8b727b38d3995b46c4b92ed0cabf367f8e6059


Way to query GPU UUID or S/N?

Is there a way to query a per-GPU identifier, such as a GUID, UUID or serial number?

Anything succeeded in this direction?

A per-gpu identifier would help us much.

Sorry that I am late to the show. The KFD supports this with /sys/class/kfd/kfd/topology/nodes/X/gpu_id, which is a decimal representation of the PCIe location of the device. This will be constant as it's based on the physical PCIe location, not on DRM enumeration, and is unique even for multi-GPU cards

Thanks, but it's not quite that, you need a unique GPU identifier, then what you offer can be received and that's right.
hwinfo --gfxcard | grep -n2 "PCI" | grep "Unique ID:" | awk '{print $ 4}'

I am working on providing an actual Unique ID instead of just using that PCIe location above (which was mediocre at best). It will hopefully make it into 2.5

@kentrussell Can you please re-open this issue since it isn't resolved? This is something I'd definitely like access to as well!

It will be in 2.5, but I can reopen until that comes out.

I unfortunately missed the 2.5 cutoff for the SMI, but I plan to upload the changes to the krussell/fixes branch once 2.5 is released. The kernel will have the required code, the SMI tie-in code will be missing, but you can just grab the krussell/fixes branch (once 2.5 is released) and just drop the rocm_smi.py file into your system and you're golden. Sorry for the workaround, but at least we'll have something we can do in the interim.

What about an API for OpenCL?

If someone in OpenCL wants to write it, they can go right ahead. It's just referencing a sysfs file called unique_id , so it's fine if someone wants to. There is also the SMI Lib, which will reference it in either 2.5 or 2.6.

./devices/pci0000:00/0000:00:17.0/ata1/host0/scsi_host/host0/unique_id
./devices/pci0000:00/0000:00:17.0/ata6/host5/scsi_host/host5/unique_id
./devices/pci0000:00/0000:00:17.0/ata4/host3/scsi_host/host3/unique_id
./devices/pci0000:00/0000:00:17.0/ata2/host1/scsi_host/host1/unique_id
./devices/pci0000:00/0000:00:17.0/ata5/host4/scsi_host/host4/unique_id
./devices/pci0000:00/0000:00:17.0/ata3/host2/scsi_host/host2/unique_id

But cat on one of those id's is just a simple number like 1.

What does "cat /sys/class/drm/card0/device/unique_id" return? Note also that this is only supported on GFX9+

0
This is on a machine running [AMD/ATI] Vega 10 [Radeon RX Vega] (rev c3)
But it also has 12 gpu's and that output only has 6 results, so maybe it
isn't related?
jon
On Tue, Jun 4, 2019 at 8:28 PM Kent Russell<EMAIL_ADDRESS>wrote:

What does "cat /sys/class/drm/card0/device/unique_id" return? Note also
that this is only supported on GFX9+
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/RadeonOpenCompute/ROCm/issues/310?email_source=notifications&email_token=AAAU224LCVHNW3R2IMJ6M6DPYZUY7A5CNFSM4ENML75KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODW4SARA#issuecomment-498671684,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAU223T5ITQWZLTRZP5U3LPYZUY7ANCNFSM4ENML75A
.

Yeah, that seems peculiar. You're using 2.5, so does dmesg say anything after trying to do the cat command?

root@:~# dmesg | tail -1
[73290.480420] perf: interrupt took too long (4995 > 4927), lowering kernel.perf_event_max_sample_rate to 40000
root@Vega0002:~# cat /sys/devices/pci0000:00/0000:00:17.0/ata1/host0/scsi_host/host0/unique_id
1
root@:~# dmesg | tail -1
[73290.480420] perf: interrupt took too long (4995 > 4927), lowering kernel.perf_event_max_sample_rate to 40000

Ok if the command that I gave further up doesn't give any results, the unique_id isn't present on the system for the gpu. The path that you list there is for the SATA devices, not for the gpu.
Do you have the 2.5 package installed. I don't know when they are actually releasing it, or if it's out already.

I missed the 2.5 cutoff for the SMI, but I have uploaded the patches to the SMI's "krussell/fixes" branch. You can use that SMI with the 2.5 release to get the unique_id for the GPUs. But the 2.5 kernel needs to be installed for it to work, 2.4 didn't have the required sysfs file

UUID is available in ROCm. 2.10 will have support for serial number, but only from server Vega20 cards. There is a specialized chip on the server cards that stores the serial number, unfortunately it is not present on consumer cards, which is frustrating. But UUID works for all AMD GPUs

@kentrussell could you please explain the difference between UUID and serial-number in this context?

Do I understand correctly that for RadeonVII (Vega20) what we have is serial-number?
What do you mean by "UUID is available in ROCm"?

In order to expose the serial in OpenCL, from what I understand, an extension would need to be defined and added to clGetDeviceInfo() (similar to the existing CL_DEVICE_GLOBAL_FREE_MEMORY_AMD), e.g.
CL_DEVICE_SERIAL_AMD or CL_DEVICE_ID_AMD
And the whole thing submitted to Khronos for approval, I imagine AMD could initiate that if desired.

UUID is a universally unique ID, but is NOT the serial number printed on the sticker on the back of the card. Unfortunately the serial number can only be read on server cards due to the configuration of the card itself, so the Radeon VII will not have this functionality (tentatively planned for ROCm 3.0 now). UUID will never change, but it will not match the serial number printed on the sticker on the back of the card. I implemented the UUID as a solution for people who wanted the serial number on consumer cards, as it was the best option we had available for uniquely identifying consumer cards.

Thanks, but it's not quite that, you need a unique GPU identifier, then what you offer can be received and that's right.
hwinfo --gfxcard | grep -n2 "PCI" | grep "Unique ID:" | awk '{print $ 4}'

"hwinfo --gfxcard" output, UUID does look like this, with a dot:
Unique ID: IluS.msy7ZveCwh5

That's a different one from what I'm talking about. You can do "cat /sys/class/drm/card0/device/unique_id" to get the one that I'm referring to (provided that your card is card0, if you have multiple cards, replace the 0 with a * to get all unique IDs). It's hard-coded to the card itself and doesn't rely on any software to generate it, thankfully (just to expose it). You can also use the SMI to get the ID via "rocm-smi --showuniqueid"

That's a different one from what I'm talking about. You can do "cat /sys/class/drm/card0/device/unique_id" to get the one that I'm referring to (provided that your card is card0, if you have multiple cards, replace the 0 with a * to get all unique IDs). It's hard-coded to the card itself and doesn't rely on any software to generate it, thankfully (just to expose it). You can also use the SMI to get the ID via "rocm-smi --showuniqueid"

The best way to confuse people: having multiple "Unique IDs".

Totally agreed. HWINFO generates their own UUID, They don't use the functionality that I implemented because they didn't know that it existed, or they just implemented their version far before our implementation (which I assume is the case). I don't know how they generate their UUID unfortunately, as I am still searching through the opensuse/hwinfo github repo to see where they get their UUID from.
I am sure that hwinfo's UUIDs are all consistent with one another, and ours are consistent with one another, unfortunately they don't quite align as you've noticed. I am pretty sure HWINFO's UUID was implemented far before ours was, ours is just something we can expose that's hard-coded on the GPU itself. So as long as you use the same tools, you're set, but mixing-and-matching doesn't seem to align as nicely as we would like.

This is a thing to remember when selecting tools, it is preferable to rely on UUID that is hard-coded on the gpu than on UUID that is generated by the tool and with a different format. One thing that could be useful is API access to the UUID so that programs can retrieve it.

Exactly. The rocm_smi_lib provides a C/C++ interface which accesses this UUID (see https://github.com/RadeonOpenCompute/rocm_smi_lib/blob/master/src/rocm_smi.cc#L2368). Plus it's part of the sysfs pool so it can be accessed by any tool that can read files, which helps too.  It's up to the various tools guys to implement it.
@valeriob01 Thanks for the advice suggesting using the same tools where possible. While UUID is Universally Unique, there is no convention requiring that everyone's UUIDs are identical. A slight downside with having multiple tools developed my multiple companies across the globe.

Exactly. The rocm_smi_lib provides a C/C++ interface which accesses this UUID (see https://github.com/RadeonOpenCompute/rocm_smi_lib/blob/master/src/rocm_smi.cc#L2368). Plus it's part of the sysfs pool so it can be accessed by any tool that can read files, which helps too. It's up to the various tools guys to implement it.
@valeriob01 Thanks for the advice suggesting using the same tools where possible. While UUID is Universally Unique, there is no convention requiring that everyone's UUIDs are identical. A slight downside with having multiple tools developed my multiple companies across the globe.

Problem in multi-gpu systems: unique_id for card0 is empty. Files missing.
Other cards work fine...
card0 : empty
card1,2,3 : not empty.

@valeriob01 That's definitely peculiar since they're all the same card. Does dmesg say anything regarding card0 (try attaching the full dmesg here for me to take a look, as well as the full "rocm-smi -a --alldevices" output)?  If there are different sysfs files like that, then maybe there's something going on with that card. Maybe dmesg/SMI output will help me figure out what's up

Thanks for look in At The issue.
Il Lun 3 Feb 2020, 14:40 Kent Russell<EMAIL_ADDRESS>ha scritto:

@valeriob01 https://github.com/valeriob01 That's definitely peculiar
since they're all the same card. Does dmesg say anything regarding card0
(try attaching the full dmesg here for me to take a look, as well as the
full "rocm-smi -a --alldevices" output)? If there are different sysfs files
like that, then maybe there's something going on with that card. Maybe
dmesg/SMI output will help me figure out what's up
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/RadeonOpenCompute/ROCm/issues/310?email_source=notifications&email_token=AGFEKPQSQ5WNYXRD56G6YNLRBANERA5CNFSM4ENML75KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKT33GY#issuecomment-581418395,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AGFEKPWDZRVJ5J7DD6GKRXDRBANERANCNFSM4ENML75A
.

Thanks for look in At The issue. Il Lun 3 Feb 2020, 14:40 Kent Russell<EMAIL_ADDRESS>ha scritto:
…
@valeriob01 https://github.com/valeriob01 That's definitely peculiar since they're all the same card. Does dmesg say anything regarding card0 (try attaching the full dmesg here for me to take a look, as well as the full "rocm-smi -a --alldevices" output)? If there are different sysfs files like that, then maybe there's something going on with that card. Maybe dmesg/SMI output will help me figure out what's up — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#310?email_source=notifications&email_token=AGFEKPQSQ5WNYXRD56G6YNLRBANERA5CNFSM4ENML75KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKT33GY#issuecomment-581418395>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGFEKPWDZRVJ5J7DD6GKRXDRBANERANCNFSM4ENML75A .

Not a problem. Once you get a chance to attach the dmesg/SMI output, hopefully it will illuminate the issue and help us to get you back to having 4 identical cards :)

Thanks for look in At The issue. Il Lun 3 Feb 2020, 14:40 Kent Russell<EMAIL_ADDRESS>ha scritto:
…
@valeriob01 https://github.com/valeriob01 That's definitely peculiar since they're all the same card. Does dmesg say anything regarding card0 (try attaching the full dmesg here for me to take a look, as well as the full "rocm-smi -a --alldevices" output)? If there are different sysfs files like that, then maybe there's something going on with that card. Maybe dmesg/SMI output will help me figure out what's up — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#310?email_source=notifications&email_token=AGFEKPQSQ5WNYXRD56G6YNLRBANERA5CNFSM4ENML75KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKT33GY#issuecomment-581418395>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGFEKPWDZRVJ5J7DD6GKRXDRBANERANCNFSM4ENML75A .

Not a problem. Once you get a chance to attach the dmesg/SMI output, hopefully it will illuminate the issue and help us to get you back to having 4 identical cards :)

Thanks, the reason is that we use unique_id to identify the card in the program gpuowl for trustworthiness.

Thanks for look in At The issue. Il Lun 3 Feb 2020, 14:40 Kent Russell<EMAIL_ADDRESS>ha scritto:
…
@valeriob01 https://github.com/valeriob01 That's definitely peculiar since they're all the same card. Does dmesg say anything regarding card0 (try attaching the full dmesg here for me to take a look, as well as the full "rocm-smi -a --alldevices" output)? If there are different sysfs files like that, then maybe there's something going on with that card. Maybe dmesg/SMI output will help me figure out what's up — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#310?email_source=notifications&email_token=AGFEKPQSQ5WNYXRD56G6YNLRBANERA5CNFSM4ENML75KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKT33GY#issuecomment-581418395>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGFEKPWDZRVJ5J7DD6GKRXDRBANERANCNFSM4ENML75A .

Not a problem. Once you get a chance to attach the dmesg/SMI output, hopefully it will illuminate the issue and help us to get you back to having 4 identical cards :)

Thanks, the reason is that we use unique_id to identify the card in the program gpuowl for trustworthiness.

That makes sense. If you're using KFD Topology for anything, there is a "location_id" field inside the properties file (e.g. /sys/class/kfd/kfd/topology/nodes/0/properties) which is a decimal representation of the PCI location, so that can always work in a pinch. But the fact that your one card isn't doing what it's supposed to means we should get that figured out, since that GPU might not work properly (based on its behaviour) and thus would mess up some of your gpuowl work.

Thanks for look in At The issue. Il Lun 3 Feb 2020, 14:40 Kent Russell<EMAIL_ADDRESS>ha scritto:
…
@valeriob01 https://github.com/valeriob01 That's definitely peculiar since they're all the same card. Does dmesg say anything regarding card0 (try attaching the full dmesg here for me to take a look, as well as the full "rocm-smi -a --alldevices" output)? If there are different sysfs files like that, then maybe there's something going on with that card. Maybe dmesg/SMI output will help me figure out what's up — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#310?email_source=notifications&email_token=AGFEKPQSQ5WNYXRD56G6YNLRBANERA5CNFSM4ENML75KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKT33GY#issuecomment-581418395>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGFEKPWDZRVJ5J7DD6GKRXDRBANERANCNFSM4ENML75A .

Not a problem. Once you get a chance to attach the dmesg/SMI output, hopefully it will illuminate the issue and help us to get you back to having 4 identical cards :)

Thanks, the reason is that we use unique_id to identify the card in the program gpuowl for trustworthiness.

That makes sense. If you're using KFD Topology for anything, there is a "location_id" field inside the properties file (e.g. /sys/class/kfd/kfd/topology/nodes/0/properties) which is a decimal representation of the PCI location, so that can always work in a pinch. But the fact that your one card isn't doing what it's supposed to means we should get that figured out, since that GPU might not work properly (based on its behaviour) and thus would mess up some of your gpuowl work.

Ah, for that gpuowl works extremely resilient to errors...

do not work on this, serious invasion of privacy
if youre doing fingerprinting this way, youre doing it wrong

Just because @g5543 posted, I had to share this... worked great for me.
atitool -i=0 -sid
ATITOOL version <IP_ADDRESS>, Copyright (c) 2019 Advanced Micro Devices, Inc.

LOT=[REMOVED],WFR=11,WFRX=12,PBI FT=0,PBI WS=0,WFRY=02,FDRY=03,FAB=01,YR=08,WK=34,REV=A1B,CHECKBIT0=0,CHECKBIT1=0,SerialID Data = [UNIQUE ID HERE]
LeakageValue = 6.139336 A,LeakageID = [REMOVED]

@kentrussell @gstoner
Any change the UUID is obtainable from Windows?


6.51 hotfix (11/14 update) opcode

{
"C2S_ActionRequest": "0x03ab",
"C2S_ActionRequestGroundTargeted": "0x02a2",
"Common_UseOodleTcp": true,
"PatchCode": [],
"S2C_ActionEffect01": "0x019f",
"S2C_ActionEffect08": "0x03a2",
"S2C_ActionEffect16": "0x0126",
"S2C_ActionEffect24": "0x0087",
"S2C_ActionEffect32": "0x0248",
"S2C_ActorCast": "0x0107",
"S2C_ActorControl": "0x03b6",
"S2C_ActorControlSelf": "0x00e2",
"Server_IpRange": "<IP_ADDRESS>/0",
"Server_PortRange": "1-65535"
}

Legit can vouch for

I can confirm it too.

It's impressive how quickly ppl can provide the opcode after new patch comes in. TY so much <3

Thanks! And here I thought that for some reason XIV did it automatically haha


Fixed Selective Reporting Parameter Labels for AEON HEMs

Amps are a measurement of current.  Watts are a measurement of power.  These parameters are all measuring watts or percentage of watts change so the label must be power especially since there are other reports that deliver current in amps messages.

I don't know what a "Semantic PR title / commit message" is.  If somebody wouldn't mind doing an edit so I can get in and out and contribute a fix without having to learn all of your culture.

I'll give you that this is true and should be fixed. A lesson in not trying to remember high school electronics class material at 2 AM.


Icon Request: Card processors

Icon Request

Icon name: mastercard, visa, amex, diners club, maestro, direct debit
Tags of this icon: mastercard, maestro, visa, amex, diners-club, direct-debit
Use case: For websites or UIs that have some sort of card processing
Screenshots of similar icons:

Sorry, I only added Visa and Mastercard icons for now.
If any other icons of bank card are very necessary for you, you can comment back and we can talk about it.

Thank you for some very high quality work on these really do appreciate it, however I would still add a few more brands as alot of websites, dashboards and even apps accept payments and use icons to show types of payments supported this is why I would like to request the following to be added:

Discover
AMEX
Diners Club
Maestro

To further add to my point about these other icons being necessary here is a example image of what a website shows regarding accept card types:

If we make the Discover logo an icon, we should make it fit in a 24px grid. However this logo is not a square, so it would look littler than other logos even they were given the same size.

Maestro looks just the same with Mastercard. Remix Icon is a library without color, but how can we make them look different without given a color?

For discover, we could perhaps have something like this:

For Maestro this is a little unorthodox but we can change Mastercard to something like a card

And change existing Mastercard icon to Maestro to tell the difference perhaps?

Thanks for re-opening this could you also please add contactless to the list


changed default image to old

Description

Image widget - default is large, now replaced with the old default

Fixes #6427
Type of change

Bugfix (non-breaking change which fixes an issue)

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[ ]  I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[ ]  I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes

/ok-to-test sha= 58d740d


Add header guards to projects/ header files

ex app.h, bindings.h
~See other headers (like in shared/util) to copy the guad naming style~
Convert all header guards to #pragma once directives and add it to the headers which do not yet have a guard

From my understanding, to the 3 projects in the projects/ directory you want all .h files to have the #pragma once directive, does that also include the empty .h files for example the one under firmware/projects/DemoMapper/inc/app.h ?

You might as well do it for DemoMapper as well.


[BUG]: getSQL on subquery not working

What version of drizzle-orm are you using?
^0.30.1
What version of drizzle-kit are you using?
No response
Describe the Bug
The following code does not work:
const inner = db
  .$with("inner")
  .as(
    db
      .select()
      .from(someTable)
  );

const s = sql.empty();
s.append(inner.getSQL());

Currently when running this code I receive the following error:
TypeError: inner.getSQL is not a function
Expected behavior
getSQL to be implemented to avoid the runtime error.
Given their is a type definition for it, it seems unintentional this method is missing at runtime.
Environment & setup
No response

Thanks for all your work on Drizzle ORM. It's such a great tool!

Hmm, maybe the intention is for the user to do:
const inner = db.select().from(someTable);

const inner = db.$with("inner").as(inner);

const s = sql`with ${inner2} as ${inner} ...;`,

Regardless I kinda feel like maybe the getSQL should be implemented or removed but could fully understand if that's just required for the types to work.


Template string args don't resolve to correct overload

TypeScript Version:  1.8.0 / nightly (2.0.0-dev.201xxxxx)
Code
let frame = 0;
window.addEventListener(`keydown`, e => {
  frame = e.which - 48;  // error: Property 'which' does not exist on type 'Event'
}, false);

Expected behavior:
It should work like a string literal does and resolve to the correct overload.
Actual behavior:
It doesn't resolve correctly so it type checks based on the most general overload.

A while back I asked if no-substitution template strings should get the same treatment as regular strings. We opted not to, but I think seeing the way that JS users have been using them in lieu of standard string literals, we should consider this again.
This means (at least) that:

String literal-typed template strings need to be made a thing.
Computed property names need to work for templates.

seems this issue has been resolved via latest version


State of the field is missing

While the purpose of the package is clear (reimplementing a R/C++ code), I think it would benefit from a broader perspective on the state of ML and interpretability in Julia.
For instance explain what MLJ is, whether there are concurrent frameworks, and which packages offer interpretability tools (even based on black box models).
https://github.com/openjournals/joss-reviews/issues/5786

This was indeed a big issue. We had failed to mention many packages. Thank you for reporting it and providing some suggestions. In the latest PDF version in the REVIEW issue, we have tried to find a balance between your suggestions here and the original checklist item in the JOSS review checklist. Can you check whether the current PDF version has solved this issue?


Randomize traffic across redundant backends for destHost traffic

This is  a necessary work around until a lock condition is fixed in konnecitivity under:
https://github.com/kubernetes-sigs/apiserver-network-proxy/issues/261
It is still under investigation but there is a situation when only one agent is available with destHost specified and a specific number of parallel requests go through it concurrently that the agent/server connection can lock up and stop processing traffic.
this resolves the problem in the interim by load balancing across instances (since we have 3 effectively makes the liklihood of hitting the condition small and in practice have not seen it in our large scale performance testing.)

/lgtm

/approve

/approved

/approve

/approve

/merge

/help


Avoid re-sending old position commands

Sister PR to https://github.com/tasts-robots/vulp/pull/26

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 51.121%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
114

Relevant Lines:
223

💛  - Coveralls


Convert to collection

Hello,
Your plugins (especially the filters) are very convenient and generic.
Do you have a plan to convert this repository to an Ansible collection? This would mostly mean creating a galaxy.yml file and reorganizing your repository structure to comply with the collection file organization.
Anyways, thanks for the work!

Thank you for the comment! Yes, I'd like to contribute it. In fact, I've already contributed the filer lists_mergeby to community.general. Would you like to cooperate on a collection? Such a project could serve the purpose of creating, testing, and submitting filters to comunity.general. There is an opportunity of creating efficient filters to manipulate data-structures specific to system management.
My capacity is limited at the moment. I'm working mostly on building FreeBSD.* collection. Feel free to propose how to proceed.


完善issue#978中对java.sql.Date的支持

在FieldSerializer.java的getPropertyValue()方法中判断Date类型时新增了对java.sql.Date的判断

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@dfdb59f). Click here to learn what that means.
The diff coverage is 0%.

@@            Coverage Diff            @@
##             master    #2705   +/-   ##
=========================================
  Coverage          ?   80.41%
  Complexity        ?     7095
=========================================
  Files             ?      145
  Lines             ?    26708
  Branches          ?     6374
=========================================
  Hits              ?    21477
  Misses            ?     3391
  Partials          ?     1840

Impacted Files
Coverage Δ
Complexity Δ

...m/alibaba/fastjson/serializer/FieldSerializer.java
79.41% <0%> (ø)
62 <0> (?)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dfdb59f...5446e01. Read the comment docs.


WASAPI: Exclusive mode support?

Hi there,
would it be possible for Cubeb to offer using WASAPI in exclusive mode?
Or would there be interest in such a feature being implemented?

We'd take a patch I'm sure. We recently got a new flag system that we could use to implement this in fact (talking in terms of API).

Tried to create a PR for it. #415 (A terrible one :/)


Rethink mipmap generation

Our previous approach was to have generate_mipmap method on the Factory. It doesn't fit DX11 model, since even though there is GenerateMipmap method, it is a rendering operation, belongs to ID3D11DeviceContext, thus can't be executed by a factory, which only has ID3D11Device.
It seems straightforward to just move the mipmap generation into the CommandBuffer interface, since this is where our rendering is done. However, this doesn't work well for the major use-case - when we have the mip[0] contents right away and we want to upload it with all the mipmaps generated. Not only this is inconvenient, it also involves blowing up the data buffer, associated with the command buffer on GL and non-deferred DX11. This means - time wasted on the copy and allocation.
As if it's not difficult enough already, DX12 doesn't have any mechanism to assist in mipmap generation (see #905). You are supposed to do it manually. We can, of course, implement some rendering inside the backend, but this rises the question - should we support mipmap generation at all?.. Once you start being serious about graphics, you end up loading all mipmaps from disk in some optimized GPU-friendly format anyway, since generating them takes time.

My understanding also is that the drivers are not always good about high-quality mipmap generation.
I guess the image crate should just handle the downscaling, and we should only support uploading fully specified textures?

Problem with doing this on the image side is that it's going to be much slower, obviously. However, I do like this approach as a middle ground - for users that already care about smooth mipmap transitions but do not care enough to actually pack their textures into a proper format with mipmaps included. So I take it your (@cmr) voice is for removing the support for generating mipmaps, and I like it :)
If we think about how to still provide some automatic generation for the backends that can do it, maybe we can have command buffer (or even factory?) extensions like:
trait MipmapCommandBuffer {
 fn generate_mipmap(...);
}

Then some backends will implement it, a the user code can put a clause like where C: gfx::MipmapCommandBuffer where they want to use it.
This approach can be extended for other features that we don't strongly support in the core, while still wanting to provide.

Could that extension be implemented with some sort of compatability fallback for DX12? For example, it could expand into commands that use compute shaders to generate the mipmaps, which should be much much faster than image.

I guess that line of thinking leads to generalized middleware architectures for gfx?

@cmr it totally can be implemented in the backend itself, even if it doesn't support mipmap generation natively. However, that bring us to the problem I expressed initially - that texture uploading would have to go through the command/data buffer, because we can't create a texture with partially initialized contents, on DX11 at least. (That was probably unclear, I updated the issue description)

Since we are waiting for winapi-rs anyway, I've added generate_mipmap() to our CommandBuffer interface. We'll probably change it later when new backends are introduced, but for GL+DX11 is fairly straightforward.


elba build fail when depending on C FFI

Hi there,
I have a project which compiles a .c source and exposes it in idris using %include C and %link C.
I used a script in order to compile the file and output the object file like so:
[scripts]
"prebuild" = "cc -O2 -c -o array.o array.c `idris --include`"

However elba build fails with the following error:
> "idris" "--check" "Data/Bytes.idr" "Data/ByteArray.idr"
 user error (Can't find file array.o)

Interestingly enough compiling in the current folder with
idris --check Data/Bytes.idr Data/ByteArray.idr

works perfectly fine.
Am I missing something or is this use case not supported?

I've found the cause of the problem.
First, the build process of elba takes place at /target/build/lib so before that elba have to copy the source into it. Second, elba will exclude all files defined by .gitingore, in which *.o is declared.
I've opened an issue for it: https://github.com/elba/elba/issues/55


dired says hit 'g' to refresh but that does something else

What did you expect to happen?
dired should refresh or it should give the correct key sequence to refresh
What actually happened?
it brought up the 'g' leader key menu
Additional details:

Include a link to your private config
I don't think this is necessary because I just installed and there's nothing remarkable in there.

Include screenshots/casts of your issue

also shouldn't be necessary because everyone with a somewhat generic install should have the same issue (and with my new minimal window manager I haven't figured out how to do this yet)

If you mention key sequences, include what commands they're bound to

inspecting keys does no good because the 'g' is a leader key and doesn't result in a complete key sequence

Include any warnings or errors logged to *Messages* (use M-x view-echo-area-messages to see it).

no warnings

If an error message is involved include a backtrace of it.
no error message
How to acquire a backtrace:
https://github.com/hlissner/doom-emacs/blob/develop/docs/getting_started.org#how-to-extract-a-backtrace-from-an-error

Steps to reproduce:
I had a bunch of PDF files left over from windows who's names were upper case but they had internal links to each other in lower case so they didn't work in Linux.   In order to try and get these running I marked them all then did a M-x dired-downcase command on them.   Some of the file names didn't get updated in the buffer (not sure how one could reproduce this as it was probably a glitch).  I tried to go into one of the still upper case ones to see what was going on and it told me the file was no longer there and I should hit 'g' to refresh it.   When I did it just gave me the 'g' leadin key menu.
System information:

emacs   version    26.2
        features   XPM JPEG TIFF GIF PNG RSVG SOUND GPM DBUS GLIB NOTIFY ACL GNUTLS LIBXML2 FREETYPE XFT ZLIB GTK3 X11 XDBE XIM THREADS LIBSYSTEMD LCMS2
        build      Jun 29, 2019
        buildopts  (--prefix=/usr --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib --disable-silent-rules --docdir=/usr/share/doc/emacs-26.2 --htmldir=/usr/share/doc/emacs-26.2/html --libdir=/usr/lib64 --program-suffix=-emacs-26 --includedir=/usr/include/emacs-26 --infodir=/usr/share/info/emacs-26 --localstatedir=/var --enable-locallisppath=/etc/emacs:/usr/share/emacs/site-lisp --without-compress-install --without-hesiod --without-pop --with-file-notification=inotify --enable-acl --with-dbus --without-modules --without-gameuser --with-gpm --with-kerberos --with-kerberos5 --with-lcms2 --with-xml2 --without-mailutils --without-selinux --with-gnutls --with-libsystemd --with-threads --without-wide-int --with-zlib --with-sound=alsa --with-x --without-ns --without-gconf --without-gsettings --without-toolkit-scroll-bars --with-gif --with-jpeg --with-png --with-rsvg --with-tiff --with-xpm --without-imagemagick --with-xft --without-cairo --without-libotf --without-m17n-flt --with-x-toolkit=gtk3 --without-xwidgets 'CFLAGS=-O2 -march=x86-64 -pipe' CPPFLAGS= 'LDFLAGS=-Wl,-O1 -Wl,--as-needed')
        windowsys  x
        daemonp    server-running
doom    version    2.0.9
        build      HEAD -> develop, origin/develop, origin/HEAD 4b20c7206 2020-03-13 17:34:44 -0400
        dir        ~/.doom.d/
system  type       gnu/linux
        config     x86_64-pc-linux-gnu
        shell      /bin/bash
        uname      Linux 5.4.0-sabayon #1 SMP Tue Feb 25 22:46:38 UTC 2020 x86_64
        path       (~/bin ~/sbin ~/.npm-global/bin ~/.emacs.d/bin /usr/local/sbin /usr/local/bin /usr/sbin /usr/bin /sbin /bin /opt/bin /usr/lib/llvm/9/bin /usr/games/bin /usr/libexec/emacs/26.2/x86_64-pc-linux-gnu/)
config  envfile    envvar-file
        elc-files  0
        modules    (:completion company ivy :ui doom doom-dashboard doom-quit hl-todo modeline nav-flash ophints (popup +defaults) vc-gutter vi-tilde-fringe window-select workspaces :editor (evil +everywhere) file-templates fold snippets :emacs dired electric vc :checkers syntax spell :tools (eval +overlay) lookup magit :lang clojure csharp data emacs-lisp fsharp javascript latex factor markdown (org +dragndrop +present) racket scheme sh :config (default +bindings +smartparens))
        packages   (n/a)
        unpin      (n/a)
        elpa       (n/a)

FYI you can update dired with gr, when using evil.

it brought up the 'g' leader key menu

Not the leader key, but the g prefix in normal mode, which has a number of sub bindings.
In any case, we are using the evil-collection package, which provides "evilified" keybinds for a number of packages, including dired. It binds gr to revert-buffer here, which is what you want. You'll find other keybinds there too.
Hope that helps!


micronaut: 4.5.1 -> 4.6.1

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/micronaut-projects/micronaut-starter/releases.
meta.description for micronaut is: Modern, JVM-based, full-stack framework for building microservice applications
meta.homepage for micronaut is: https://micronaut.io/
Updates performed

Version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 4.6.0 with grep in /nix/store/gs89107ihwx5zp9y0ahdizp5k15ddkn9-micronaut-4.6.0
found 4.6.0 in filename of file in /nix/store/gs89107ihwx5zp9y0ahdizp5k15ddkn9-micronaut-4.6.0

Rebuild report (if merged into master) (click to expand)

2 total rebuild path(s)

2 package rebuild(s)

First fifty rebuilds by attrpath

micronaut

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/gs89107ihwx5zp9y0ahdizp5k15ddkn9-micronaut-4.6.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A micronaut https://github.com/r-ryantm/nixpkgs/archive/472581ed80e2172a43872e6211449b28206f06f5.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/472581ed80e2172a43872e6211449b28206f06f5#micronaut

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/gs89107ihwx5zp9y0ahdizp5k15ddkn9-micronaut-4.6.0
ls -la /nix/store/gs89107ihwx5zp9y0ahdizp5k15ddkn9-micronaut-4.6.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

micronaut

Maintainer pings
cc @moaxcp for testing.

[!TIP]
As a maintainer, if your package is located under pkgs/by-name/*, you can comment @NixOS/nixpkgs-merge-bot merge to automatically merge this update using the nixpkgs-merge-bot.

Add a :+1: reaction to pull requests you find important.

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/micronaut-projects/micronaut-starter/releases.
meta.description for micronaut is: Modern, JVM-based, full-stack framework for building microservice applications
meta.homepage for micronaut is: https://micronaut.io/
Updates performed

Version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 4.6.1 with grep in /nix/store/zhzx8fsqx16icdw4fjcqlxpv9gq7a6lg-micronaut-4.6.1
found 4.6.1 in filename of file in /nix/store/zhzx8fsqx16icdw4fjcqlxpv9gq7a6lg-micronaut-4.6.1

Rebuild report (if merged into master) (click to expand)

2 total rebuild path(s)

2 package rebuild(s)

First fifty rebuilds by attrpath

micronaut

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/zhzx8fsqx16icdw4fjcqlxpv9gq7a6lg-micronaut-4.6.1 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A micronaut https://github.com/r-ryantm/nixpkgs/archive/bbb4f627d26d5a15e9c9f88bcc48de40aca79bcc.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/bbb4f627d26d5a15e9c9f88bcc48de40aca79bcc#micronaut

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/zhzx8fsqx16icdw4fjcqlxpv9gq7a6lg-micronaut-4.6.1
ls -la /nix/store/zhzx8fsqx16icdw4fjcqlxpv9gq7a6lg-micronaut-4.6.1/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

micronaut

Maintainer pings
cc @moaxcp for testing.

[!TIP]
As a maintainer, if your package is located under pkgs/by-name/*, you can comment @NixOS/nixpkgs-merge-bot merge to automatically merge this update using the nixpkgs-merge-bot.

Add a :+1: reaction to pull requests you find important.


GHA Workflow for running tests under a more strict memory checking environment

[X] I agree to follow the project's code of conduct.
[X] I added an entry to CHANGES.md if knowledge of this change could be valuable to users.

If I've configured this correctly, once sanitizer.yml is on the master branch, we should be able to manually initiate running of this workflow on PRs via the workflow_dispatch trigger.
Notes:

Makes use of cargo-careful
Among other things, cargo-careful enables AddressSanitizer, whose options can be configured via ASAN_OPTIONS

bors +r

bors r+

@lnicola Thank you for your thorough review and support. I got what I was hoping for 😄:


feat: UniversalFactory

What does this PR introduce?

This PR introduces a new contract UniversalFactory that should be used to deploy contracts deterministically.
This contract gives the ability to deploy normal and EIP 1167 minimal proxy contracts, with the ability to specify if the contract should be initialized or not by passing the initializeCallData bytes.
Deploying and initializing in one tx eliminate the risk of an initialize race (someone initializing before the right deployer).
If the UniversalFactory is deployed using nick's method on several chains, given the same salt and the same parameters, the contract created will have the same address on each chain.

This method is especially useful for UniversalProfiles, where you will have the ability to reproduce the same UniversalProfile at the same address on each chain you deploy to.

we need tests as well, do we have some?

@frozeman tests are here: https://github.com/lukso-network/lsp-smart-contracts/blob/3b74798107d7f2cee4497a2e6d62ed8b61accfb8/tests/Factories/UniversalFactory.test.ts


[App]: wandb hover: Waiting for W&B process to finish... (success).

Current Behavior
wandb is stuck, it's always been like this
wandb: Waiting for W&B process to finish... (success).

Expected Behavior
No response
Steps To Reproduce
No response
Screenshots
No response
Environment
OS:
Browsers:
Version:
Additional Context
No response

Hi @PussInCode ,
Could you share the debug.log and debug-internal.log files associated to an affected run? They should be present on your machine in the wandb folder relative to your working directory and should contain some more information about these runs so that we can dig in deeper.

Hi @PussInCode, since we have not heard back from you we are going to close this request. If you would like to re-open the conversation, please let us know!

Hi, I'm experiencing the same issue.
I'm running an hyperparameters sweep and after each run (made of 10 epochs) the process stucks on "wandb: Waiting for W&B process to finish... (success)."
I noticed that it is just very very slow at uploading data to the cloud dashboard, like 15-20 minute per run, much than it takes for the train.
This happens on my local computer using pure python and Jupiter notebook but also using wandb command line and config.yaml.
The same thing doesn't happen when I use Google Colab.

I also have the same issue, I init the run with
run = wandb.init(project=PROJECT, sync_tensorboard=True, job_type='training')
and end with
run.finish()
but it doesn't end and outputs
Waiting for W&B process to finish... (success).
while still running.
I'll upload my debug.log and debug-internal.log (the latter is updating while finish() is running)
debug.log
debug-internal.log

same issue occurs

same issue

Same issue

Same issue

Same issue

same issue

The same issue here.

The same issue here.
It seems that the tfevents file can not be sent to the server and the client retries 2 more times but waits 15 minutes before between them.

same issue

Same issue.

same issue

same issue

same issue...

I have encountered the same issue on one of my agent machines I use to run sweeps.
It happens not after every sweep end so workaround could be to kill all the hung processes manually. I have executed ps aux | grep python and found following two processes:

python -c from joblib.externals.loky.backend.resource_tracker import main; main(63, False)
python -c from multiprocessing.resource_tracker import main;main(70)

I have killed them using pkill -9 python (although it is better to use kill -9 <PID> for each of them) and the next sweep started without any issues.

Same issue here too. It would be nice to have a verbose output to see the progress of wandb syncing.

Same issue here too.

Same issue here too.

Same issue here too.

Same issue here too.

Same issue. Why is this closed?

I have the same issue. In fact wandb upload tracked information but no avoid to finish my script

Same issue.

same issue

Same issue. Even the offline mode cannot work.

I have a sequence of running trainings and this just hangs the program there, though the correct stats were uploaded to wandb. After checking the debug-internal.log, I have found this: 2023-12-08 17:18:17,913 INFO    HandlerThread:2521165 [system_monitor.py:probe():211] Finished collecting system info 2023-12-08 17:18:17,913 INFO    HandlerThread:2521165 [system_monitor.py:probe():214] Publishing system info 2023-12-08 17:18:17,913 DEBUG   HandlerThread:2521165 [system_info.py:_save_pip():51] Saving list of pip packages installed into the current environment 2023-12-08 17:18:17,916 DEBUG   HandlerThread:2521165 [system_info.py:_save_pip():67] Saving pip packages done 2023-12-08 17:18:17,916 DEBUG   HandlerThread:2521165 [system_info.py:_save_conda():74] Saving list of conda packages installed into the current environment 2023-12-08 17:18:17,924 ERROR   HandlerThread:2521165 [system_info.py:_save_conda():85] Error saving conda packages: [Errno 2] No such file or directory: 'conda' And I also checked other logs previously, they don't have such error; maybe this is what blocks the wandb.finish()?

same issue

I somehow addressed this problem by changing the order of wandb close and run close:
run.finish()
wandb.finish()


CAMEL-7809 : Quartz PollConsumerScheduler in a cluster tries to create duplicate triggers, fails

Remember to run all unit tests when you submit a PR - there was a test failure due your PR. I am fixing this.


Fix TranslatorInterface param types

The first param of the trans interface is not nullable in symfony 5.0, see: https://github.com/symfony/symfony/blob/5.0/src/Symfony/Contracts/Translation/TranslatorInterface.php
Refer to this wrong demo: https://getrector.com/demo/daac0fbf-2e8c-4a62-8379-cae9df8e9de7

Thank you :clap:


When terminal is open the debug console shortcut isn't working

VSCode Version: Code 1.14.2 (cb82febafda0c8c199b9201ad274e25d9a76874e, 2017-07-19T23:26:08.116Z)
OS Version: Linux x64 4.4.0-83-generic
Extensions:

Extension
Author (truncated)
Version

vscode-docker
Pet
0.0.16

ejs-language-support
Qas
0.0.1

Gradle
caz
1.1.2

vscode-standardjs
che
1.1.7

vscode-semistandard
fle
1.1.2

vue
liu
0.1.5

proto
pet
0.0.2

java
red
0.8.0

gitblame
wad
2.0.2

Steps to Reproduce:

Open the terminal embed inside vscode
Do the debug console shortcut
Observe that the shortcut doesn't work

Note that it work in the opposite way, if i have the debug console open and then i do the terminal shortcut, the terminal popup

Reproduces without extensions: Yes

ctrl+shift+y works for me on my machine.
You may need to add the "workbench.debug.action.toggleRepl" command to the "terminal.integrated.commandsToSkipShell" setting if your shell is eating the keystrokes.

Sorry indeed it was the shell that was eating the keystrokes, thanks for the tips


English version should be equivalent to Norwegian version in content

Use @gdubya and @markwest1972 as proof readers for your English.

The English version of java.no was made last year, at the request of the University of Oslo, in order to be able to present an English version of javaBin to James Gosling. The current solution based on Jekyll (written by me) could be replaced with a more permanent solution, that does not rely on duplicated Ruby code. Could probably be a separate issue though.

Feel free to make another issue - I can then close this one as a duplicate :)


Add PowerDownPhy() and PowerUpPhy()

I read an old note of mine saying something about the fact that the calls to PowerDownPhy() and PowerUpPhy() are missing in SetDPPowerState(). In AppleIntelAzulController::SetDPPowerState() they are called, but they are missing in the Skylake/Kabylake frame buffers.
Thing is. When I boot a Skylake based system then I see warnings in the log stating that the frame buffer attributes can't be read or written to, because it's not woken up yet.
This may very well be the reason why wake after sleep isn't working. Can't we just add something like this in the IGFX plugin?
PowerDownPhy()
SetDPPowerState()
PowerUpPhy()
Possibly with the also missing calls to ReadAUX()
What do you think?

Hi,
The idea sounds pretty interesting, although I cannot be sure if it is the cause. I had a talk with lvs, and he also said that it is worth trying. Just wrapping SetDPPowerState should be more than enough, and we have pretty much everything from the arguments to call Power*Phy functions (except the last integer, but it should be 0/1 anyway, could just bruteforce I guess), so it is fairly straight-forward.
The only issue behind trying is that lvs is currently on holidays and I am rather busy irl. Since neither of us has SKL/KBL anyway, it will most likely be faster for you to try.
Vit

I cannot even boot with the stock version of Lilu.kext and IntelGraphicsFixup.kext (both at v1.2.0) but my patched copy of v1.1.3 works.

The easiest question, do you mean that your patched IntelGraphicsFixup 1.1.3 works and the system wakes now or do you mean that IntelGraphicsFixup 1.1.3 works at least somehow?
As for Lilu and latest IntelGraphicsFixup it sounds pretty terrific. They were both tested heavily and are used by many people at the moment, furthermore IntelGraphicsFixup simply got no updates across the releases. It would be nice to discover and fix the issue if it is in Lilu itself. Mind helping me out?
 If a panic, perhaps you could show it to me with keepsyms and so on (in case you have no working nvram you could try this patch?
— Are you sure your kextcache is not corrupt for some weird reason?
— Are Lilu/IGFX injected via a bootloader or if they are in /L/E or /S/L/E do you have LiluFriend configured?
— Might there be any chance for some other Lilu plugin/Lilu kext that was designed to work with Lilu 1.1.x branch?
— Could you boot with -lilulowmem boot argument, this should make Lilu work in a more 1.1.x way?

1.) It was a real panic.
2.) I don't use a prelinkedkernel.
3.) Lilu.kext and IntelGraphicsFixup.kext are injected by RevoBoot from /Extra/Extensions.
4.) No other plugins are used on this system.
5.) -lilulowmem won't change anything (not using Clover).
6.) macOS 10.13 build 17A360a
And nope. I first want to be able to boot with the latest version of Lilu.kext and since that still fails... perhaps due to the fact that I am using RevoBoot without loading the prelinkedkernel on a version of macOS that nobody uses anymore, that also means that I may run into issues from time to time that other people never even noticed. Like ReadAUX() returning a value that even I cannot locate in the kext.
Time to go hunting...

Attempts to load prelinkedkernel may actually be a reason why it panics. For example, I could imagine macOS to create a prelinkedkernel, then UUID check will succeed, but in reality prelinkedkernel will not be used and the kexts loaded from prelinkedkernel will have wrong addresses.
I would expect -lilulowmem to actually disable this though. However, to think of it differently, this boot arg did not work for #26, and I wonder if the issues are similar.
You could probably try removing prelinked kernel entries explicitly here or just compile with compression disabled. If it works, then we will somehow have to figure out how to distinguish between used and not used prelinkedkernel from Lilu.

I'll check that later today, but first a quick question. Is this really what you want?
https://github.com/vit9696/Lilu/blob/dd03718cec6a625672b930edbb2eea67378d14ba/Lilu/Headers/kern_disasm.hpp#L41-L57
https://github.com/vit9696/Lilu/blob/dd03718cec6a625672b930edbb2eea67378d14ba/Lilu/Headers/kern_config.hpp#L24
I mean the init() and deist() functions are not disabled in 1.1.3-1.1.7 right?

Yes, this is correct. When not using advanced disassembly Disassembly is stateless, I saved function naming for ABI compatibility, but I still should update the comments. Thanks for noticing this.

While I am not sure that it would fix your issues, it is probably worth trying the latest commit I added =)

1.) Tried with the suggested compression disabled. Still panics.
2.) Used your latest work. Still panics.
3.) I also tried the latest version of AppleALC.kext with Lilu.kext (without IntelGraphicsFixup.kext) and the last thing that I see before the panics is: "missing OSPrelinkKextCount property!".
Which is good. Meaning that usingPrelinkedCache() returns false.
4.) I added keepsyms=1 and saw three _ZN... (I think) symbols from Lilu.kext but the output is unreadable. It's just too fast.
5.) With only Lilu.kext it stops when it should enter the desktop. The good news is that it doesn't panic and/or hang. Remove desktop should probably also work.
Same thing happens when I use an older version of IntelGraphicsFixup.kext in combination with Lilu.kext.

Hmm, in my opinion, patch the kernel (that's for release):
73 41 42 4d 69 31 58 77 0a  → 53 49 50 45 51 46 33 44 0a

And let's see where it panics in this case With debug=0x100 keepsyms=1 it should give us a clear picture after all… Use a debug one please, and upload a binary in case there are issues determining the offsets.

Actually I made a mistake in the previous commit, so apart from the kernel patch you will also need to recompile with latest.

I pulled the latest commit so that is fine, but I searched for 73 41 42 4d 69 31 58 77 0a and 0a 77 58 31 69 4d 42 41 73 in the kernel of 10.13.1 and it's not there.

Hi, it looks like I pasted something wrong for some reason.
8A 02 84 C0 74 44 → 8A 02 84 C0 EB 44

I don't have 10.13.1 but it is there for 10.13.0 and 10.13.2 at least.

Ok. Patched. But I already installed 10.13.1 and now it boots properly. Without the panic.
One thing though. With the stock Lily.kext (debug) and no prelinkedkernel use I see this in the log:
unable to load missing 0 image com.apple.driver.AppleIntelHD4000Graphics from prelink
unable to load missing 0 image com.apple.driver.AppleIntelHD5000Graphics from prelink
unable to load missing 0 image com.apple.driver.AppleIntelSKLGraphics from prelink
unable to load missing 0 image com.apple.driver.AppleIntelSKLGraphicsFramebuffer from prelink
unable to load missing 0 image com.apple.driver.AppleIntelKBLGraphics from prelink
unable to load missing 0 image com.apple.driver.AppleIntelKBLGraphicsFramebuffer from prelink
unable to load missing 0 image com.apple.iokit.IOGraphicsFamily from prelink
I think that you may want to change this:
kern_return_t MachInfo::init(const char * const paths[], size_t num, MachInfo *prelink, bool fsfallback) {
	kern_return_t error = KERN_FAILURE;
../..
	// Attempt to get linkedit from prelink
	if (prelink && objectId) {

I think that relink should be false/null here.

Nice to hear. You are right about the warning, this code was not expecting no prelinked cache in normal case. Patched it up :)

Thanks.
 The code looks absolutely valid to me. Perhaps you do not notice the successful codes? It looks like this error is normally returned when it fails to read from the register, which implies to me that it may think it is "time out/busy", and thus could happen pretty frequently.
Also, since the original SetDPPowerState also contains a write before read and in fact it is a rather common pattern in the other functions as well… I wonder if we either need to write something first or not focus on aux commands at all and try if it works as is?

Yes. I am sure. Also. There are only two different call to readAUX()
kernel: (kernel) igfx : readAUX retValue = 0xe00002d6, unk2 = 0x0, unk3 = 0x1
kernel: (kernel) igfx : readAUX retValue = 0xe00002d6, unk2 = 0x600, unk3 = 0x1

A lot of them but only these two. That can't be good. I also added this little fellow:
uint32_t IGFX::getDPCDInfo(void * framebufferController, void * AppleIntelFramebuffer, void * AppleIntelDisplayPath)
{
	uint32_t retValue = 0;
	if (callbackIgfx && callbackIgfx->orgGetDPCDInfo)
	{
		retValue = callbackIgfx->orgGetDPCDInfo(framebufferController, AppleIntelFramebuffer, AppleIntelDisplayPath);
		printf("igfx : getDPCDInfo retValue = 0x%x\n", retValue);
	}

	// return 0 = black screen/no hang.
	return retValue;
}

Note the text. Even with your computeLaneCount()

Do you currently use HDMI or DP? Also, are these commands happening at boot or at wake? I checked the disasm a little further and funcs like GetOnlineInfo kind of imply that GetDPCDInfo is expected to fail when using a HDMI display (which makes sense to me).
0x600 DPCD register appears to be pretty special, it contains the actual powerstate of the display. There is a good picture on the 19th page of this pdf for DP newbies like I am. Additionally, I stumbled across this Intel pdf. And now I have a strange deja vu against what we see in Apple drivers and this docs (e.g. ReadAUX is btc_dptx_aux_read and WriteAUX is btc_dptx_aux_write and so on).
All in all I am starting to agree that Apple drivers may actually not set a correct state on sleep/wake, which results in black screen, however, the fact that aux commands fail implies that we have borked display initialisation. Therefore just doing PowerUpPhy several times might be a cure (well, a pretty straight-forward idea at least). I wonder if it works for Apple due to some ACPI code or maybe their special GOP that kicks the display earlier and they are good enough to work.

I use DVI and that is why the reads/writes fail (the driver only supports eDP, iDP and DP).
The log output is after a (cold/warm) boot.

Hmm, but in this case it means that it is most likely the same for e.g. Azul GPUs. And in this case PowerUpPhy/PowerDownPhy are also never called on them, since they are guarded by successful ReadAUX calls.

Ok, I did some checks, and it appears that PowerDownPhy() / PowerUpPhy() are not the case. They seem to be dedicated to DP support, and pretty much unused on Azul just like I said. What may be are two things:
After I implemented GuC support in IntelGraphicsFixup, I discovered that reference firmware scheduler (IGScheduler4) is half-complete, in particular its sleep wake part, which I think may be the key to the problem.
Another issue to kick in is AGDC or rather its AGDP part. I explored the computeLaneCount issue further, and came to a conclusion that Apple driver does support HDMI (and DVI) displays, but it does not support boot displays that are not DP-like. computeLaneCount is not meant to be invoked for anything but DP, and it actually is guarded:
    if ( ctrl->fAGDCCallback )
    {
      if ( !detailsAGDC.modeType ) // This must be 2, and it is for displays other than boot display
      {
        if ( this->fDisplayPath[0]->fPortType != 3 )// LVDS
        {
          laneCount = 0;
          if ( !AppleIntelFramebufferController::ComputeLaneCount(
                  ctrl,
                  &descr→timingInfo.detailedInfo.v2,
                  this->fBitsPerPixel,
                  this->fAvailableLaneCount,    // 0
                  &laneCount)
            || (laneCount > this->fMaxLaneCount) )
          {
            return kIOReturnUnsupported;
          }
        }
      }
    }

But this guard fails, because the structure returned by AppleGraphicsDevicePolicy::VendorEventHandler method 10, which is not described in 10.9 SDK (last SDK with at least something in regards to AGDP), contains 0 in what I called modeType. This appears to be the default for all boot displays in accelerated drivers. And for a 2nd display it returns 2 just as "normal". 10.13.4 kind of patched it by an extra HDMI 2.0 check, but it is hardly related to the problem.
I guess I will close this due to the circumstances, but perhaps it could work as a hint at least.


server profile: disable AutoNATService

@RubenKelevra no worries, this is a useful signal that we may need to improve docs or something.
Do you remember why you made invalid assumptions?

Nope, but I'll start over again and try to make sense of this from the docs and improve what's wrong.
Btw: How do we deactivate on the nodes the autonat polling? That's on by default IIUC and should be off - while we want to let the service running.


The timers used for GPU libraries are inaccurate

Since we're using the CPU timer perf_counter() as the proxy (technically we should use CUDA events, but it's OK), we need to do device-wide sync before and after the sandwiched code section; that is, the synchronization should also be inserted before line 29 that calls welch:
https://github.com/data-apis/scipy-2023-presentation/blob/26db2c0d45c0715dc146f3124c05a17920b05a40/benchmarks/scipy_bench.py#L28-L33

Same applies to the other file: A sync should be inserted before line 43 that calls fit():
https://github.com/data-apis/scipy-2023-presentation/blob/26db2c0d45c0715dc146f3124c05a17920b05a40/benchmarks/scikit_learn_bench.py#L41-L45
in case the GPU is still busy (initializing input data) while the CPU already reaches to line 43.

Another note is these benchmarks do not do any warm-up for stabilizing the GPU state and preparing the array libraries, and it's problematic as the first call is always the slowest (initializing the CUDA context, loading the CUDA libraries, JIT'ing the kernels, if any, etc).
Generally, we want something like this
https://github.com/NVIDIA/cuQuantum/blob/2cc68ede865d1f2028f28c9ad411f85748e20b58/benchmarks/cuquantum_benchmarks/_utils.py#L453-L498
in which the function is run in a warm-up loop, and then we start the benchmark loop, measure the times, and report average (+ confirm the std is low).

Regarding a warmup, yes, I noticed that the scikit-learn fit benchmark is considerably faster if you run it a second time in the same process (pytorch cuda goes from 2.4 seconds down to 0.3 seconds, cupy only goes from 1.1 seconds down to .97 seconds). For predict re-running in the same process doesn't affect the output, and I haven't tested welch yet.
The reason I am running the benchmarks in a separate process is because Tyler had mentioned this as an issue in his benchmarks https://github.com/tylerjereddy/bench_welch_array_api/blob/main/bench.py#L35-L38. I also did it because I felt this was more accurate to a real world application to run a "cold start".
I'd love to hear your thoughts on whether this makes sense, and what is standard to do?

Tyler's comment is a bit confusing, I think if the script focuses on one library at a time, it's OK to do a warm-up loop in the same process. There's no context switch, no destruction of the mempools, nothing. But we still want to launch one process per library, yes.
Cold-start data are frowned upon by CUDA ninjas, unfortunately 😅 From CUDA initialization to launching the first kernel, there are many unknowns (and some being or have been addressed by the CUDA team, such as lazy loading) irrelevant to the quality of a workload implementation that we should exclude to reflect the status. You can argue that "hey but CUDA takes 10 seconds to initialize on my laptop," and no doubt it's the real world experience, but it's really irrelevant to our focus (on making an array API compatible implementation run on GPUs). TBH even for CPU-only workloads, cold start is unfair.

So in my testing, I am already observing this just from running the timings in separate processes. For example, at https://github.com/data-apis/scipy-2023-presentation/blob/main/benchmarks/scipy_timings.csv you can see that the first torch_gpu, False run is slower than the rest, and the same for cupy, False.
The main thing that concerns me is the significant speedup of pytorch on scikit-learn's LinearDiscriminantAnalysis.fit benchmark. Like I said, if you run it in separate processes, it takes about 2.4 seconds, but if you re-run it in the same process, it takes 0.3 seconds subsequently. I'm not completely clear why this is, but I suspect the torch jit is doing something to make things faster. This doesn't happen with cupy, and it doesn't happen with the other two benchmarks (LinearDiscriminantAnalysis.predict or welch).

By the way, can you confirm that the synchronize calls should both go within the perf_counter block (i.e., be including in the timings)?

So in my testing, I am already observing this just from running the timings in separate processes.

Can't speak for PyTorch, but it's kinda expected for CuPy, for two reasons:

The CUDA driver would JIT & cache the kernel code on disk
CuPy would do the same on its own (different JIT)

so for subsequent process launches, some JIT overhead is avoided. So this further justifies that we should really exclude unnecessary noise.

The main thing that concerns me is the significant speedup of pytorch on scikit-learn's LinearDiscriminantAnalysis.fit benchmark.

Too good to be true? 😄 I don't know if PyTorch would optimize for repeated workloads/graphs internally, or if the PyTorch path somehow would trigger another cache mechanism, if so it'd explain. CuPy has no smart runtime or compute graph system backing it.
We should fix the timer and retry.

By the way, can you confirm that the synchronize calls should both go within the perf_counter block (i.e., be including in the timings)?

It appears that I wasn't being accurate, sorry 😓 We should do
sync()
t1 = timer()
...
sync()
t2 = timer()

so that between t1 & t2 only the target workload is under exception, since we're using the CPU timer as proxy here.
You might be wondering why the benchmark helper from cuquantum-benchmarks that I linked above does it differently. It's because we don't use the CPU timer to time the GPU workload there (note there is no sync before the 2nd CPU timer call). The CPU timers are only used to measure the time spent on host, which due to asynchronicity could be drastically different from the GPU elapsed time (measured via CUDA events).

Too good to be true? 😄

I honestly don't know. I don't have enough experience with GPU to know if this is reasonable or not or to guess why it is happening. I don't really even know what sorts of things fit is doing under the hood (@thomasjpfan would know more about that).

You might be wondering why the benchmark helper from cuquantum-benchmarks that I linked above does it differently. It's because we don't use the CPU timer to time the GPU workload there (note there is no sync before the 2nd CPU timer call). The CPU timers are only used to measure the time spent on host, which due to asynchronicity could be drastically different from the GPU elapsed time (measured via CUDA events).

Is there a more accurate GPU timer that I should be using? Or is that even worth bothering with?
Note that aside from the first run, the timings appear to have pretty low variance.

I updated the benchmarks script to run all benchmarks in the same process, sync before timing, and do a warmup run that isn't included in the timings. The biggest difference from before is that pytorch comes out as much faster for the scikit-learn fit run, and a little faster for the SciPy welch benchmark. See https://github.com/data-apis/scipy-2023-presentation/blob/main/benchmarks/bench.py. Let me know if you have any additional concerns.

@asmeurer LGTM! Note that NumPy results also get slightly better, if you compare the perf data before/after the changes (I assumed the benchmarks were done on the same machine) 🙂

btw has the plot in the paper been updated?

Yes, I updated the plot in the paper.

FYI, both PyTorch and CuPy noted the importance of proper synchronization and warm-up runs.

https://pytorch.org/tutorials/recipes/recipes/benchmark.html
https://docs.cupy.dev/en/stable/user_guide/performance.html#benchmarking

In particular, essentially PyTorch's Timer util is doing what we do here too (link).


PHP Extension

Any chance you can spare a little time to package the code into a PHP extension? It can be done using something like https://github.com/hjr3/rust-php-ext

Thanks for the idea! I might get around to it someday, though I'm afraid it won't be particularly soon.


Listbox widget does not work if used inside a group

Hello!
using the latest version 0.9.5, I think I have encountered a new bug with the new Listbox feature.
`from gooey import Gooey, GooeyParser
@Gooey
def main():
parser = GooeyParser()
parser.add_argument('--works_main',
choices=["a", "b"],
nargs="*",
widget='Listbox')
group = parser.add_argument_group('a name')

group.add_argument('--works',
                   choices=["a", "b"],
                   nargs='*',
                   )
group.add_argument('--does_not_work',
                   choices=["a", "b"],
                   nargs='*',
                   widget='Listbox'
                   )

return parser.parse_args()

if name == 'main':
main()
`
error message:

File "C:\Python27\lib\argparse.py", line 1294, in add_argument
action = action_class(**kwargs)
TypeError: init() got an unexpected keyword argument 'widget'

Oh, hrm.. that actually looks like it's been a bug since the very beginning 😱. While there was a Gooey wrapper for MutuallyExclusiveGroups, I didn't have one for the basic ArgumentGroup type.
Fixed in 8cc6d1d
Latest build is now on PyPi


Reset blockchain and BTC is now missing

Hi,
On Friday I sent 0.318 BTC
I sent money to this wallet 1DqGrCgKkXM8Fks8eAcj3R8NkjfKwnnM7x
from my multibit wallet 1G93BmtiR3FYLDqyvzt522xuuGXuAtcRfD
Today the money had not gone to the new wallet, I had a square and one peer had seen the transaction so I reset the blockchain.
Now the money is no longer there. What can I do?
Help please,
John

Further, I have contacted the people [coinjar] today and they have not received the BTC

In your post you don't mention which version of MultiBit CLassic you are using.
If you are not on the most recent version (0.5.19) then most likely the fee being attached is too low.
Older versions of MultiBit set a fee which was ok at the time but it is far too low nowadays.
If this is the case then:

Update using the latest installer from //https://multibit.org/releases/multibit-classic/
Do a Tools | Reset blockchain and transactions to resync your wallet to what is on the blockchain.
Do the send again.

My god it actually worked. I installed MultiBit Classic, from 0.5.15 which I had been using and reset the block chain and there was the missing BTC. Thanks Jim, now I understand it was the fees that caused the delay.

Glad you sorted it out.
Fees in bitcoin are a bit problematic.
In Classic 0.5.19 we have put in a fee slider (in Preferences) so that users can tweak the fees they add too.