3d models problem

BkCommonLib version: BKCommonLib-1.12.2-v5-SNAPSHOT
TrainCarts version: TrainCarts-1.12.2-v3-SNAPSHOT
Spigot version: 1.12.2

Problem or bug:
problem
Best,
I make a model in my program 3 blocks wide but in minecraft I get only 2 blocks wide.
How come?
thank you in advance.
with kind regards,
Ben

Models have 'display' transformations that can rescale the original model based on where its displayed. Minecraft has some defaults provided when none are set that usually result in a downscaled model. Did you set these transforms in the exported model .json file?
	"display": {
		"thirdperson_righthand": {
			"rotation":  [ 90, -75, 0 ],
			"translation":  [ 0, 0, -2.079 ],
			"scale":  [ 0.25, 0.25, 0.25 ]
		},
		"thirdperson_lefthand": {
			"rotation":  [ 90, -75, 0 ],
			"translation":  [ 0, 0, -2.079 ],
			"scale":  [ -0.25, 0.25, 0.25 ]
		},
		"firstperson_righthand": {
			"rotation":  [ 35, -75, 0 ],
			"translation":  [ 2, 2, 0 ],
			"scale":  [ 0.25, 0.25, 0.25 ]
		},
		"firstperson_lefthand": {
			"rotation":  [ 35, -75, 0 ],
			"translation":  [ 2, 2, 0 ],
			"scale":  [ -0.25, 0.25, 0.25 ]
		},
		"gui": {
			"rotation":  [ 0, 90, 25 ],
			"translation":  [ 0, -1.369, 0 ],
			"scale":  [ 0.32, 0.32, 0.32 ]
		},
		"head": {
			"rotation":  [ 0, -90, 0 ],
			"translation":  [ 0, -1, 0 ],
			"scale":  [ 3, 3, 3 ]
		},
		"ground": {
			"translation":  [ 0, 12, 0 ],
			"scale":  [ 0.4, 0.4, 0.4 ]
		},
		"fixed": {
			"rotation":  [ 0, 0, -45 ],
			"translation":  [ -1, 0, -2 ],
			"scale":  [ 0.3, 0.3, 0.3 ]
		}
	}

Thank you for the information


fix upload bar

don't be a hobbit in widget
make broken pattern smooth
use small pattern asset like the payment one now that repeat works
for Android
fix bug on mobile where upload bar only shows in mounted screens

sorry this one's still having issues. Will tag review again when it's ready.

As per our chat, it's not working right for me

OK this is ready again!

bump!

bumpy bump ⪮ @keybase/react-hackers

⪮

@songgao I think you need to fix some conflicts. FWIW I've tested and it works well, although my filename is always getting cut off by an ellipsis.

(that's random.bin)

Ah; good catch!

Tested and approved


diagnostics: add iam v2 upgrade test

Spin-off of #886. Should be good without the rest 🤞

Commits merged with https://github.com/chef/automate/pull/886


windows compatibility - patch 3 for web_llm.py

Updated input handling to use msvcrt
Modified get_multiline_input function
Updated system messages for Windows (CTRL+Z instead of CTRL+D)
Added Windows-specific checks
Removed Unix-specific imports

Thank you so much for your contribution! It is 3am where I live and I am going to pass out now, but rest assured I will review all the stuff you did tommorow!
Thanks mate!

@TheBlewish Alright, Its 11pm here, I re-checked again and found some issues, will fix em tomorrow. Sleep well. Goodnight!


Fix #32 Added fullscreen option

Have added a --fullscreen flag so VM can be run in full screen if so desired.
If fullscreen is applied, monitor x and y resolution are determined in order to resolve mouse syncing issue.

With the GTK UI, you can already put full screen (there is already an option), but yeah that's useful.

I'm on Manjaro KDE, so I don't have that GUI option.
Your PR works very good, I've merged into my fork. You event don't have to set the --fullscreen flag when issuing the command, a simple Ctl + Alt + F does the trick of entering and exiting fullscren mode.
Thanks for that!

@daPhipz I found doing just Ctrl + Alt + f, the mouse would still be bound to the size of the window so mouse movement was misaligned when in full screen. Hence when you invoke the flag it calculates the full resolution of the screen to resolve this issue and my main reason for the PR.

I would highly recommend getting this into mainline as it's quite useful when running i3 and the likes for example.

Please rebase this PR so it can be merged :smiley:

Hi, I'm not sure what is being ask of me to do. I've not done PR requests like this before. Can you please help explain.

Following a guide on what rebase is about. I think I've done it.
Please let me know if I've done something wrong. I'm learning :smile:

@flexiondotorg Are you waiting for me to do anything more to get this committed? :smile:

thank you @digitalight, this is very helpful.
@daPhipz

a simple Ctl + Alt + F does the trick of entering and exiting fullscren mode.

and thank you for that. 👍

@evereux Be aware:

I found doing just Ctrl + Alt + f, the mouse would still be bound to the size of the window so mouse movement was misaligned when in full screen.

@evereux Be aware:

I found doing just Ctrl + Alt + f, the mouse would still be bound to the size of the window so mouse movement was misaligned when in full screen.
The --fullscreen flag prevents that.

I've already seen this behavior when manually changing the size of my Windows 10 guest window so kind of expect this. Cheers.

Thanks for working on this :+1:


MersenneTwister off by one error

It may be that the MersenneTwister has some sort of off by one error.  I was using MathNet to verify some results, I was getting a mismatch.  After a day or two I now have 4 MT implementations.  Using a seed so each generates the same value(s), MathNet is off sometimes.  The "sometimes" is what is bothersome.
It would appear that any seed value <= 274 results in an off by one error.  Example:  Using a seed of 42, the first random int is 804318770 while 3 other implementations return 804318771.  Seeds of 275-287 match but that is as far as I went.  It may start up again elsewhere.
The "other" implementations do not all have a great provenance, but they do agree.  One is the Troschuetz.Random package from NuGet, one is an old VBA translation and I cant recall where I got the other.
Its entirely possible they are all wrong, but with all 3 in agreement, I though I would pass it along.

The C reference implementation also returns 804318771.
http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/emt.html

I think I see the problem. I'll try to fix it this weekend.

The error does start up again in various places (testing 10million values). That page is where I got the VBA version; which I now have converted to VB.NET.  Negative int seed values are handled differently in Troschuetz vs MathNet.  S/he uses  Math.Abs(seed) which results in totally different values. I cant see that step in the C source, so it seems wrong there (and not your problem).

Our genrand_int32() method output matches the C code output perfectly. The problem is when we convert the double back to an int in RandomSource (DoSample()*int.MaxValue). We are generarting a 32-bit in the MT code, but converting it to a 31-bit in RandomSource. I think we should override Next in the MT class and generate the 31-bit int there as the C code does in genrand_int31 (genrand_int32()>>1). This would remove any rounding issues. The problem is that Next is sealed in RandomSource. @cdrnet Would it be problem to "unseal" the method?

The sealing currently helps ensuring some guarantees on the threading behavior.
Would it be an option to provide a DoSample overload in RandomSource for integer sampling instead, something along the lines of protected virtual int DoSample(int minInclusive, int maxExclusive), with a base implementation of (int)(DoSample()*(max-min))+min? All integer routines would then use this overload internally, while the floating-point routines would continue calling the existing one. MersenneTwister could then override this overload, at least for the case where (minInclusive=0, maxExclusive=int.MaxValue).
Note that Next must never generate int.MaxValue. Is this consistent with the standard?

OK,  an int DoSample makes sense.

Note that Next must never generate int.MaxValue. Is this consistent with the standard?
Right. I think what I suggested could return int.MaxValue.  hmm, thinking...

We already have cases where we simply reject/skip such numbers and call the sample routine (recursively) for the next number. This should not influence the distribution and randomness properties.

Related: in MersenneTwister.DoSample we divide by uint.MaxValue+1 to avoid generating 1.0.

We already have cases where we simply reject/skip such numbers and call the sample routine
(recursively) for the next number. This should not influence the distribution and randomness properties.

OK, then what you suggested makes perfect sense.

Related: in MersenneTwister.DoSample we divide by uint.MaxValue+1 to avoid generating 1.0.

That was in the reference code as well (it had methods to generate [0,1] and [0,1) )


WIP: Care Plugin Support

Proposed Changes

Fixes #6050
Introduces Frontend Plugins

@ohcnetwork/care-fe-code-reviewers
Merge Checklist

[ ] Add specs that demonstrate bug / test a new feature.
[ ] Update product documentation.
[ ] Ensure that UI text is kept in I18n files.
[ ] Prep screenshot or demo video for changelog entry, and attach it to issue.
[ ] Request for Peer Reviews
[ ] Completion of QA

Beside npm i we need to use npm run install-all before npm run dev command : Update it in the readme

LGTM

@gigincg can you update the status here?

@gigincg Since you're using the Digital Ocean backend, the deploy preview is stuck in an indefinite loading state because a CORS error is being thrown by the getCurrentUser API

@nihal467 Please test it out in the Local. I don't want to delay the PR, due to the delays on the DO instance.
cc: @tellmeY18

Updated DigitalOcean to setup to enable allow CORS on cf and netlify previews.
@gigincg

[ ] The individual CSV export for each process, such as "Destination Approved," "Completed," etc., in the Shifting Board is not functioning as expected. All exports are displaying the same cumulative data for all shifts, without applying the necessary filtering for each specific step.


Show more details about "risk encounters"

Current Implementation
Currently only the number of days since the last "risk encounter" are shown.
Suggested Enhancement
A list of all "risk encounters" as provided by the GAEN API should be made available to the user. This should include all the available information (which isn't much, but includes the day, duration and attenuation info).
Expected Benefits
With this additional information, users who receive a warning (or even without a warning, a number of risk encounters > 0) will understand better how much they are affected.

Internal Tracking Id: EXPOSUREAPP-2099

The keyfact in a monitoring program ist a control of the reliability. I am working in a datacenter, developing applications to monitor the health of machines. "all ok" alone is never excepted by the operators. Additional information is required to see the application is working correctly. This is controlling the control. Without this feature people cannot trust the application. I had an event with some friends of mine, and we tried to look at the app to see our contacts (anonymous of course), but nothing is found. So the suggestion to show day, duration and attenuation info is highly recomended and shouldn't be not to much work. Up to this I assume the app ist not working correctly, even though error messages are not shown.

In my opinion, the day, duration and the time on that day of risk encounter are crucial information in some cases, consider the following case:

Your parents haven't got a smartphone.  So the time of risk encounter may decide if your parents were at risk, too.

I would like to add that my request is about making those details visible to the user which are made available by the Google/Apple Framework to the app.
Exact time is not made available to the app, nor is any detail made available for any contacts who do not upload their keys after obtaining a teleTAN. This was decided by Google/Apple for privacy reasons, and if someone thinks Google/Apple should change this, they’d better contact Google/Apple directly.

I totally agree with your enhancement request, this would be a valuable feature.👍🏽

I had an event with some friends of mine, and we tried to look at the app to see our contacts (anonymous of course), but nothing is found.

This would only work if one of your colleagues tested positive AFTERWARDS.

I am interested in more detail information about risk encounters. Also low risk encounters, for example riding on a bus for only 3 minutes where there was an infected person.
But I know of possible data security issues with more detailed risk description: I possibly could guess the name of a person I know, who is infected, if I get multiple risk descriptions of the risky encounter. But nevertheless another argument PRO details:
Gedankenexperiment:  I get an info about a risky encounter on a day I know I have been often there on the same location with ever the same people (Kleingartenverein for example). The info tells me the day was 3 days ago. I know I have been 14 days ago there with the exact same people also. I am positively thinking that I have gotten the infection 14 days ago at the location, because I am young and healthy and I now guess I am through the infection without having had symptoms. I further on will think I am immune and not a thread to others nor CoVid-19 is any thread for me in the future.
Why shouldn't I think like that if I am simple minded ?

@Ulenrich In your Gedankenexperiment you should let yourself get tested. And if an infection circulates in a regularly meeting group it may be the other way round, that you infected the other guy. And you may also have been infected a week ago by a person on the bus or in that Kleingartenverein who doesn't own a phone. By myself, I would also cancel parties if I had any symptoms. During lockdown I was able to guess even without the app.

Related issue: https://github.com/corona-warn-app/cwa-documentation/issues/344

Two things to have in mind:
1. Accuracy
Deriving the the distance based on the RSSI in a BLE setting is very unrelaible. Doing so praticially requieres a controlled lab enviroment in a calibrated setting. Having a smartphone in a jacket or not does have bigger impact on the RSSI, than a distance of a few meters - by far. This is a very active area of research. Typically, positioning is assisted by different attributes such as fingerprinting, machine learning on time series, etc. (https://scholar.google.com/scholar?hl=de&as_sdt=0%2C5&q=BLE+indoor+positioning) - for fieldtracks.org we came to the conclusion, that BLEs accuracy is binary: Either you have a contact or you don't have a contact. There is no in between. Deriving a distance is bogus. From a pandemic point of view, the environment (e.g. indoor / outdoor), stable aerosols or not appears to be relevant. When not knowing the timing of an encounter people cannot check the enviroment (e.g. using googls location history) and can hardly asses the individual risk. (E.g. was there just somebody passing my window, while I was at home alone?, was there somebody sitting just right beside me, while I was taking the tram for 10 minutes?)
2. Privacy.
The App is not desgined to provide contacts in a non-interactive-zero-knowledge proof of knowledge (NIZK, c.f. zcash). The protocol provides information on an encounter (timing, signal level), that is way more than just a single bit bit, i.e. "There is in encounter" (from an information theory point of view). Altough somebody could consider to implement a system using NIZKs, the app is not.  In consequence people could possible be blamed for being infectious afterwards. Providing privacy (i.e. "Informationelle Selbstbestimmung") requires informing the users about the actual data processing. That is - in the most simple setting: Presenting it in a nice way. However, the app does not. The app hides information from the user. I consider this to be a privacy violation.

@yanosz

This is why the app/API isn't calculating distance based on RSSI alone. Furthermore, the API transmits RPIs every ~250 ms, but it only receives every ~5 min for 4 s, iirc.

Data processing is laid out and documented and you opt in by installing the app. By your argumentation, smartphones in general would be a privacy violation. Furthermore, the design of the app is limited by the API. The app simply cannot give you the statistics that you ask for.

The point is to display the timestamp of the beacon received event. That doesn't have anything to do with RPIs, transmission times, etc. Just do timestamping, when a beacon is received by the app. When an RPI is classified as infectiosous, the corresponding received intervals have to be shown.

If the API cannot provide timestamps, than the API is bogus and there's a requirement for utilizing (i.e. developing) another one.

Beacons ARE RPIs. The app doesn't receive beacons, the API does. The app never learns anything about received beacons. And no, if you would give the timestamp, you would be able to identify with who you might have been in contact at that time and thus it would expose his identity.

Nothing happens outside of the API with the beacons.  But this was done intentionally: the more information you offer, the easier it is to use this information for malicious actions. There are really many things you can do just with simple statistics. I believe it is more privacy-preserving not to expose this information.

If you want to see beacons in your surrounding, just use one of the BLE scanner apps. Then you can do all the timestamping you want.

Its the other way around RPIs are beacons (e.g. iBeacons are beacons but do not use any RPI techniques). As you already outlined, you can receive beacons using arbirtary applications. Identifying exposures is realistic and users should be aware of this sitatuation. As a requirement, the app should illustrate the situation by showing the corresponding data, hence educating them.

No API is set in stone - and as you said, using 3rd apps are able to provide the functionality as discussed. Hence, it is possible to integrate this functionality into this app - just alike. And yes, it is outside the current API.

This still needs API change. CWA cannot scan the surrounding, because it doesn't - and CANNOT - have location permissions. Either you use a 3rd party app, or you need to press Gapple to adjust the API. Which they will not do, because the statistics would allow de-anonymization of users.
And why wouldn't users be aware of it? The app is sending via Bluetooth. If you install the app, you know that you start transmitting data via Bluetooth?!

I guess, that we can agree, that such functionality has breaking API changes, that's fine.  Still, it can be implemented (you've already suggested, what kind of Apps do so), when using a different API. Pushing Google or Apple boils down to regulatory and is not a technical concern.
The technical solution would be to rely on Googles / Apples API, write a custom API componet, bundle it with the application and have google and Apple exclude it from power saving restrictions, etc. Having google / apple accepting the App then also boils down to regulatory issues, i.e. laws.

Google can't just exclude apps from power saving restrictions. This is up to the OS. Since you won't get manufacturers to release OS updates for older devices, going the way of Google with the PlayServices was the best approach. Yes, it would be nicer to be able to do it without Google, but that's just not feasible.

Due to a lack of visualization.

Make a pulsating element, like in the SwissCovid app. If you're transmitting, you're transmitting. You can't really be more specific about that, can you?

You can require Google by law to distribute a certain app. That's not a technical limitation.

Make a pulsating element, like in the SwissCovid app. If you're transmitting, you're transmitting. You can't really be more
specific about that, can you?

No, the content of send and received transmissions is still hidden from the user.

@treysis A pulsating element when tracing is active has been added and will be included in the next minor version.
@yanosz If an app doesn't meet the Play Store requirements, then no, you cannot encounter this with any application of law.

Well, if Google was willing, they could implement it though. The question is still if that would change anything. As far as I understand this wouldn't give the app sufficient rights. And Google cannot change that. That can only be changed by upgrading the OS. And this is up to the manufacturers and they won't support this for older devices, making the whole concept of the app even less effective.

@tens0rfl0w ... so you're saying, that this 68.000.000 € project, which has federal attentation cannot result an a law, requiring google and to modify its API? So, this country is paying 68.000.000 € for a simple GUI + backend for an already existing API without any option to changes its technical framework?
... IMHO people should considere hiring a new software architects, then.

Ehem, Germany isn't the only country accessing the API.

@yanosz Please see this link.
A new API specification has been formed and will be
rolled out soon (1.5). This newer version allows for much more freedom in the processing of the data stack and can give the app much more information about the actual data than before.

Ah - 1.5 is:

ExposureWindow     (...)
day | Day of the exposure, using UTC, encapsulated as the time of the beginning of that day.

Welll, that's not quite accurate.  So, the suitable work around remains to have an additional beacon scanner.
Minding the costs of 68.000.000 € and the having https://play.google.com/store/apps/details?id=net.alea.beaconsimulator for free in the app store (hint: it scans and beacons, too), this is a strange project.

Again, you cannot implement a beacon scanner inside CWA. This requires location permission. With location permission, the app is not allowed to access the API. This is to prevent any exposure tracking app to create location profiles of its users. This is a privacy-aware design-choice!

Again, you cannot implement a beacon scanner inside CWA. This requires location permission. With location permission, the app is not allowed to access the API.

Well, one could have this project having two apps, one accessing the API, one accessing the regular BLE API. One could OpenSource both, having verifiable builts and make sure, that no such location tracking takes place.
Well, one could argue, that an alternative app, not having the play store dependencies for google runs on lineageos, even with google services disabled. One could argue, that 68.000.000 € are enough to sustain both.
Well, one has argued, that exluding false positives / false negatives is vital for the acceptence of the application (https://www.schneier.com/blog/archives/2020/05/me_on_covad-19_.html) - fruther information on encounters helps to do so.

Additional/related questions from https://github.com/corona-warn-app/cwa-app-android/issues/896:

Is there a time which should set the Risiko-Begegnung timer to zero after 14 days as well?
Did the friend had another Risiko-Begegnung in the meantime/overlapping time? (But then he should have seen 2 Risikio-Begegnungen, right?)
Will the Risiko-Begegnung stay with 1 as a counter for longer than 14 days? Maybe unlimited?
How to differentiate if there was 1 other/new Risiko-Begegnung overlapping with a timer which sets the "old" value to zero? -
Would be good to see a difference if how many values are really "new" / different from "old" (Does this make sense? The use case with an tiny time slot resetting the old count to 0 and having a new risk could be very low I guess)

I think the current implementation makes sense for the average user (simplicity and easy understanding) and from a privacy perspective.
However, the current implementation is forgoing a lot of potential of the app for health experts, i.e. helping the health authorities in making sensible decisions. Consider the following situation: I am a doctor/nurse using the CWA and suddenly have a low risk encounter and my status is green? Tomorrow at work I would see 50+ high risk patients for who a corona infection could be deadly. In this situation (for risk assessment) it might be crucial to know:

date of risk encounter

duration and estimated distance

Maybe a solution would be to have an additional module of the app targeted towards health authorities (Gesundheitsämter) that could access the additional information from the app for me and give a more detailed individual recommendation how to behave in this specific (low risk, potentially high impact) situations - also taking into account the current status of the pandemic in my region. I guess it would make sense to tell this doctor/nurse to get tested and stay at home, if there is low transmission going on in my region, but it would not be a suitable solution, if I already have a shortage of medical staff.
I would suggest this to be discussed with RKI in order to use the information from the app in a better way than it can be currently done. A second level of access to the data underlying additional data privacy regulations might make sense.

I just want to add that you might miss potential app users due to this (I do understand there are privacy concerns).
I personally worried for a low risk contact until it disappeared in the log and I could trace what I have done that day (I still don't know exactly, but I know I haven't been in a closed space/public transport/shop/didn't talk to anyone so I hope it was a really far away/very short contact).
I have just spoken with 3 friends, working and moving in the city, they all not use the CWA because it would just increase their anxiety. It's a small set for a survey :) but on 4 people I have asked, 3 are not using it for this exact reason.
And I do not wish myself to go through the same doubts/anxiety I had this week for the notification :)
(I do trust the calculations, but I can't trust they are 100% so I would prefer to be able to do also my own assessment or not to be shown it at all the notification if it's so irrelevant)

Please discuss this wishlist item with RKI. For me as a user this is a basic requirement for the app.
Right now it shows risk encounters with low risk without any additional information like day, duration or calculated distance. I do not understand why this information is then shown at all. Any AHA rules must be applied with or without these risk encounters so there is no difference for handling both status on my side.
Right now this information without any context just results in unease. We might see a surge of cases in autumn/winter and I would expect to show a much higher counter, e.g. "10 risk encounters with low risk". If I would know this happens always around the time I was shopping or taking a train + was on 10 different days it would give much more context then just the basic number. Without this context it will only result in anxiety resulting in users stopping to use the app.

On my app (Android) I can't see any counters. Only hints to wash hands. Please tell me were I can find counters. Is there a different app for Android and IOS?

On my app (Android) I can't see any counters. Only hints to wash hands. Please tell me were I can find counters. Is there a different app for Android and IOS?

It is only shown when matching tokens were found: Screenshot_20200909_130734_de rki coronawarnapp

Keeping the information in the dark, might be a hint to hide, that the app is not working correctly (no or only a few contacts are stored).

@mathiaswohlfarth do I understand you correctly, that you'd also want to get information on how many non-positive contacts you had? This is not possible for privacy reasons: https://github.com/corona-warn-app/cwa-wishlist/issues/124#issuecomment-660121125.
But if you have a rooted Android device you can circumvent some of those limitations with @mh-'s corona-warn-companion-android 🙂.

Counters give no information about privacy. but they give the possibility to check functionallity: having a contact lets increase the counter - the app is working. If not the app is not working. I don't believe it is working. I have been a programmer. I know the business. Its better to wash hands.

Ok, so you only want this as a functionality check, to see whether RPIs have been recorded?
I'll quote @mh- comment:

You should be able to see what is received (and the RSSI) with adb logcat|grep ExposureNotification.

(but afaik you also need root for this).
Other than that there is no option for CWA to do this, as non-positive RPIs are not exposed by the Exposure Notification Framework (ENF) to the app: https://github.com/corona-warn-app/cwa-app-ios/issues/678#issuecomment-646686363

poor design ... and root does not fit with privacy

@mathiaswohlfarth No, this was done intentionally for privacy reasons by Google. CWA cannot change this, if Google doesn't change it.

@svengabr : I'm not an app developer but the changes in the 1.3.1 RC do not look like they are providing a solution for this issue. I can only see text changes but nothing about providing the additional information, correct?
According to the statement this wishlist item is going to be evaluated for a later update:

In addition, we will check whether a later update will allow a risk encounter history to be displayed in the app.

Is this part tracked in another issue?
Thanks and kind regards,
Kristian

I don't think this issue should be closed

I don't think this issue should be closed, as it targets improvements which seemingly haven't been ruled out by the RKI as @kherpel mentioned above:

In addition, we will check whether a later update will allow a risk encounter history to be displayed in the app.

It is the typical way of business: get the (tax) money, then keep the customer busy. A wishlist is a good idea. Then "thank you - we will do it later" is the same as "Do Nothing".  I think nothing more will happen here.

@svengabr @mtb77 could you either confirm that the request for more information on exposures has been declined or else reopen this issue? B/c just from the statement posted above this is not clear to me.

@daimpi unfortunately at the current point in time, I don't have any additional information from the RKI regarding this issue.

@svengabr

unfortunately at the current point in time, I don't have any additional information from the RKI regarding this issue.

Thanks for the update 🙂.
In this case: could you keep this issue open until we get more info from the RKI?

Please show Date and Time of later on announced risk contacts, even if this leads to still having a green status.
As such contacts might occur only at shopping or eating in a restaurant, one can reconstruct with Date and Time where one habe been and if shopping or queuing anywhere is the risk reason.
This would enhance the analyse and discussion and avoiding of behavior in the future much, when it ia shown if the risk contact was at 8:00 in the bus ir at 12:00 in the McDonalds restaurant or at work at 10:00 o'clock at a certain date/day.
so it is not about "10 days ago" but about exact time and day.
Thanks for implementing that local showup fast.

Time is not supported by the API. Date could be shown in principle.

Date could be shown in principle.

@MikeJayDee yes, the 'timestamp' of the 'youngest' encounter is available at the API with day granularity; exactly the number that is now shown in cwa.

Hello, what? For sure an app and a computer and - so should - an api can add a time stamp on hourly or better exact basis.
Just use the app to add to the ID and regular time stamp into the db. Thanks. It is important to know if the contact was at bureau desk, at lunchtime or within the bus. Ideally also add a notepad where users add names in a textfield, which generates anl list as history with all manually added names
Mike at date time
Simon at date time
Etc. Then users can Look up the List of contacts in case for each last 14 days.

@webermike
This is even more impossible.
The ENF (made by Apple & Google - SAP/T-Systems do not have any influence on this) does not reveal the number of not positive contacts. It also would be impossible because the Bluetooth beacons which your Handy sends and receives from other Handys are changing every 15 Minutes, so you never know if this is a new person or just the same person with a changed beacon.
Since the scanning is done by ENF and CWA doesn't broadcasts/receives anything it is impossible for CWA to

add a time stamp on hourly or better exact basis

This must be added by Google/Apple to the ENF. And that this happens, I think (at the moment), is very unlikely.

When my machine adds a beacon string to the database, for sure my machine can add an exact time stamp. The positve person just needs to announce all the beacons of its own over the server. Done.

@webermike
Yes this would be possible but do you understand that this is nothing what can be influenced by SAP/T-Systems because the underlying API doesn't work like this.
CWA itself gets only this information handed over by the ENF: https://github.com/corona-warn-app/cwa-backlog/issues/23#issuecomment-678692630

@webermike CWA has to use the Google/Apple Exposure Notifications API (otherwise the whole setup wouldn't be possible at least on iPhones, and would be extremely battery consuming on Android).
So you would need to convince Google to change this.
(Good luck with that...) A more practical advice for you would probably be to use this app: https://play.google.com/store/apps/details?id=com.contextis.android.BLEScanner

@svengabr
Reading your statement form here again I can only repeat what @daimpi stated here. This Issue (the OP) is not about changing any colors or intorducing a new risk card but only about showing more details about risk encoutners.
Could you (the team) please clarify with the RKI if they generally reject to show more Informations about Encounters or if they would theoretically show some f.e. the date of Low-Risk Enocunters, etc.
Thanks!

@Ein-Tim & @daimpi
Thanks for pointing out to #100
When #100 was closed by RKI we had around 1900 cases per day.
At that time it was  correct to have only limited information as likelihood of exposure was low and capacity of laboratories was high. In addition hardly anybody wore FFP2 mask.
Now we have more than ten time as much cases and the ratio exposure / lab capacity has changed totally from 10th September. And walking through a supermarket around 30% of shoppers wear FFP2 mask
RKI has to consider to give out more information to prevent user from either ignoring the information or overflowing the Lab test capacities.

@rvbaer I'll mirror my answer from here:
I agree that it would be nice to have the warn-app-companion (WAC) functionality provided by CWA. But this is not possible currently b/c Bluetooth access is not allowed for ENF apps by Google/Apple (see above for further discussion on this topic). But even if CWA could access BT: this could only work on Android devices as iOS is filtering out all the BLE signals containing rolling proximity identifiers (RPIs) already on the OS level. So nothing except for the Exposure Notification Framework (ENF) which is directly integrated into iOS can even see them.
And even under Android this hypothetical "direct CWA recording" would probably end up being quite unreliable, as many manufacturers implement "energy optimizations" which kill background apps all the time.

@daimpi  I'll mirror may answer from #251
@daimpi
Thanks for the background on why something is possible with warn-app-companion (WAC) & RaMBLE but not with an integrated App.
I understand that pressing the "export database" button in RaMBLE and the import button in WAC is okay for Google in Android. But for Google it is not okay not have an integrated App with the same functionalty.
Sounds strange but obviously these are Google's rules - which are as a consequence also strange.
I am still convinced that it is worth the effort to fight for a better app which supports the users by giving time and location of critical enncounters. Such an improved App would
A) reduce the number of people who want a test without a need
B) reduce the number of people who do not bring themself in quarantaine after the App alarmed them
Such a feature would thus simply increase the "quality"
Especially now with the high exposure risk the RKI should join and also fight for this improvement.

FYI corona-warn-app/cwa-wishlist#178

I still believe both the duration and attenuation should be displayed.
These are available via the ENF and would let people judge their risk better. They could discern different cases: eg. one of my neighbors (large attenuation >12 hours),  likely bus rides (20m), likely high risk restaurant (2hrs short range). This change would not affect fundamental privacy since all of this information is already available to technically savvy users and provided by the ENF.
Since currently some people have permanent high risk status, this would let them judge their risk better and democratize access to information that is already available to users like me to protect those around them while reducing the need for frequent PCR tests.

I still believe both the duration and attenuation should be displayed.
These are available via the ENF

@OlympianRevolution The duration that ENF will return seems to be arbitrarily limited to max. 30 minutes, because of Apple's / Google's opinion on how the reporting user's privacy could be preserved in this context - see https://developers.google.com/android/exposure-notifications/exposure-notifications-api#getexposurewindows
The time will be randomly cut between 15 and 30 minutes: https://developers.google.com/android/reference/com/google/android/gms/nearby/exposurenotification/ExposureWindow
If the observed duration from one DK was longer, they split it into multiple windows and return them in a shuffled order. So if you met a reporting user for 2 hours, you will get the information from ENF that you had at least 4 encounters that day, each between 15 and 30 minutes length.
Still pretty confusing, so I will not yet retire https://github.com/mh-/corona-warn-companion-android

VIa the ScanInstances List there should be more information.
I would be happy with even an approximate total time and mean distance even if it cannot differentiate between 4 people 14 minutes or one person for one hour.
Thanks for the Warn-App-Companion @mh- ! I use it all the time to judge risk and functionality. It is one of the reasons I think it is unfair and hypocritical to prevent normal users from having more information on the encounters, when it is possible for us to know all the details.


Events only fired once

I am creating WebComponents in LitElement for use in our AngularJS app. This is how I defined a component:
import {LitElement, html, query} from 'lit-element';

export class InputComponent extends LitElement {
    @query('input') input;
    protected render() {
        return html`<input @click="${this.iOnlyGetFiredOnce}"/>`;
    }

    protected firstUpdated() {
        this.input.addEventListener('click', (e) => this.onClick(e));
    }

    onClick(e) {
        console.log(e);
    }

    iOnlyGetFiredOnce(e) {
        console.log('i got fired', e);
    }
}
customElements.define('wc-input-micro', InputComponent);

I just simply import and use this component in our Angular 1 app <wc-input-micro></wc-input-micro>.
Expected behaviour
Whenever I click the input element I should see 2 console for each handler.
Actual behaviour
The iOnlyGetFiredOnce handler is only fired once and then it magically gets removed and never gets fired on subsequent clicks. The other onClick handler continues to fire on each click.
Debug steps taken

Monkey patched Element.prototype.removeEventListener to check if something is calling that, but NONE.

Tried stopPropagation/preventDefault on the @click handler.

I see in dev-tools that the handler gets removed from element in the Event Listeners panel.

lit-element: v2.2.1

Browser: Chrome/Firefox

I found the culprit. It was SugarJS. We use it in our app, it puts some methods on the Function prototype, one of them is called once.
Now, LitElement's EventParts expects eventOptions to be put on Handler Function as properties. once is one of them, it becomes truthy since sugar has put once on the prototype. Thus, event handlers added via @event become one-time event handlers.
This is a result of two bad practices, augmenting methods on Native prototypes. And, setting props on functions to pass data/options. It would be great if Lit refrains from doing the latter.

The corresponding code is in lit-html so the issue should be submitted there.
Also, once is a valid option for an event listener:
https://developer.mozilla.org/en-US/docs/Web/API/EventTarget/addEventListener
So, I suppose this should be fixed in SugarJS.

@web-padawan once is a valid option for an event listener, but lit passes that as a property on the handler itself. it sets newListener.once = true internally and then call the native method like el.addEventListener(newListener, {once: newListener.once, passive: newListener.passive ... }).
One may argue its internal detail. https://github.com/Polymer/lit-html/blob/master/src/lib/parts.ts#L632 for reference.
And you are right I should post it to lit-html.

I can transfer the issue


Fix php docs and comments

Thanks you @MathiasReker. I must close this PR because it doesn't bring any value.


Create Database model diagram (MVP 1).

Acceptance criteria:

[ ] ER diagram is ready and screenshot is attached to comments.

https://dbdiagram.io/d/60f728cd4ed9be1c05d51a05 - Diagram in dbdiagram.io

Result:


Updated XML preview pane with XML Syntax highlighting using highlightjs

This is my first ever pull request to anything so I'm not sure if I'm doing it right. I thought from the instructions I should be doing a pull request on the feature branch but that was so far out of date with what's actually in use I thought I would submit direct to master and see what happens.
Would welcome feedback on the code and any ideas. I like it because it makes the XML a bit more readable when viewing it.

Hi @Tiberriver256, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

In order for us to evaluate and accept your PR, we ask that you sign a contribution license agreement. It's all electronic and will take just minutes. I promise there's no faxing. https://cla.microsoft.com.

TTYL, MSBOT;

@Tiberriver256, Thanks for signing the contribution license agreement so quickly! Actual humans will now validate the agreement and then evaluate the PR.
Thanks, MSBOT;

@Tiberriver256 We really appreciate your contribution! We will review your code and let you know as soon as we are done!

@Tiberriver256 We have taken your pull request and merged it into our site.   We have adjusted the styling to fit with the Office UI Fabric but we really like the syntax highlighting.  We have the actual page posted to http://officedev.github.io/Office-IT-Pro-Deployment-Scripts/XmlEditor.html if you would like to view it.  We really appreciate your contribution!

@v-rusmit Awesome! Looks great! Glad to make a contribution.


When I write data to the device, the device is automatically disconnected.

When I write data to the device, the data is written to the success.But the Bluetooth device will automatically disconnect.The error that is displayed in Samsung S6 is "BleGattException{status=10, bleGattOperation=BleGattOperation{description='CHARACTERISTIC_WRITE'}}",but in other mobile phones, it is " BleGattException{status=133, bleGattOperation=BleGattOperation{description='CHARACTERISTIC_WRITE'}} ".I want to know how to solve it.I'm running an example.thanks

Hello there.
Do you know to what device you're talking to?
Both of these statuses come from Android Bluetooth Stack and we cannot do much.
Status 133 is GATT_ERROR and status 10 is GATT_NOT_FOUND.
Perhaps you're trying to write a characteristic that is read only?


[core] Fix yarn docs:api on windows

[x] I have followed (at least) the PR section of the contributing guide.

Properly handle line endings on windows

Extracted from #16146
Closes #15657

@joshwooding Uses Windows. He will be able to review the changes :)


Support escaping newlines

Fixes https://github.com/casey/just/issues/1157.

Updated.

Nice, merged!

@casey, @laniakea64 has noticed that this works in a bunch of other cases. See https://github.com/nk9/just_sublime/issues/23 and this gist: https://gist.github.com/laniakea64/b3381389beca11226cb8fefc4e1abf85
Just wanted to check: is this intentional/desired? If it is, there should probably be a lot more tests.


Prevent LiveSearch textfield from getting focus (open keyboard on mobile)

When on mobile, tapping on the picker (with liveSearch=true) will open the keyboard, which takes some time to show up, and covers half the screen, making it hard to pick items. Is it possible to prevent the keyboard from showing up (prevent focus to the textfield?), unless the field is specifically clicked on?

bump. same problem.

Not only problematic on mobile:
when selecting items from a multiselect with more than a page of items,
clicking each item refocuses the search box, making that item to scroll out of view.
So, it is impossible to select multiple items after each other without scrolling again and again, unless the items are near the top of the list.

Sure, I'm on 1.12.4... adding a screengrab:

I have problem that bootstrap select moves up in mobile.
Latest bootstrap and bootstrap-select version


[v9 RC Blocker] Disable Stableswap

Background
We need to comment out the stableswap instantiation in message services and any other present calls. This is a blocker for RC0 release of v9. This change should be competed by tomorrow, Tuesday May 17th.
Acceptance Criteria

stableswap is not instantiated in v9
all existing and new tests should pass

Do we want to simply comment things out or do we want to completely remove them from implementation?

Comment out IMO


Remove jcenter from init scripts

Per the JFrog JCenter announcement here:

https://jfrog.com/blog/into-the-sunset-bintray-jcenter-gocenter-and-chartcenter/

Brown-outs will happen on 4/12 and 4/26, so these build scripts will break on those days. 2/1/2022 it will be closed indefinitely.
It's too bad this will break older Gradle releases init functionality.

FWIW, to reproduce:
./gradlew init

Select Java application and you will see jcenter added.

This is already fixed in 7.0-rc-1


C3_W4_Model Remediation: a reference link is broken

Item link: Not provide
Specific issue area: Optional resources
Description: One of the links provides to additional resources is broken with 404 error
Proposed solution: Update or remove the link from the list provide.

A learner report a issue with a broken link:
https://community.deeplearning.ai/t/link-broken/399178

Thanks, elirod!

you welcome


import: check why dms not migrating

DMs work according to livenet testing, we were intentionally not importing them on testnet


Safari: createElement in onLoad - Event

i habe a following Script in my site (this is a shorted version of a third-party-script):
window.onload = function() {
  UpdateInputElements();
}

function UpdateInputElements(){
	for(var i = 0; i < document.forms.length; i++){
		var form = document.forms[i];
		var inputElement = document.createElement("INPUT");
		form.appendChild(inputElement);
	}
}

It works perfect on chrome, but not on Safari. I get following Exception:
NotFoundError: DOM Exception 8: An attempt was made to reference a Node in a context where it does not exist.
What can i do to fix it? Maybe you know a workaround?
Thank you! :)

Issue referenced is now here: https://github.com/webcomponents/webcomponentsjs/issues/109


It's not possible to open apk files

This fork is a great job, but unfortunately it's not possible to open apk files since you removed the "REQUEST_INSTALL" permission. Is there any plan to put it again?

This fork is a great job, but unfortunately it's not possible to open apk files since you removed the "REQUEST_INSTALL" permission. Is there any plan to put it again?

Use the open_file package if you want to open APK.

Read readme, this package will never add apk functionality.

This fork is a great job, but unfortunately it's not possible to open apk files since you removed the "REQUEST_INSTALL" permission. Is there any plan to put it again?

@Harishwarrior is correct.
The plugin shouldn't add the install package permission by default but if you want it, You can simply add permission in your AndroidManifest.xml :
<uses-permission android:name="android.permission.REQUEST_INSTALL_PACKAGES" />

This fork is a great job, but unfortunately it's not possible to open apk files since you removed the "REQUEST_INSTALL" permission. Is there any plan to put it again?

@Harishwarrior is correct.
The plugin shouldn't add the install package permission by default but if you want it, You can simply add the permission in your AndroidManifest.xml :
<uses-permission android:name="android.permission.REQUEST_INSTALL_PACKAGES" />

Thanks! It works like a charm 😁


Is it possible to limit the rate of a stream?

I'm using quiche to run a QUIC/H3 server. Sometimes we wanted to rate limit (say, at most 2Mbps) a stream (Http POST) from a client, is it possible with quiche?

Flow control is given back to the peer (in your example the client) when the application reads data via stream_recv. So you can rate limit a stream by managing your read rate in the application.

In this case I'm focusing on POST requests from the client, which uploads big files to the server. The client will also be using quiche.  Is there something in quiche API to set max rate (via pacing maybe?) for the upload in the client?


Update Resolv::DNS definitions based on comments.

Motivation

Update Resolv::DNS definitions based on comments. In particular, honor comments mentioning input/output other than String.  This was motivated by a prod. issue due to inaccurate typing of Resolv::DNS.getaddress.
Test plan

See included automated tests.
Verification script
Note that verification script was run against Ruby 2.6.3, and in that version, each_name cannot take IPv4/IPv6 addresses (added after  released https://github.com/ruby/ruby/commit/66508992483ae5d77b56a98427c50c772341c0ac#diff-73ed31bea018f3a994e757faebca1dbf4edb3027ca4a1ecdd623abc955b8e2f6). This affects getname and genames too.
irb(main):063:0> resolver = Resolv::DNS.new
=> #<Resolv::DNS:0x00007f8567213f10 @mutex=#<Thread::Mutex:0x00007f8567213ec0>, @config=#<Resolv::DNS::Config:0x00007f8567213e70 @mutex=#<Thread::Mutex:0x00007f8567213e48>, @config_info=nil, @initialized=nil, @timeouts=nil>, @initialized=nil>
irb(main):064:0> addr = resolver.getaddress('www.google.com')
=> #<Resolv::IPv4 <IP_ADDRESS>>
irb(main):065:0> resolver.getaddress(Resolv::DNS::Name.create('www.google.com'))
=> #<Resolv::IPv4 <IP_ADDRESS>>
irb(main):066:0> addr_arr = resolver.getaddresses('www.google.com')
=> [#<Resolv::IPv4 <IP_ADDRESS>>]
irb(main):067:0> resolver.getaddresses(Resolv::DNS::Name.create('www.google.com'))
=> [#<Resolv::IPv4 <IP_ADDRESS>>]
irb(main):068:0> name = resolver.getname('<IP_ADDRESS>')
=> #<Resolv::DNS::Name: sfo03s08-in-f4.1e100.net.>
irb(main):069:0> name_arr = resolver.getnames('<IP_ADDRESS>')
=> [#<Resolv::DNS::Name: sfo03s08-in-f4.1e100.net.>, #<Resolv::DNS::Name: sfo03s08-in-f36.1e100.net.>]
irb(main):070:0> resolver.getresource('www.google.com', Resolv::DNS::Resource::IN::A)
=> #<Resolv::DNS::Resource::IN::A:0x00007f856717ab08 @address=#<Resolv::IPv4 <IP_ADDRESS>>, @ttl=279>
irb(main):071:0> resolver.getresource(Resolv::DNS::Name.create('www.google.com'),  Resolv::DNS::Resource::IN::A)
=> #<Resolv::DNS::Resource::IN::A:0x00007f85671a0290 @address=#<Resolv::IPv4 <IP_ADDRESS>>, @ttl=245>
irb(main):072:0> resolver.getresources('www.google.com', Resolv::DNS::Resource::IN::A)
=> [#<Resolv::DNS::Resource::IN::A:0x00007f85671c94b0 @address=#<Resolv::IPv4 <IP_ADDRESS>>, @ttl=223>]
irb(main):073:0> resolver.getresources(Resolv::DNS::Name.create('www.google.com'),  Resolv::DNS::Resource::IN::A)
=> [#<Resolv::DNS::Resource::IN::A:0x00007f85671ea200 @address=#<Resolv::IPv4 <IP_ADDRESS>>, @ttl=211>]
irb(main):074:0> resolver.each_address('www.google.com') { |a| puts a.class }
Resolv::IPv4
=> nil
irb(main):075:0> resolver.each_address(Resolv::DNS::Name.create('www.google.com')) { |a| puts a.class }
Resolv::IPv4
=> nil
irb(main):076:0> resolver.each_name('<IP_ADDRESS>') { |a| puts a.class }
Resolv::DNS::Name
Resolv::DNS::Name
=> nil
irb(main):077:0> resolver.each_name(addr) { |a| puts a.class }
Traceback (most recent call last):
        5: from /Users/damolina/.rbenv/versions/2.6/bin/irb:23:in `<main>'
        4: from /Users/damolina/.rbenv/versions/2.6/bin/irb:23:in `load'
        3: from /Users/damolina/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/irb-1.0.0/exe/irb:11:in `<top (required)>'
        2: from (irb):77
        1: from /Users/damolina/.rbenv/versions/2.6.3/lib/ruby/2.6.0/resolv.rb:459:in `each_name'
Resolv::ResolvError (cannot interpret as address: <IP_ADDRESS>)
irb(main):078:0> resolver.getname(addr)
Traceback (most recent call last):
        7: from /Users/damolina/.rbenv/versions/2.6/bin/irb:23:in `<main>'
        6: from /Users/damolina/.rbenv/versions/2.6/bin/irb:23:in `load'
        5: from /Users/damolina/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/irb-1.0.0/exe/irb:11:in `<top (required)>'
        4: from (irb):78
        3: from (irb):78:in `rescue in irb_binding'
        2: from /Users/damolina/.rbenv/versions/2.6.3/lib/ruby/2.6.0/resolv.rb:427:in `getname'
        1: from /Users/damolina/.rbenv/versions/2.6.3/lib/ruby/2.6.0/resolv.rb:459:in `each_name'
Resolv::ResolvError (cannot interpret as address: <IP_ADDRESS>)

Closed as there were a couple of issues and want to verify.

We have a policy of testing changes to Sorbet against Stripe's codebase before
merging them. I've kicked off a test run for the current PR.  When the build
finishes, I'll share with you whether or how it failed. Thanks!
Stripe employees can see the build results here:
→ https://go/builds/bui_IcsMZDsbmk09R2
→ https://go/builds/bui_IcsNMC3epos65G

We have a policy of testing changes to Sorbet against Stripe's codebase before
merging them. I've kicked off a test run for the current PR.  When the build
finishes, I'll share with you whether or how it failed. Thanks!
Stripe employees can see the build results here:
→ https://go/builds/bui_IcsZhLiE6lTHO3
→ https://go/builds/bui_IcsZW2xhVwLnnc


Enhance project messaging

A concise description and key features can be very helpful for new users (across several domains) looking into CSP.
We can discuss some ideas here, and update the wiki homepage and README with this information.
Some ideas for "key features" from discussions with @ptomecek:

Infrastructure-agnostic
Highly-customizable (with adapters)
Interoperable with PyData ecosystem
Support for historical data, which enables robust testing

The wiki homepage and the launch blog post now mention these features. I think it's fine to not include it in the README -- it was updated to show a quick explain and links to the wiki for further information.
Closing this issue as complete.


INT4-AWQ PPL results for LLaMA-2 model are not as expected

Hi, I have a question about why the int4-AWQ PPL results of the LLaMA-2 model very different from the paper?
in paper result, LLaMA-2 + int4-AWQ PPL result is 5.60, but I got PPL result is 16.38.
I want to confirm if my 16.38 result is correct? And why are the results so different?
Implementation steps：

Perform AWQ search and save search results (already did it in awq_cache)
Evaluate the AWQ quantized model on WikiText-2 (simulated pseudo quantization)
Generate real quantized weights (INT4)
Load and evaluate the real quantized model (now you can see smaller gpu memory usage)
python -m awq.entry --model_path llama-2-7b-hf
--tasks wikitext
--w_bit 4 --q_group_size 128
--load_quant quant/llama-2-7b-hf-w4-g128-awq.pt

Hi, I think the PPL is definitely too high. Are you using the pre-computed scales from our huggingface repo?

Hi @tonylins
Yes, I used llm-awq/awq_cache/llama-7b-w4-g128.pt .

git clone https://huggingface.co/datasets/mit-han-lab/awq-model-zoo awq_cache

and Generate real quantized weights (INT4) commands:

python  -m awq.entry --model_path ../models/llama-2-7b-hf --w_bit 4 --q_group_size 128 --load_awq awq_cache/llama-2-7b-hf-w4-g128.pt --q_backend real --dump_quant quant/llama-2-7b-hf-w4-g128-awq.pt

Any other suggestions, please?

I think you are using Llama-1 cache instead of Llama-2 cache. You should use llama-2-7b-w4-g128.pt instead.

Hi @casper-hansen ,thanks for your advice.
I don't quite understand why you think I'm using llama-1cache, I specified on the command line to use the llama-2-7b-w4-g128.pt. There are no changes to the code.
Do I need to set cache usage somewhere in the code?

Hi @casper-hansen , thank you for raising this issue!
The PPL results in our paper were obtained using the GPTQ evaluation, but our current codebase uses lm-eval-harness for evaluation, which could lead to some PPL differences. To reproduce our results in the paper, you can try using the GPTQ evaluation code.
But I think the 16.38 PPL on lm-eval-harness evaluation still seems too high. I would suggest first checking your fake quantization PPL is consistent with the real quantized model's PPL. If there is a large gap, it could indicate some problems in the real quantization process.

Hi @Sakits , thanks for your reply.
I verified fake quantization PPL with awq_cache/llama-2-7b-w4-g128.pt, and got 16.387 PPL result.

python  -m awq.entry --model_path ../models/llama-2-7b --tasks wikitext  --w_bit 4 --q_group_size 128  --load_awq awq_cache/llama-2-7b-w4-g128.pt --q_backend fake

It looks like a problem with the PPL implementation, I will verify in the GPTQ evaluation, thank you again.

Hi @Sakits, I just verified the PPL with llama-2-7b-hf and the benchmark in GPTQ-for-LLaMa
# fp16
python llama.py ../llama-2-7b-hf/ wikitext2 --benchmark 2048 --check

# INT4g128
python llama.py ../llama-2-7b-hf/ wikitext2 --wbits 4 --true-sequential --groupsize 128 --save ./llama-7b-4bit-gs128_test.pt

python llama.py ../llama-2-7b-hf/ wikitext2 --wbits 4 --true-sequential --groupsize 128 --load ./llama-7b-4bit-gs128_test.pt --benchmark 2048 --check

But I got 6.78 and 7.03 for FP16 and INT4g128 respectively, which are still bigger than 5.47(Llama-2-7B FP16) and 5.69(Llama-2-7B GPTQ) in the Table 4 of paper.
Do I use the correct model and GPTQ evaluation code?

Hi @noob-cod , I noticed the perplexity you reported for FP16 model is quite high at 6.78. It's possible you may be using the wrong evaluation code. Please double check that you evaluated on the wikitext dataset. I would suggest you use the code from https://github.com/mit-han-lab/llm-awq/pull/111 for evaluation.

Hi @Sakits , thanks for your reply. I can get the same perplexity value (5.47) as the FP16 model in the paper.


Gem is still on v2.1.2

Hi,
The v2.1.2 is release back on 2014. Is it still holds good?
$ gem list | grep gmap
gmaps4rails (2.1.2)
I see master branch has updates from last year i.e. 2016.
Why there is no update for gem ?
Thanks.

You are right, I should have made a new release a long time ago.
I'd need to resetup everything to make specs run on my new environment and havent taken the time yet. I'd suggest you point directly to github's repo if you need the latest changes.
Sorry.


$Expand needs more specific documentation for the related table

This section related to $expand 'Retrieve related table records with a query' need to have a note or something that the destination table uses the Lookup column SchemaName on the source table, and the it's case sensitive. For some reason, the example provided works even though the proper case is PrimaryContactId. I'm assuming it has something to do with custom tables, but I'm not sure. This may or may not be a bug.
Doesn't work using Logical Name:
GET https://redacted.powerappsportals.com/_api/okr_okrintakes?$expand=okr_krtargetunit
Error 9004010A: An unexpected error occurred while processing the request
Works using Schema Name:
GET https://redacted.powerappsportals.com/_api/okr_okrintakes?$expand=okr_KRTargetunit
Here is a script to output schema name for a lookup field.
Connect-AzAccount;# | Out-Null;
$Org = "MyOrg";

$Resource =  "https://$($Org).crm.dynamics.com/";
$ApiUri =  "https://$($Org).crm.dynamics.com/api/data/v9.0";
$token = Get-AzAccessToken -Resource $Resource;
$headers = @{"Authorization" = "bearer $($token.Token)";};

$Entities = Invoke-RestMethod -Method Get -Headers $headers -Uri "$ApiUri/EntityDefinitions";
$Entity = $Entities.value | Where-Object {$_.LogicalName -eq 'account'} | Select-Object -First 1
$Fields = Invoke-RestMethod -Method Get -Headers $headers -Uri "$ApiUri/EntityDefinitions(LogicalName='$($Entity.LogicalName)')/Attributes";
$Fields.value | Where-Object {$_.AttributeType -eq "Lookup"} | Select-Object -Property LogicalName, SchemaName;

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 9a8c4f16-4467-9735-7c8c-b983cd0d4787
Version Independent ID: 9a8c4f16-4467-9735-7c8c-b983cd0d4787
Content: Portals read operations using the Web API - Power Apps
Content Source: powerapps-docs/maker/portals/read-operations.md
Service: powerapps
Sub-service: portals
GitHub Login: @neerajnandwana-msft
Microsoft Alias: nenandw

Thanks for your comment, @jsh121988. This issue has been assigned to the doc author for review.

Actually, I found that in the response there are the below values, and people should use @Microsoft.Dynamics.CRM.associatednavigationproperty to expand a related table.
Example Response:
<EMAIL_ADDRESS>"FY23",
<EMAIL_ADDRESS>"okr_FiscalYear", <---- USE THIS -----
<EMAIL_ADDRESS>"okr_iteration",
"_okr_fiscalyear_value": "a204f8fe-fbae-ec11-9840-00224805ffcb",

Another thing I found is that in this section 'Retrieve related tables by expanding collection-valued navigation properties' it needs to be specified that you $expand the relationship name for 1 to many relationships, and it is case sensitive.
This works:
redacted.powerappsportals.com/_api/okr_okrintakes?$expand=okr_KRCustomTarget_KRName_okr_OKRIntake

Thank you for your feedback. We have updated the documentation to include information on determining the lookup associated navigation property to better explain what is required for the $expand property.


Make librivets search the mods directory to extract & inject all valid factorio mods.

Currently you can only load one mod which is rivets itself.
Once injected, rivets must search the mods directory for any other rivets mods and also inject these.
Unzipped mod .dll/.so binaries should land in the factorio/temp directory to be automatically deleted by factorio during next startup.

#3


[Python] Conversation summary skill porting part I

Motivation and Context
Porting conversation summary skill. Integration test is added as well. Action item and topic will be ported later.
Contribution Checklist

[-] The code builds clean without any errors or warnings
[-] The PR follows SK Contribution Guidelines (https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
[-] The code follows the .NET coding conventions (https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions) verified with dotnet format
[-] All unit tests pass, and I have added new tests where possible
[-] I didn't break anyone :smile:

Need to add that tomorrow. Copyright (c) Microsoft. All rights reserved.


Draft 0.1 release plan

The issue is to list out what we would like to mark as 0.1.

Build and Installation:

Support out-of-box installation with docker image
Support source code install on native system and docker
Support devices like CPU, CUDA GPUs, and ROCm GPUs.

Models, Framework and Operators:

Support popular DNN  model formats including TensorFlow, ONNX, and TorchScript
Support commonly used models including AlexNet, VGG11, ResNet50, seq2seq, BERT_large, etc.
Support more than 100 commonly used operators.

Model Compilation and Execution:

Provide a full-stack optimization mechanism, including data-flow graph optimizations, model-specific kernel selection, kernel co-scheduling, etc.
Provide ahead-of-time and source-to-source(model-to-code) compilation to reduce runtime overhead
Remove library and framework dependencies

Model Optimization Research:

Provide command line tool nnfusion for further research
Replacing hand-crafted kernels on the generated human-readable code for users to rewrite

Others:

Provide tools for users to frozen TensorFlow and PyTorch models

NNFusion V0.1 发版说明

构建及安装

支持使用docker镜像直接安装
支持根据项目源代码在本机系统或docker环境中自行编译安装
支持包括CUDA GPU以及ROCm GPU等设备

模型、框架及运算符

支持目前流行的深度神经网络模型格式，包括TensorFlow、ONNX和TorchScript
支持多种常用模型，包括AlexNet、VGG11、ResNet50、seq2seq以及BERT_large等
支持超过100种常用的运算符

模型编译和执行

提供全栈的优化机制，包括但不限于数据流图的优化策略、特定的模型内核选择策略以及内核的协同调度策略
使用提前的模型到代码的编译机制来减少模型的运行时开销
消除库和框架的依赖

模型优化研究：

提供命令行工具 nnfusion 供进一步研究
用生成的可读代码替代了手工指定的内核，代码可供用户重写来提升运行性能

其他：

提供了工具供用户固化TensorFlow及PyTorch模型

@AlisaChen98 @scarlett2018 i have updated the release notes, pls take a look.

@AlisaChen98 @scarlett2018 i have updated the release notes, pls take a look.
Edit to:

Provide ahead-of-time and source-to-source(model-to-code) compilation to reduce runtime overhead
Remove third-party library or framework dependencies


fix add more checks for match expression in nereids, to fix regression case test_index_match_select and test_index_match_phrase

Proposed changes

add more checks for match expression in nereids
and fix regression case test_index_match_select and test_index_match_phrase

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall


Can not build more than 101 pages with Next.js

What happened?
I clone markdoc's Next.js starter demo, randomly add 97 markdom files to pages/docs/ folder, then run npm run build, and Next.js will stuck on info  - Creating an optimized production build ... forever.
To reproduce

git clone https://github.com/markdoc/markdoc-starter
Add random markdown files to pages/docs/ folder like below:

const fs = require("fs");
for (let i = 0; i < 98; i++) {
  fs.writeFileSync(`./pages/docs/${i}.md`, `##${i}`);
}

npm run build
It will stuck on info  - Creating an optimized production build ... forever.
Delete one markdown file from pages/docs/, and it will build successfully.
And then, without deleting .next folder, add some more markdown files to pages/docs/, will build successfully. But if you delete .next before adding more files, it will hang forever.

I try to refactor my blog using markdoc, then found this issue. Had tried several next versions and markdoc versions, no luck. I think it's more likely markdoc related? Looking for help, thanks!
Version
try latest and older versions
Additional context
No response

I think I know what's wrong now, but don't know how to fix it.
This issue is related to @markdoc/next.js. Somehow when having more than certain markdown files, await this.importModule()` just can't resolve, which cause the forever hang.

What does command 'limits' in terminal print?

Cc @ijjk in case you have ideas. I doubt this is a @markdoc/next.js bug, but it could be.
Regardless, I'm sure we could speed up our builds with some sort of caching.

Hey folks, sorry for the delay here. I dug into this deeper, and there is something weird going on within Webpack and/or Next.js that prevents us from importing a module more than a certain amount of times at once. The trouble is we don't know that number ahead of time, nor are we able to stop if from hanging forever without introducing even worse behavior.
I fixed this issue here (https://github.com/markdoc/next.js/pull/19) by removing the build-time importModule call, which disables the Markdoc validation support we had previously included. If you want to validate all your content (for example in CI), you should call Markdoc.validate in a separate script.
I will release this fix on Monday.
Full validation was never supported in @markdoc/next.js so this shouldn't break any applications.
As a follow up, I will open an issue that re-thinks build time schema validation for the Markdoc Nexts.js plugin.


Runtime Error: Blob dimensions mismatch in the collate_minibatch function

Hi all,
I tried to run the codes with a configuration as following to load the VOC2007 dataset into the main thread:
CUDA_VISIBLE_DEVICES=0 python tools/train_net_step.py --dataset voc2007 --cfg configs/baselines/vgg16_voc2007.yaml --bs 5 --nw 0 --iter_size 4
I got the error below :
Traceback (most recent call last):   File "tools/train_net_step.py", line 404, in main     input_data = next(dataiterator)   File "/home/nguyen/anaconda3/envs/pcl2/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 322, in __next__     batch = self.collate_fn([self.dataset[i] for i in indices])   File "/home/nguyen/Documents/test-pcl.pytorch/pcl.pytorch/lib/roi_data/loader.py", line 154, in collate_minibatch     minibatch = default_collate(mini_list)   File "/home/nguyen/anaconda3/envs/pcl2/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 189, in default_collate     return {key: default_collate([d[key] for d in batch]) for key in batch[0]}   File "/home/nguyen/anaconda3/envs/pcl2/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 189, in <dictcomp>     return {key: default_collate([d[key] for d in batch]) for key in batch[0]}   File "/home/nguyen/anaconda3/envs/pcl2/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 178, in default_collate     return torch.stack([torch.from_numpy(b) for b in batch], 0) RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0. Got 900 and 320 in dimension 2 at /opt/conda/conda-bld/pytorch_1532579245307/work/aten/src/TH/generic/THTensorMath.cpp:3616
The important notice is RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0. Got 900 and 320 in dimension 2 ...
What I understand is that blobs which form a mini list to feed into 'default_collate' need to have the same dimension, but when I checked that blob dimensions (batch_size = 5), what I got :
(3, 900, 1200) (3, 320, 480) (3, 381, 576) (3, 648, 864) (3, 799, 1200)
How can I fix this issue?

Hi, our current implementation does not support batch size larger than 1. So we must us --bs 1. Alternatively, we could use --iter_size for large batch size. The real batch size equals to bs*iter_size.

Hi,
I am testing your models on some custom datasets. Could you share code by which you used to generate proposals for images?

Hi,
I am testing your models on some custom datasets. Could you share codes by which you used to generate proposals for images?

Hi, you could follow these codes to generate .mat format selective search proposals. I feel so sorry for that I cannot find the codes to convert .mat format selective search proposals to .pkl format selective search proposals. Maybe you could try to implement it by yourself.

Hi, don't worry about that, I will do it by myself. Many thanks for your instruction.


Add context for external links on the Gateways list page

API Services Portal Issue
User Story
As an API provider using the API Services Portal,
I want additional context for links to external sites on the Gateways list
so that I know what to expect from the link behaviour.
Test Case
ENV

[x] DEV
[x] TEST
[x] PROD

TESTCASE

Sign on to the API Services Portal and go to the Gateways manager by using the navigation tabs: https://api.gov.bc.ca/manager/gateways/list
EXPECTED

External links to DevHub have a visual indicator.
I propose to use the existing icon (up-right-from-square) from the Help dropdown menu.

ACTUAL

External links to DevHub do not have a visual indicator.

The link to DevHub in the publishing tooltip can also be updated:

Found a pattern that could be re-used from the API Directory, but it seems to use an arbitrary chakra-colors-gray-500 colour and a different icon.


Fixed #28399 -- Fixed QuerySet.count() for union(), difference(), and intersection() queries.

buildbot, test on oracle.

Since get_aggregation() also powers the aggregate() method it would be great to see if it makes much effort to get aggregate(Avg('num')) to work as well!

buildbot, test on oracle.

Add tests for intersection() and difference()?

Done, although the usefulness of those tests is somewhat questionable since the executed SQL is basically the same.

mhm, good catch. Updated per your suggestion and pushed


Product bands tables

Changes:

Added Specifications Bands table content to all products that are indexed in the Open Data Cube.
Made the aliases in the bands table be linebreak-separated instead of comma-separated.

Preview: https://pr-350-preview.khpreview.dea.ga.gov.au/data/
Note: Don't merge this branch until merging this other branch from which it is forked: https://github.com/GeoscienceAustralia/dea-knowledge-hub/pull/306
Reviews:

DEA Burnt Area Characteristic Layers Sentinel-2 Near Real-Time - Ivan - Approved
DEA High and Low Tide Imagery (Landsat) - Sagar - Approved
DEA Intertidal - Sagar - Approved
DEA Intertidal Extents (Landsat) - Sagar - Approved
DEA Mangroves (Landsat) - Claire P - Approved
DEA Fractional Cover (Landsat) - James - Approved
DEA Fractional Cover Percentiles (Landsat) - James - Approved
DEA Land Cover (Landsat) - James - Approved

In the background, I would consider changing:
"We need to make accurate comparisons of imagery acquired at different times, seasons and geographic locations."
to
"We need to make accurate comparisons of imagery acquired at different times of day, seasons and geographic locations."

in background I would consider changing:
"In particular, the light reflected from the Earth’s surface (surface reflectance) is important for monitoring environmental resources – such as agricultural production and mining activities – over time."
To
"In particular, the energy reflected from the Earth’s surface (surface reflectance) is important for monitoring environmental resources – such as agricultural production, forestry and mining activities – over time."

consider the following change under "what this product offers"
"The resolution is a 30 m grid based on the USGS Landsat Collection 1 archive."
to
"The data has a spatial resolution of 30m and sits within the UTM based MGA94 coordinate reference system."

consider the following changes under "applications":
add the following below "emergency management"
"- bushfire response

flood response"

Consider removing current "applications" and update to the following:
Agriculture, Forestry

Discriminating vegetation, crop and timber types
Monitoring crop and forest harvests
Determining range readiness, biomass and health
Determining soil conditions and associations
Crop production estimates
Quantifying burn severity
Crop water stress
Monitoring deforestation

Land Use and Mapping

Classifying Land Uses
Cartographic mapping and updates
Monitoring urban growth
Aiding town planning

Coastal and water

Mapping turbidity
Mapping shoreline changes
Mapping reefs and shallow areas
Tracking beach erosion and flooding
Monitoring coral health
Measuring sea surface temperature

Environmental monitoring

Monitoring deforestation
Monitoring water pollution
Assessing drought impact
Tracking oil spills
Monitoring mine waste pollution
Assessing carbon stocks
Atmospheric modelling

Additional comments from Aaron:
For S2, under "What this product offers" I would suggest changing "The introduction of a maturity concept." to just "Maturity"
I think we should add the maturity section to the Landsat descriptions as well

All bands tables have been approved by stakeholders.


Update UG

Updated list_week, list_month, filter. and also flag and command summary.
removed set and parent

LGTM!


Working arrays

"Let" is a keyword for creating an array together with the keywords "at" and "be"
ToDo:

[ ] Implement array parsing
[ ] Add tests for arrays

"Join" on arrays

[ ] Implement Join
[ ] Add aliase "unite"
[ ] Add tests


KBFS (Fuse kext) Install Error post High Sierra upgrade

Upgrade from Sierra to High Sierra
Receive KBFS (Fuse kext) Install Error… my log id: be662a9cf30450e9819b831c
Click Ignore
Keybase folder error

Likely duplicates... #8549 #8906

Yeah sadly there is nothing we can do here.  You can quit some of those other apps if you are using them.  But seems like they grabbed the slots and there were none left for keybase. Let me know if you don’t run any of those apps and we are off base.

I'm also having this issue.  Log ID is 026c32e2fbaf438442f3161c.
I'm not running any of the listed apps, nor am I running anything similar.

@maxtaco Womp. Looks like I'm out of luck… What's up exactly? Are there some High Sierra release notes that point to what's changed? I know about TN2459 but that's not this.
kextstat | grep -v com.apple
Index Refs Address            Size       Wired      Name (Version) UUID <Linked Against>
  130    0 0xffffff7f80b4b000 0x9000     0x9000     net.telestream.driver.TelestreamAudio (1.1.1) 97394489-75E6-39C3-A0AF-658A94F1CA3B <93 5 4 3 1>
  163    0 0xffffff7f80c34000 0x7000     0x7000     net.sf.tuntaposx.tap (1.0) 23FDB715-3D0D-3A26-ACBA-E3794C231CB7 <7 5 4 1>
  166    0 0xffffff7f83650000 0x14000    0x14000    com.intel.kext.intelhaxm (6.0.5) 8C4C5339-F7AA-36E5-A052-3D2E7DC6DF16 <7 5 4 3 1>
  167    3 0xffffff7f83664000 0x62000    0x62000    org.virtualbox.kext.VBoxDrv (5.1.28) C0F0EDBE-F7FA-3189-8D24-48F02415C348 <7 5 4 3 1>
  172    0 0xffffff7f836c6000 0x8000     0x8000     org.virtualbox.kext.VBoxUSB (5.1.28) 93025612-D707-3185-AA16-5D987EA2F72E <171 167 52 7 5 4 3 1>
  173    0 0xffffff7f836ce000 0x5000     0x5000     org.virtualbox.kext.VBoxNetFlt (5.1.28) F8FCB36B-980B-3823-BFCB-765E43593678 <167 7 5 4 3 1>
  174    0 0xffffff7f836d3000 0x6000     0x6000     org.virtualbox.kext.VBoxNetAdp (5.1.28) 6AE4C456-BA43-328E-8464-1CAA4C6F7BB8 <167 5 4 1>
  176    0 0xffffff7f836d9000 0x16000    0x16000    com.google.dfsfuse.filesystems.dfsfuse (3.5.3) CD32A127-3580-3A13-B2CD-D0193BB13E2A <7 5 4 3 1>

I wish I knew what changed.  I guess we'd have to see that list before the upgrade.  Maybe your last remaining slot was taken up by one of those software packages, and they previously used no or fewer slots.

@maxtaco Unfortunately I didn't think to check prior to the upgrade. I cleaned the system after hitting the error. For example I had VMWare Fusion installed in addition to VirtualBox… of course that was the case on Sierra but KeyBase didn't have any issues. If device slot availability is the issue one would imagine hitting this when the system wasn't as "clean".


List repositories inside a Docker Registry v2

The frontend shows a list of all images inside a Docker Registry v2.

:+1: I need it

This could be a great additional resource for listing repos: https://github.com/docker/distribution/blob/master/docs/spec/api.md#listing-repositories

+1

+1

It works already


Added responsive option for size

Responsive Size

size="responsive" When the size is set to responsive, the map size will be set automatically based on the dimensions of the window size.

Thanks for being so responsive on this issue!

Thanks for being so responsive on this issue!


pipenv install from inside pipenv shell installs globally

I run these commands and get some unexpected behavior:
pipenv shell
pipenv install leather
(open new tab in iterm outside of virtualenv subshell)
python3
import leather # don't get ModuleNotFoundError

pipenv uninstall from inside the  subshell wipes it from the global packages.
Since pipenv --venv inside the subshell (in the project directory) works, I'd expect pipenv install/uninstall to work or give some sort of error.
Also, running pipenv uninstall in a directory where there isn't a python project will install pipenv and then attempt to wipe the package (unsuccessfully, because it's not there).

i assume you have pipenv running in a virtualenv

make sure you're using latest

e.g. we fixed this yesterday


[RFC] Updated VMM Extension Integration

Overview
This RFC proposes a new mechanism for integrating the base Bareflank hypervisor repo with extensions (including the extended apis). I beleive that an ideal implementation would have the following characteristics:

Absolute minimal work required for the extension writer - Developers should be able to start writing an extension by doing something as simple as including one file, or writing one line of code in their extension project.
Homogeneity - The process for integrating the extended APIs and hyperkernel should be exactly the same as for custom developer extensions.
Out-of-tree - Make it easy and desirable for Bareflank extensions to reside outside of the main hypervisor source tree. For example, it should not be necessary or encouraged to clone the extended APIs to hypervisor/eapis
Encourage inter-extension integration - Extensions should be encouraged to integrate together, and depend on other extensions. For example, many people will probably want to use the extended APIs as a dependency of their own extension

Some other points to think about:

Should we require extension writers to use cmake as the extension's build system, even if its just a minimal CMakeLists.txt file?
Should the extension be built using the extension's build system, or should the we let the base Bareflank build system carry out the heavy lifting? Is it possible to allow for both options?

Proposed API Changes
Here's an initial stab at how extension integration might work. If you'd rather read these changes in the form of source code, I have a mostly-working branch with these features implemented, and an example extension. To get my example branch working:
git clone -b extensions_overhaul https://github.com/JaredWright/hypervisor
mkdir hypervisor/build && cd hypervisor/build
cmake .. -DBFCONFIG=extensions_example.cmake
make -j<# cores>

Extension Registration
Let extension writers declare their extensions to the base hypervisor build system using a build system configuration file (already part of the new build system), and a new vmm_extension() macro. The vmm_extension macro will support any arguments that cmake's built-in ExternalProject_Add macro supports. The hypervisor build system will do the heavy lifting, and "reach out" to the extension to clone it, build it, and do whatever else you declare in the configuration file. The hypervisor's build system will also copy the entire build configuration (cmake cache variables, etc) into the extension's scope. That way, the extension can access build configurations the exact same way the base hypervisor repo does.
An example build configuration might look like this:
# extensions_example.cmake

# First, you can override any default build configurations that you'd like:
set(BUILD_VMM_STATIC ON)
set(BUILD_VMM_SHARED OFF)
# etc...

# Next, you can add extensions to the build system using the vmm_extension
# macro, passing any arguments supported by cmake's built-in
# ExternalProject_Add() function.
vmm_extension(
    extended_apis
    GIT_REPOSITORY https://github.com/jaredwright/extended_apis.git
    GIT_TAG dev
)

# Or, you could add extensions from a local directory like this:
# vmm_extension(
#     extended_apis
#     SOURCE_DIR /home/wrightj/extended_apis
# )

# Extensions can depend on other extensions:
vmm_extension(
    jareds_example_vmm
    GIT_REPOSITORY https://github.com/jaredwright/example_extension.git
    DEPENDS extended_apis
)

# You can tell the build system which vmm you want BFM to load during
# 'make quick' like this:
set(BFM_DEFAULT_VMM jareds_example_vmm)

Extension's CMakeLists.txt
The extension author will be required to provide a CMakeLists.txt file at the top-level of the extension's source tree, with a minimal amount of configuration (one include() statement). The base hypervisor's build system will provide some convenience macros for interacting with the build system. For example:
cmake_minimum_required(VERSION 3.6)
project(jareds_example_extension C CXX)

# Bareflank extensions will integrate with the base hypervisor build system
# using a single include:
include(${BF_VMM_EXTENSION})

# Make an executable or library out of the given source files, linked with all bfvmm and
# vmm support libraries:
add_vmm_executable(jareds_example_vmm vcpu_factory_rdtsc.cpp)
# add_vmm_library(jareds_vmm_lib some_support_functions.cpp)

# Install the executable to the base build system's vmm sysroot:
install_vmm_executable(jareds_example_vmm)

Build tree structure
Extensions will be integrated into the build tree of the base hypervisor project exactly like external dependencies, except they will be built under build/extensions instead of build/depends. I also suggest grouping the built-in bareflank-vmm components into the build directory at build/hypervisor to make it extremely obvious which components belong where. The full build tree would then look like this:
hypervisor
|  - build
    |  -  hypervisor
         |  - bfdriver
         |  - bfelf_loader
         |  - bfm
         |  - bfsdk
         |  - bfsupport
         |  - bfunwind
         |  - bfvmm
    |  -  depends
         |  - astyle
         |  - binutils
         |  - gsl
         |  - json
         |  - libcxx
         |  - libcxxabi
         |  - llvm
         |  - newlib
    |  -  extensions
         |  - extended_apis
         |  - jareds_example_extension
    |  -  install
         |  - vmm-x86_64-release
         |  - linux-x86_64-release
         |  - linux-x86_64-test

The extension developer

# First, you can override any default build configurations that you'd like:
set(BUILD_VMM_STATIC ON)
set(BUILD_VMM_SHARED OFF)
# etc...

I wonder if we should also have the ability for extensions to add conditions to be validated at the build config validation stage.

+1 x1000. That is a fantastic idea!

This looks great to me. One thing I would be sure to add is some documentation/examples of how to create the config files.
I think it is ok to require a single CMakeLists, especially since there really aren't very many lines that would have to be written. I also like that the extension developer could use a different build system if they wanted/needed to just by passing the appropriate step commands to vmm_extension

+1. I think you nailed it.
Out of curiosity, what does the source tree look like if you use the local source for an extension? Also, will all of the make targets work with the extensions so that we can do our testing on the extensions as well? How will Travis CI work for the extensions?

@connojd I completely agree with adding some documentation. I can probably sneak in some README info along with my PR when its ready.
@rianquinn The source tree can look like whatever you want if you're using local source for an extension. Even though the source can be scatterred anywhere you want, the build tree will all be located in the same place.
For make targets and unit testing, I will probably need to add a way for extensions to indicate wheather or not they support unit tests, and then might want to provide a seperate vmm_extension_test() macro that will build the extension's tests, add them to the 'make test' target, etc. That way, the build configuration file is what determines whether or not you build tests for any given extension.
As for CI, I envision extensions providing one build configuration file per CI build. Those config files can be located anywhere really, as long as CI can specify a path to them.

Another thing that I'd like to suggest making a conscious effort on with extensions is to encourage a "convention over configuration" mindset for extension devlopers. In other words, our build system should provide defaults that make up the Bareflank suggested convension. For example, we could have a convention like this:
If you put a file named build_rules.cmake at the top of your extension's source tree, the build system will automatically use it to validate your extension's build rules
Then, allow developers who want to "break" that convension to do so via configuration:
set(VMM_EX_BUILD_RULES_FILE /custom/path/to/extension/build_rules.cmake

The important part is: developers who follow our convensions should be free from having to know any details of our build system.

+1

Status update: I technically have a PR ready for this, but I think I'm going to spend the rest of today creating some build system documentation before I submit the PR. If anyone wants to play around or start reviewing what I have so far, here's a couple places to look:

Here's some quick instructions (see the README) on how to get started building the base hypervisor + the extended apis + a custom extension: https://github.com/JaredWright/example_extension

Here's the branch where I've been working on with changes to the base hypervisor build system: https://github.com/JaredWright/hypervisor/tree/extensions_overhaul

I'll have a PR ready with documentation shortly!

Documentation to be inluded along with this RFC can be found on PR #560 at /hypervisor/scripts/docs


Fix gettext call - django.gettext is undefined

On a couple of installs I have (Django 3.2 and Django 4.0) django.gettext returns 'undefined', meaning the map never loads.
This checks for that, and just uses the standard built-in gettext that is defined by Django: https://docs.djangoproject.com/en/4.0/topics/i18n/translation/#using-the-javascript-translation-catalog

Wait, I note that this issue is similar to the existing (unmerged) #10.
It seems that django.gettext is being loaded in /admin/jsi18n/, but then subsequently jquery.init.js is loaded which wipes out the django variable:
/*global jQuery:false*/
'use strict';
/* Puts the included jQuery into our own namespace using noConflict and passing
 * it 'true'. This ensures that the included jQuery doesn't pollute the global
 * namespace (i.e. this preserves pre-existing values for both window.$ and
 * window.jQuery).
 */
window.django = {jQuery: jQuery.noConflict(true)};

Not sure of the best way to solve this.


[feature] Element Screenshot

Is your feature request related to a problem? Please describe.
Recently we used screenshot tests with detox to detect breaking UI changes in our components.
In order to do that we compared screenshots of the entire screen that showcase the component.
Unfortunately, our showcase screens contain other elements other than the tested component (titles, controls, etc..).
If something has changed with those elements the screenshot test failed.
It turned those tests to be very flaky so we turned them off for now.
Describe the solution you'd like
A great solution will be to capture a screenshot of a specific element (by testID), this will help isolate to test case and prevent flakiness.
Describe alternatives you've considered
none.
Additional context
I discussed this some time ago with @amitdwix and @rotemmiz, I understand this is applicable.
Would be very beneficial for UI testing.

The problem with such tests is that you are relying on many different systems, and the results are bound to be flaky. You are relying on two different layout systems (UIKit/Android and RN) and two different drawing systems (UIKit/Android and RN). Other technical limitations will make it even worse for you: iOS, for example, is unable to properly screenshot certain elements, and screenshots change with every version of iOS.

We'll start with Android to see if it even makes sense, as an experimental feature. It's fairly easy to create such a screenshot with Android

It's not a problem to support on iOS as well. XCUITest supports it too. I'm just not sure it's wise to encourage something as part of detox which is flaky in nature. We've resisted this in the past for good reason.

@LeoNatan Regarding what you said

The problem with such tests is that you are relying on many different systems, and the results are bound to be flaky. You are relying on two different layout systems (UIKit/Android and RN) and two different drawing systems (UIKit/Android and RN). Other technical limitations will make it even worse for you: iOS, for example, is unable to properly screenshot certain elements, and screenshots change with every version of iOS.

Clearly we will have separate screenshot for each system.
And even if we can measure most of the basic UI elements - it's good enough for us.
If you don't want to commit to this feature, let's release it as an experimental feature, like Rotem mentioned.

I don't mean comparing between Android and iOS. I mean comparing across iOS versions is already too flaky. iOS simulator now uses Metal to render, so even different GPU drivers for the host (VM GPU driver) may have influence on the output. Too flaky.

You may be right, but it's still worth a try

FWIW, an automated system that takes the artifacts, uploads them to S3 and then allows designers to review changes to components would be useful for my organisation.

Why snapshot tests are bad:
https://twitter.com/gspiers/status/1240951050896539649?s=21

https://github.com/uber/ios-snapshot-test-case/issues/109

Hey @LeoNatan @rotemmiz
I agree that snapshot testing might be flaky and not the best solution. (I know, we tried it)
But first, they don't replace our e2e tests in any way.
Second, I believe it is up to the users' responsibility to decide if they want to use it or not.
I don't think we need to argue on whether those kind of tests are good or bad.
Btw, the whole purpose of this specific feature request is take snapshot of specific element which should reduce visual background noise and give better results.

Providing the element snapshot is not a problem in and of itself. The problem is encouraging bad practice, which I don't want to do in Detox. This is by all means our responsibility to argue.
We're already seeing issues on Android where same problems occur as iOS:
https://stackoverflow.com/questions/62177822/mobile-emulator-simulator-detox-screenshot-artifacts-differ-when-run-on-differen


[i2c, rtl] FIFO Depth Parameterization

Signed-off-by: Igor Kouznetsov<EMAIL_ADDRESS>

@igk8 could you adopt FifoDepth in FIFO instances?

@tunghoang290780 FifoDepth is now used in FIFO instances.


Color coding to identify environment

Is your feature request related to a problem? Please describe.

Risk of performing operations on an unwanted environment
Identifying tab environment easily (tab management)

Describe the solution you'd like
User can select an environment color that will appply to

the favicon
the user-info div

It must to be selected by the user so it can be consistent with other tools that provide color coding
Describe alternatives you've considered
Tampermonkey...but can't edit other extensions sadly
Additional context

several extensions also modify the favicon (organizer, salesforce login by synebo...)
Using stylus I personally modify the color of the environment banner Sandbox: <envName> | Log out to quickly identify the current environment

Relates with #197

Hi @Yarnikar,
It will be released in the 1.24 version

The actual feature allows you to choose a favicon by entering its url, this functionality will be blocked by the future update  on the Updated Content Security Policy.

To be able to use an icon from an external website you will have to add the domain to the CSP trusted domains:


example data

I want to include a suite of example trait data. Criteria:

data are already publicly available
authors agree to re-publication within the package
they should cover the different scenarios for trait data: factorial vs. numeric, logical; matrix format vs. table format; literature vs. measured data

Possible data:

trait data from Gossner et al
pollinator traits of C. Weiner (Open Access on Bexis)
...

I import a traitdata of arthropods (Morphological traits and ecological traits of Martin and Nadja, including a raw-data table of replicated measurements) and birds (Ricklefs dataset).

see implementation of data.R.


Miri-friendly coding

The current EBR code is not quite friendly to Miri while the code is not as dangerous as Miri thinks it is. Need to make the code more friendly to Miri.

SDD 1.3.0 enabled SCC to enable a number of MIRI test cases. Still, a long way to go.

while the code is not as dangerous as Miri thinks it is

I'm worried you're underestimating how accurate Miri's UB detection is. The UB warned about in the following diagnostic is currently used for optimizations. We have a MIR optimization that explicitly reasons based on it, and rustc tells LLVM about the requirement here with the combination of noalias and readonly.
error: Undefined Behavior: attempting a write access using <3690> at alloc1139[0x8], but that tag only grants SharedReadOnly permission for this location
   --> /build/src/bag.rs:428:33
    |
428 | ...                   (self.storage[index].as_ptr().cast_mut()).write(val);
    |                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |                       |
    |                       attempting a write access using <3690> at alloc1139[0x8], but that tag only grants SharedReadOnly permission for this location
    |                       this error occurs as part of an access at alloc1139[0x8..0x10]
    |
    = help: this indicates a potential bug in the program: it performed an invalid operation, but the Stacked Borrows rules it violated are still experimental
    = help: see https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md for further information
help: <3690> was created by a SharedReadOnly retag at offsets [0x8..0x10]
   --> /build/src/bag.rs:428:34
    |
428 | ...                   (self.storage[index].as_ptr().cast_mut()).write(val);
    |                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    = note: BACKTRACE (of the first span):
    = note: inside `scc::bag::Storage::<usize, 32>::push` at /build/src/bag.rs:428:33: 428:85
note: inside `scc::Bag::<usize, 32>::push`
   --> /build/src/bag.rs:102:28
    |
102 |         if let Some(val) = self.primary_storage.push(val, true) {
    |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside `main::_doctest_main_src_bag_rs_217_0`
   --> src/bag.rs:224:1
    |
10  | bag.push(7);
    | ^^^^^^^^^^^
note: inside `main`
   --> src/bag.rs:229:3
    |
15  | } _doctest_main_src_bag_rs_217_0() }
    |   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

In fact as far as I can tell, all of the UB diagnostics in this crate and in SDD are for scenarios where rustc explicitly permit LLVM to make optimizations that break this crate. Most of the rest of the diagnostics are about protectors, which is how Miri models the meaning of the LLVM attribute dereferenceable.
I'll see if I can contribute some PRs to address some of the Miri diagnostics that I find most concerning.

@saethlin Super thanks! I agree that I may have underestimated MIRI's ability to detect UB. Really appreciate your comment, and I'll also look into those errors soon!

You may be interested in setting MIRIFLAGS=-Zmiri-tree-borrows, which opts into a much more complicated but generally more permissive aliasing model. There's a manual for the model here: https://perso.crans.org/vanille/treebor/

Fixed most MIRI errors in bag.

Fixes all the Miri errors in bin/doc tests; had to deactivate async/multi-threaded tests due to the test run-time and tokio dependencies.