Refactor data research API

The initial API was exploratory and leaned on a metadata helper class
called FIPS. Now that all data structures are better understood, we can
refactor the API to remove reliance on the FIPS helper, simplify
save and aggregation methods and speed up operations by normalizing metadata.
Additions

data model for "non-MSA" areas, which represent counties in a state that are not part of any Metropolitan Statistical Area. Some may call it rural; I call it antimatter.
new geo models for county, metro area and state to handle metadata.

The refactor implements excellent suggestions from @chosak and @richaagarwal to remove reliance on the FIPS class, simplify save and data aggregation methods and make use of the django-jsonfield module.
Testing
To see the API in action:

Run migrationspython cfgov/manage.py migrate --noinput

Download the data set http://files.consumerfinance.gov.s3.amazonaws.com/data/mortgage-performance/initial_data/mortgagedata_refactor2.sql.gz
Load itgunzip -c ~/Downloads/mortgagedata_refactor2.sql.gz | mysql -uroot v1

Check out some endpoints:
- county time series data
- metro map data
- metadata: sampling dates

@willbarton

It's also unclear to me when and where the data load/validation/export scripts are going to end up running, and how often.

Load, validation and export scripts and related aggregations will run roughly once a quarter.  We don't have an exact timetable because our source, the National Mortgage Database, does not schedule its updates.
The intention is to automate the use of loading scripts into a partial pipeline, but that is for a later sprint.

@willbarton

I feel like there are an awful lot of queries all over the place ...

This is a fair point -- there are lots of queries, thanks to metadata extrapolations -- but from the API's perspective it's not as bad as it seems.
The views do have a number of potential queries that can run, but generally only one (or two, for MSA/non-MSAs) will actually fire, depending on which geography has been requested. I was hoping to keep things DRY by passing different geos to the same views.
The hairiest queries are model validation and aggregation methods, but they will only run once a quarter, and not on production servers.
That's not to say we won't have API optimizations to make, but at this stage API response isn't noticeable in the app.


Work with plugin authors to get compatible with 0.5.0

Existing Plugins

[x] ember-deploy-redis, index-adapter for Redis
[x] ember-deploy-s3, assets-adapter for AWS S3
[x] ember-deploy-s3-index, index-adapter for AWS S3
[x] ember-deploy-azure, index-adapter and assets-adapter for Azure Tables & Blob Storage respectively

We're in good shape on this.


The question of Accuracy and F1-score

Sir, I use your Pre-trained checkpoints(Result Emotion-6 ensemble) in sensembling.py, the classification report are shown in figure1. But I only got the Precision and F1-score of each classification, the F1-score is lower than what you wrote in Table 1 of your paper. And the Precision is not same with the Accuracy you write in Table 1. I think the Precision is not Accuracy, I don’t know what mistake I have done. How can I get the Accuracy and F1-score of each classification like you?
I think the reason may be that the samples in the dataset have multiple category labels, and each sample correspond to more than one emotion. Therefore, the precision, recall and F1 score of a single category are relatively low, but the overall accuracy is very high.

The reason why the F1 score is very low (closer to 0) is actually that the True Positive for some classes is  0 if you try to print the confusion matrix, I have the same problem, I found out that the dataset is not balanced, so the loss function should be a weighted loss or something like that. Another thing I noticed is both the loss and the accuracy are not changing at all from the very first epoch I get about 79% of accuracy which is the best accuracy using only  Acoustic and Visual features. I'm working on improving this model but I have to only use A and V modalities. If you can answer some of my questions especially about the fact that the accuracy and the loss are not improving during the training I would be grateful.


Simpler checkbox

@GeorgeSheppard @blondejamtart how about this?

I like the look, my one criticism would be that it only toggles if you click the text not the actual textbox. Looking through a lot of the .opi's I have been finding widgets without properties that should really be defined (labels without the text to put in them for example), so in case there is a checkbox without text it would be good if it toggled from the checkbox. Not sure why it doesn't work when clicking the checkbox because the onClick function is wrapping the whole form element.

That's weird - it works for me clicking either the text or the checkbox itself.

Hmm, that might be a firefox issue then.

It's working for me on Firefox, Chrome and Edge on Windows. I guess you are Firefox on Linux.

@willrogers, seems to be an entirely linux problem with the checkbox not toggling when clicked on directly (instead of the text) (I checked chrome and firefox and had the same issue on both). I think it's worth merging in as I have only seen one checkbox so it's most likely not worth the time. If we care more about clicking on it directly then my more complicated checkbox could be merged instead, but personally I think the simplicity is worth this small drawback (at least for now).
I made a possibly controversial decision to delete the test associated with the checkbox because it is a pretty simple component and the test relied on specific naming of the props. I'd like to think my new javascript hero Kent Dodds would agree with that decision but your opinion is welcome.

I agree regarding the test, I'm puzzled about the behaviour but let's merge ad move on.

I've tried it briefly in chromium v86.0.4240.75 on linux (ubuntu) and it seems to work perfectly; I suspect it's an issue with out-of-date versions on the Diamond RHEL7 system

The checkbox is working fine. On the BL21I synoptic there is a transparent label over the top of the checkbox part that is intercepting the clicks.
Try the one on the widget test page instead.

That is a good spot, just checked it works well.


Invalid webhook signature.

I follow all the instruction from the wiki documentation and I added a webhook to capture events every after payment.
Steps to Reproduce

Created a new job via php artisan shopify-app:make:webhook [name] [orders/paid].
Created a webhook from shop admin settings under Notification
From Store admin Send test notification

I added Log::info in this file AuthWebhook.php. Line 35
Log::info('hmacLocal -> ' . $hmacLocal);
Log::info('hmac -> ' . $hmac);
Log::info('shop -> ' . $shop);
Result (hmac doesn't match):
[2020-05-20 16:54:46] local.INFO: hmacLocal -> 5+/COHjFGIcoYfy8rE1aUDpBXRSlnFv+35WMr7OqGL8=
[2020-05-20 16:54:46] local.INFO: hmac -> IuB15RhAusbXjgirF8wm7f5Fv8ImRMHQEPsTkprjfHk=
[2020-05-20 16:54:46] local.INFO: shop -> jcbolor.myshopify.com
Context

Package Version: v11.5
Laravel Version: v7.x
PHP Version: v7.2

I don't know why the webhook signature is invalid, but isn't your webhook registered by the WebhookInstaller job, instead of creating a webhook via Shopify Admin?
If not, you should add this line to your hosting to run the cronjob:
* * * * * cd /path-to-your-project && php artisan schedule:run >> /dev/null 2>&1

See: https://laravel.com/docs/7.x/scheduling

I found the issue, my custom app has its own API_SECRET_KEY and for the webhook it has a diffrerent API_SECCRET_KEY. I made a temporary solution by adding another WEBHOOK_API_KEY from the shopify config then that is the one that I used on the AuthWebhook.php file.
$hmacLocal = createHmac(['data' => $data, 'raw' => true, 'encode' => true], $this->getConfig('api_webhook_secret'));

Notifications setting in Shopify Admin is unreleated to app notifications. Will close, glad you found a solution though.

In case the above comment isn't clear to anyone coming to this. Here's a thing: You shouldn't use admin notification to handle Webhook events.
Webhook events
Admin Notifications


Jupyter link in “Geospatial search" is broken

The "Jupyter notebook" links in the blog "Journey through the city: Geospatial search made simple with LLM and Elasticsearch" are all broken. I found that their values are https://colab.research.google.com/github/elastic/elasticsearch-labs/blob/main/geospatial-llm/09-geospatial-search.ipynb. However, the actual location for the notebook is https://github.com/elastic/elasticsearch-labs/blob/main/supporting-blog-content/geospatial-llm/09-geospatial-search.ipynb

this has now been fixed. thanks for the spot!


Misc UI tweaks

Add a table to display imports on the imports page more clearly.
Closes https://github.com/artsy/bearden/issues/136
Closes https://github.com/artsy/bearden/issues/134
Imports: Before

Imports: After

Sources: Before

Sources: After

Thanks for the review @anandaroop


chore: adjust auto update

:tada: This PR is included in version 1.0.53 :tada:
The release is available on:

GitHub release
npm package (@latest dist-tag)

Your semantic-release bot :package::rocket:


[dv/hmac] TL outstanding request max value

This is a temp fix for issue #860, when HMAC and SPI device has an extra
storage for outstanding access.

This does not solve issue https://github.com/lowRISC/opentitan/issues/860, but it is a temp fix. I think the better fix will be on the design side.

After talking with Weicai, we decided to temp disable the check for hmac.

This is a fix for TL outstanding request >2.
After discussion with @sriyerg and @eunchan , we are thinking about setting the max outstanding req to a very large value (currently 16). Then when collecting coverage, DV engineers will be able to provide a covered upper bound. During coverage review, designers will be able to check if it meets their expectation. I understand we wanted to know an exact number, but if any design change, it could affect our TB. What do you all think? @weicaiyang @gaurangchitroda @shakushw

This is a fix for TL outstanding request >2.
After discussion with @sriyerg and @eunchan , we are thinking about setting the max outstanding req to a very large value (currently 16). Then when collecting coverage, DV engineers will be able to provide a covered upper bound. During coverage review, designers will be able to check if it meets their expectation. I understand we wanted to know an exact number, but if any design change, it could affect our TB. What do you all think?

LGTM, understand that this number isn't final yet. When the design is frozen, need to get the extra number. we will see coverage hole, if we don't use the extra number.


Tweaks

Render: https://github.com/fission-codes/skip-ratchet-paper/blob/tweaks/skip-ratchet.pdf

Some copy errors on the right side of page 5:

Access to the entire internal state grants provides keys from a point onwards, including potentially the initial state. The number of elements in the internals state must be at least n + 2 to provide if intended for use in this setting, where n is the number of levels that are desired to be exposed.

Should be this, I think (grants provides -> provides, internals state -> internal state, at least n + 2 to provide -> at least n + 2 if intended):

Access to the entire internal state provides keys from a point onwards, including potentially the initial state. The number of elements in the internal state must be at least n + 2 if intended for use in this setting, where n is the number of levels that are desired to be exposed.


Updated crate python to version 0.19.2

Renamed blackbox/docs/src/crate to blackbox/docs/src/doc_tests
since crate.testing.layer module was shadowed by the directory name.

Codecov Report

Merging #5443 into master will increase coverage by 0.01%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master    #5443      +/-   ##
============================================
+ Coverage     82.32%   82.34%   +0.01%
- Complexity    11278    11280       +2
============================================
  Files          1359     1359
  Lines         44304    44304
  Branches       5558     5558
============================================
+ Hits          36473    36481       +8
+ Misses         5225     5214      -11
- Partials       2606     2609       +3

Impacted Files
Coverage Δ
Complexity Δ

...o/crate/analyze/SafeExpressionToStringVisitor.java
83.33% <0%> (-16.67%)
7% <0%> (-1%)

...rc/main/java/io/crate/analyze/UnboundAnalyzer.java
75% <0%> (-10%)
2% <0%> (ø)

...ration/reference/doc/lucene/IpColumnReference.java
77.27% <0%> (-9.1%)
4% <0%> (-1%)

...crate/planner/projection/MergeCountProjection.java
63.63% <0%> (-9.1%)
5% <0%> (-1%)

core/src/main/java/io/crate/types/ArrayType.java
73.84% <0%> (-7.7%)
24% <0%> (-2%)

...rate/action/sql/BatchConsumerToResultReceiver.java
87.5% <0%> (-6.25%)
16% <0%> (-1%)

...io/crate/planner/projection/ProjectionVisitor.java
35.29% <0%> (-5.89%)
6% <0%> (-1%)

...a/io/crate/metadata/sys/SysSnapshotsTableInfo.java
82.85% <0%> (-5.72%)
6% <0%> (-1%)

.../java/io/crate/blob/TransportDeleteBlobAction.java
75% <0%> (-5%)
4% <0%> (-1%)

...te/blob/pending_transfer/PutHeadChunkRunnable.java
76.71% <0%> (-4.11%)
12% <0%> (-2%)

... and 22 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b17d58d...4f95d89. Read the comment docs.


Test Custom View API improvements

Refs: #48198 #49311 #50662

[ ] anyOS @isidorn
[x] anyOS @weinand

Complexity: 2
Write an extension that contributes a tree view to the Explorer. Refer to documentation and [samples] (https://github.com/Microsoft/vscode-extension-samples/tree/master/tree-view-sample).

Test listening to onDidChangeSelection event when the selection changes.
Test visible property and the corresponding onDidChangeVisibility event are working as expected when the tree view contents are visible or not.  NOTE: visible is false when the tree view is collapsed.
Test passing focus flag to reveal api and make sure that revealed element is focused - reveal(element: T, options?: { select?: boolean, focus?: boolean }): Thenable<void>;

@sandy081 Writing an extension can only have a complexity of 3 and more...

@sandy081 please provide an extension that is creating a tree view, since the one you attached is not using the new api.
Writing a new extension that does all this seems like a much higher compleixty
Thanks

@isidorn I've updated https://github.com/weinand/vscode-processes to the new API. You can use this as a sample.

@weinand thanks!

@isidorn FTP Explorer uses the Tree view implementation. Updated the description to use that sample.


Draft: Note renaming

Proposed change
This PR adds a function to rename a note on the fly. It also adds an option to automatically update all links pointing to the modified note.

Type of change

[ ] Bugfix (non-breaking change which fixes an issue)
[x] New feature (thank you!)
[ ] Code quality improvements to existing code or addition of tests
[ ] Documentation update

Additional information

This PR is related to discussion: #56

The PR is a WORK IN PROGRESS. It seems to be working ok for simple filenames. What is still to do

[x] Work with subdirectories
[ ] Ensure sed function works properly and will not screw everything up (could be an issue if people do not have their notes under version control)
[ ] Maybe prompt before actually replacing ?
[ ] Ensure graceful fallback if sed or find not available (windows?)
[x] Work with links to specific titles within the file

Checklist

[x] I am running the latest version of the plugin.
[x] The code change is tested and works locally.
[x] There is no commented out code in this PR.
[x] The code has been formatted using Stylua (a .stylua.toml file is provided)
[x] The code has been checked with luacheck (a .luacheckrc file is provided)
[x] The README.md has been updated according to this change.
[x] The doc/telekasten.txt helpfile has been updated according to this change.

This looks quite promising so far! :+1:
I would change "... and possibly update the linkks..." to "... and update any links that may point to it..." - but that's just nitpicking :smile:
One thing that comes to mind is: to better save all (or all telekasten) buffers before running find/sed - or else some links that are not yet written to disk will not get updated.
You can check for sed and find via vim.fn.executable("find") - I do the same with xclip.

I would change "... and possibly update the linkks..." to "... and update any links that may point to it..." - but that's just nitpicking smile

Yeah, I wanted to hint to the fact that this was an option that could be disabled. But English is not my first language, so maybe you could word it better.

One thing that comes to mind is: to better save all (or all telekasten) buffers before running find/sed - or else some links that are not yet written to disk will not get updated.

Good point. I think only telekasten buffers should be saved though.

You can check for sed and find via vim.fn.executable("find") - I do the same with xclip.
Will do, thank you.

Just a side thought: wouldn't it be useful to also expose the recursive_substitution functions publicly (or a similar version of it)? I have no example in mind, but maybe some people may have the need to run a global search and replace on all their notes. What do you think?

@lambtho12 tell me when you're ready. The guarded save-all seems like an OK compromise. Tell me what you think. Shall we merge?

@renerocksai just managed to save only telekasten buffers :partying_face:.
But this is no ready to merge yet. I would like to make the sed command a bit more robust (using regex to ensure we are indeed in a link for instance).
On top of that, I still have the following: sed: couldn't close stdout: Bad file descriptor message output, even tough everything works fine.
Whenever it will be ready for merge I will drop the "draft" tag in the title and tag you. Hopefully later today.

@renerocksai this should be good to go. Please review everything just to make sure, I tried to think about all the most common situations and prevent basic issues (people trying to move stuff in non existing directories, or giving name of an already existing note). Hopefully this should be all good.
(do not hesitate to squash all the commits before merging, because this is a bit ugly :smile: )

@renerocksai , this PR is normally ready to merge. Do not forget to have a look and merge it whenever you have some time.

This looks OK, I trust that you've tested it. Will merge now.

Awesome BTW!!! The only thing I'd nitpick about is the rg ..... markdown business which will not work for people who have custom file extensions, so this is something to think about for the future...

Awesome BTW!!! The only thing I'd nitpick about is the rg ..... markdown business which will not work for people who have custom file extensions, so this is something to think about for the future...

Oh I did not thought of that... I opted for ripgrep because it is supposedly faster than find and it was already a prerequisite for telekasten (as telescope uses it). I can change it back to find with M.Cfg.extension so this problem should be solved.

Is it possible to use « the one used by Telescope » as a default ?
This abstraction layer could degrade a little the perfs, but make the configuration simpler, and ensure compatibility with any method one can configure for Telescope (Using find on Windows ?) : if Telescope works, Telekasten works :-)
(it is just a thought : I'm far away from using all this tools efficiently on a daily basis !)

Is it possible to use « the one used by Telescope » as a default ?

WHAAAAAAAT? There is a remplace tool from Telescope?!? If so, I am not sure it will update the links. I will have to look into it.

While I see that telescope checks for multiple file search commands (find, rg, where, ...) I am not sure that dropping the rg requirement is really helpful.
Without rg you can't do live grep in telescope and telekasten uses rg in a similar way when searching for tags, headings, parapgraph ids, etc.
So instead of removing rg from the rename feature, I'd double down on rg and list it as a requirement for full functionality of telekasten. I thought this was clear - if telescope has it as a requirement for searches (within files) that telekasten inherits this requirement.

Agree with @renerocksai . I already specified ripgrep as a required utility in the wiki I am currently writing


Delete move and copy of virtual *GradState classes

Following practices to avoid object slicing.

Jenkins, test this please

Jenkins CI test (for commit 6fdcd4a1dc0927b9bc09f7df8dd55b4733dfd220, target branch master) succeeded!


Create a gradle plugin to generate webpack config to inline metadata files

i.e. autogenerate https://github.com/luca992/libphonenumber-kotlin/blob/master/gradle/pack-library-resources.gradle.kts

I don't think this is needed the metadata files seem to be successfully distributed when published


Unit test: add tests for distances  estimation (standarized test)

What?
Add tests for the distance estimation method

Think the best solution is to take two images from this dataset: https://robotics.dei.unipd.it/reid/index.php/8-dataset/9-overview-face

Dataset:
IAS-Lab RGB-D Face Dataset
Using the provided
Link to the dataset home page:
https://robotics.dei.unipd.it/reid/index.php/8-dataset/9-overview-face
Link to the dataset download page:
https://robotics.dei.unipd.it/reid/index.php/downloads
Citations
Authors recommend citing this
M. Munaro, S. Ghidoni, D. Tartaro Dizmen and E. Menegatti.
"A feature-based approach to people re-identification using skeleton keypoints".
In Proceedings of IEEE International Conference on Robotics and Automation (ICRA), Hong Kong (China), pp. 5644-5651, 2014.
M. Munaro, A. Basso, A. Fossati, L. Van Gool and E. Menegatti.
"3D Reconstruction of freely moving persons for re-identification with a depth sensor".
In Proceedings of IEEE International Conference on Robotics and Automation (ICRA), Hong Kong (China), pp. 4512-4519, 2014.

Needs review
Struggle to test. Gets as a result that the distance is 216 cm for images where it should be 100 cm. Dont know how to fix. Dont know what is wrong. No clear way to find focal lenght or the sensor width of the camera. No realiable way of testing. Either the equation is wrong or the units in the equation does not correlate
@marhjoh @haugeSander


Rework media uploader class

I plan on reworking the MediaUploader class to work with a number of different sources - i.e  a file contents string, a stream or a url. This will make the class less restrictive, as currently you can only upload media from an UploadedFile instance.
Consider the following api:
MediaUploader::fromStream($stream)->upload();
MediaUploader::fromString($string)->upload();
MediaUploader::fromUrl($url)->upload();

The class is also very tightly coupled with Laravel's config helper, which is generally considered bad practice for testing purposes. I'm contemplating accepting Laravel's Config\Repository - or perhaps an instance of the desired media model and target disk - in the constructor, and then binding an instance of the class to the service container. This means that it can be dependency injected or loaded via a facade. Another benefit of this would be that I could make all the from methods non-static.
For example:
class MediaUploader
{
    protected $config;

    public function __construct(Config $config) // or (Media $model, string $disk) ?
    {
        $this->config = $config;
    }
}

It could then be used in your application like so:
// Via a new facade
Facades\MediaUploader::fromFile($file);

// From the container
$injectedMediaUploader->fromFile($file);

It would be nice if MediaUploader also accepted uploads to different type of disks. It should be handled by adding optional parameter to upload function

I've been experimenting locally and a setDisk method will definitely be included in the next version.
I've also decided against the proposed fromStream and fromString methods in favour of a fromPath method so that the file reading process will be handled in the class.

Any updates for setDisk method. Of course I can do some workaround but it should be supported.

@jenky I've just added this in v0.4.0 😄

Addressed in next version, see next branch.


A little complex first example

In my opinion, your first example should be the localStorage one. A lot more simple an easy to understand.
Maybe a optimistic update example too?

Optimistic update example would certainly be nice. About using the localStorage example first, the original intention was to focus on the fact that asynchronous side effects can be operated that way.
At this point though I'm not 100% sure what direction should this project take, since there are simpler ways of achieving the same. It got somewhat overridden by https://github.com/xaviervia/tessellation
@pirelenito you showed me a solution for wiring side effects to redux that looked pretty neat, more cursor-like than this library. Do you think we could make that into a lib?

It is currently part of an internal project, but we should be soon able to extract it.

"you showed me a solution for wiring side effects to redux that looked pretty neat"
@pirelenito @xaviervia
now, i'm curious. what is it about?

@bedorlan I'll hope to be able to post soon, but essentially it's a wrapper around the subscribe method of the store that allows you to listen for changes on a specific "path" of the state. For example, say you have a state with the structure:
{
  shared: {
    articles: [
      { ... }
    ]
  }
}

you can provide a selector that will return all articles and a callback and the helper will only call the callback when the articles change. As far as I remember the API looks something like:
const selector = state => state.shared.articles
const callback = ({getState, dispatch}) => console.log(getState().shared.articles)

const subscriber = onChange(selector, callback)

store.subscribe(subscriber(store))

Using the selector like this makes it feels closer to Elm's cursors, and the getDiff from redux-haiku is unnecessary (which I think it's a good thing since it can be expensive). Also the subscriber is rather easy to test in isolation.
I hope we have time soon to work on publishing this solution.

@bedorlan here is a first iteration of the idea: https://github.com/batata-frita/redux-heat/
But bear in mind that it is going to change 😉 (probably a lot)


refactor: update tsconfig and its refs

** PR Checklist
Please check if your PR fulfills the following requirements:

[ ] The commit message follows our guidelines: https://github.com/ng-seed/universal/blob/master/CONTRIBUTING.md#commit
[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)

** PR Type
What kind of change does this PR introduce?

[x] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[ ] Documentation content changes
[ ] Other... Please describe:

** What is the current behavior?

Issue Number: #101
** What is the new behavior?
** Does this PR introduce a breaking change?
[ ] Yes
[x] No

** Other information

Codecov Report

Merging #102 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #102   +/-   ##
=======================================
  Coverage   61.48%   61.48%
=======================================
  Files          37       37
  Lines         470      470
  Branches       44       44
=======================================
  Hits          289      289
  Misses        181      181

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 47185c8...1bb1d6f. Read the comment docs.

Codecov Report

Merging #102 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #102   +/-   ##
=======================================
  Coverage   61.48%   61.48%
=======================================
  Files          37       37
  Lines         470      470
  Branches       44       44
=======================================
  Hits          289      289
  Misses        181      181

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 47185c8...1bb1d6f. Read the comment docs.


Engine not saving the model if it contains an Optional EmbeddedModel set to None

Bug
I have an Optional EmbeddedModel inside my main model. If I don't set a value for it, when the engine tries to save the data in the database, I get the error: TypeError: 'NoneType' object is not iterable
Current Behavior
This is a basic example that shows this behavior (it's the same with the AIOEngine):
from odmantic import Model, EmbeddedModel, SyncEngine

class Profile(EmbeddedModel):
    name: str
    age: int

class User(Model):
    username: str
    password: str
    profile: Profile | None

engine = SyncEngine()

user = User(username="John", password="password")
engine.save(user)

Expected behavior
The engine should save the model even if the EmbeddedModel is initially not set.
Environment

ODMantic version: 0.9.0
MongoDB version: 6.0.1
Pydantic infos (output of python -c "import pydantic.utils; print(pydantic.utils.version_info())):

             pydantic version: 1.10.2
            pydantic compiled: True
                 install path: D:\Projects\Python\odmantic-test\.venv\Lib\site-packages\pydantic
               python version: 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
                     platform: Windows-10-10.0.19043-SP0
     optional deps. installed: ['typing-extensions']

Hello, thanks for the bug report.
I'll fix this asap.

Hello. Thanks for the library.
Any predictions on when this will be merged?

Hello, this is still an issue in odmantic 0.9.2

Seeing same bug in 1.0.2 odmantic


initialConfig is not working

Please check initial-config attribute, it is currently not accept object. I have debugged it and saw in your source code: (c3-angular.js line 861)
var initialConfig = attrs.initialConfig; // return string ???
My html:
<c3chart bindto-id="chart_test" initial-config="myOptions">
My controller:
$scope.myOptions = {data: {}};
Am I missing something?
Thank you anyway!

I am also having the same issue
<c3chart bindto-id="pie-chart-1" show-labels="false" chart-columns="pieColumns" chart-data="pieData"/>

$scope.pieChartInitialConfig = {
			data: {
				data: [
					['data1', 0],
					['data2', 0],
					['data3', 0],
					['data3', 0],
					['data4', 0],
					['data5', 0]
				],
				type: 'pie'
			},
			legend: {
				show: false
			}
		};

Console Log
    at ChartController.d [as showGraph] (c3-angular.js:1180)
    at c3-angular.js:898
    at angular.js:19902
    at completeOutstandingRequest (angular.js:6108)
    at angular.js:6387```


Gatling cannot be used

The latest version of gatling uses Netty 4.0 whereas grpc requires version 4.1.
Gatling needs to be upgraded to a version that includes https://github.com/gatling/gatling/commit/1307add32901d92a50313ee161d2c06c45cc5123

Not in gatling 2.2.5. Will probably be for 2.3

Nope. Not in 2.3...

Fixed on master for JH5 app


Problem loading the model from checkpoint

Training the model with train_script.py, I save for all the epochs only the weights using the following code:
# ModelCheckpoint callback to save model weights
checkpoint_callback = ModelCheckpoint(
    filepath=os.path.join(checkpoint_path, "SCDD_vgg_unet_epoch_{epoch:02d}.h5"),
    save_weights_only=True,
    save_best_only=False,
    monitor='loss',
    verbose=1
)

while the predictions made after the training are correct the predictions made by the predict_script.py, where I load the model weights from checkpoint with the following code does not work properly.
# Load the model
if os.path.exists(checkpoint_path):
    model = vgg_unet(n_classes=wandb.config.n_classes ,  input_height=wandb.config.input_height, input_width=wandb.config.input_width)
    model.load_weights(checkpoint_path, by_name=True)
else:
    raise FileNotFoundError(f"Checkpoint not found at: {checkpoint_path}")

predictions after training
predictions using checkpoint

Making sure it is in inference mode
tried the following to make sure the model is in inference mode, but the predictions were not right again.
if os.path.exists(checkpoint_path):
    model = vgg_unet(n_classes=wandb.config.n_classes ,  input_height=wandb.config.input_height, input_width=wandb.config.input_width)
    model.trainable = False
    K.set_learning_phase(0)  # Set to inference mode
    model.load_weights(checkpoint_path)

Loading model from latest checkpoint is solved
Using the CheckpointsCallback from the train.py' file I managed to have the checkpoints properly saved. The latest saved chackpoint is successfully used in 'predict_script.py utilizing the pre-defined model_from_checkpoint_path function (it is defined in predict.py file)
# Initialize the CheckpointsCallback
checkpoint_callback = CheckpointsCallback(checkpoint_path)

# Train the model with the custom callback
model.train(
    train_images=train_image_path,
    train_annotations=train_annotations_path,
    checkpoints_path=checkpoint_path,
    epochs=wandb.config.epochs,
    batch_size = 2,
    steps_per_epoch=len(os.listdir(train_image_path)) // 2,
    callbacks=[WandbCallback(), checkpoint_callback]
)

and in predict_script.py usign the model_from_checkpoint_path I loaded the model from the latest checkpoint:
# Load the model
if os.path.exists(checkpoint_path):
    model = model_from_checkpoint_path(checkpoint_path)
else:
    raise FileNotFoundError(f"Checkpoint not found at: {checkpoint_path}")

TO DO: Load model from an arbitrary checkpoint


Cannot connect from Windows 11 to Fedora 35 via Xrdp

I just installed Fedora (upgrade from Ubuntu)  and I cannot get xrdp to allow me to login remotely
Everything installed correctly.  But here is the message I am getting when connect with Windows Remote Desktop
Sending login info to session manager.  Please wait... login failed for user phsonnek
The tail of /var/log/xrdp.log
[20220327-16:17:21] [INFO ] connecting to sesman on <IP_ADDRESS>:3350
[20220327-16:17:21] [INFO ] xrdp_wm_log_msg: sesman connect ok
[20220327-16:17:21] [INFO ] sesman connect ok
[20220327-16:17:21] [INFO ] sending login info to session manager. Please wait...
[20220327-16:17:23] [INFO ] xrdp_wm_log_msg: login failed for user phsonnek
[20220327-16:17:23] [INFO ] login failed for user phsonnek
I am not sure where to even start with this.

It's simply telling you that it can't authenticate 'phsonnek'.

Have a look in /var/log/xrdp-sesman.log, where you might get more information.
Also, check your keyboard mapping is working correctly for the password by entering your password characters into the username field, to check they are being processed correctly.

Fedora uses SELinux, and you may need to edit /etc/pam.d/xrdp-sesman to get the session to transition to the correct SELinux context. There's more information on this buried in #2094

So its just not installing xorg. If you try to use sudo apt install xorgxrdp it will try to install it but you get "package xorgxrdp not available, but is referred to by another package. This may mean the package is missing, has been obsoleted, or is only available from another source."

We don't do videos here I'm afraid - they make it hard for other people reading the thread to follow the discussion.
What do you get for these commands:-
apt search xrdp
xrdp -v


Fixed error in dark theme

Description

When the theme is dark, the legend text is not visible in the chart editing screen.
The paging text of the chart legend is invisible when dark themes.
I can't see the Edit Description button for Wrangled Dataset when it is dark.
When the theme is dark, the text is not visible when editing the title.

Related Issue :

How Has This Been Tested?

Check that the text is displayed correctly when the legend is displayed on the chart editing screen.
Verify that the paging text of the chart legend is displayed.
Under Preferences, verify that the Modify Description button for the Wrangled Dataset is displayed.
Make sure the text is visible when editing the title / description.

Need additional checks?
Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[ ] I have read the CONTRIBUTING document.
[ ] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have added tests to cover my changes.

Additional Context

run build
deploy to 1


bluetooth pairing keys install

I would like to install pre-existing bluetooth pairing keys with Ansible to make BT mouse work after system install.
Is it possible?

I don't have such a script, but here is the code to copy BLE keys from Linux to Windows:
https://github.com/x2es/bt-dualboot/pull/14/files

The description is in:

GitHub https://github.com/x2es/bt-dualboot
PyPi https://pypi.org/project/bt-dualboot/

Quote: "Sync Bluetooth for dual-boot Linux and Windows"
If you install bt-dualboot the simple task below should do the job
- hosts: all
  tasks:
    - command: 'sudo bt-dualboot --sync-all'

Let me know if this works for you. We can add it to the role then.

What I can see from the doc, --sync-all is suitable for syncing all the devices between Linux and Windows.
What I need is:

pull the keys from existing clients as a backup measure
sync the backed up keys with Linux hosts needed

I see. Unfortunately, I have no Linux/Windows dual-boot to test it. Still the same Let me know if bt-dualboot is working for you. Feel free to submit a PR.

the syncing between Linux and Windows works fine, I tested it

Good to know. Feel free to submit a PR.

I thought maybe the measures to gather bt keys as part of other hw info are already exist?
That would be start I guess. And then the copy itself.
I'm not familiar with Ansible architecture enough to say that.

Alternatively, if it possible to forcibly set a key for BT connection, that would be good to know

Alternatively, if it possible to forcibly set a key for BT connection, that would be good to know

You can ask this in the bt-dualboot project.

OK, so the idea is to handle it the same way as WiFi key then.
IIUC, the script above does reading the key from Linux configs, we would need to rework it so it could write instead.
I'm not a python programmer so some help would be welcomed.
There are also many approaches supporting Windows to Linux sync, those might be more interesting for our case as they support "write" to Linux from exported file:
https://github.com/x2es/bt-dualboot#alternatives
We would need to investigate which of the projects suits best for our needs.


Don't allow adding or removing review requests for a merged PR

Resolves #29965.

Manually tested this by:

Following the installation guide (but built a local Docker image instead)
Creating 2 users, one who is the Owner of a newly-created repository and the other a Collaborator
Had the Collaborator create a PR that the Owner reviews
Collaborator resolves conversation and Owner merges PR

And with this change we see that we can no longer see re-request review button for the Owner:

Maybe closed pull requests cannot be reviewed anymore.

Maybe closed pull requests cannot be reviewed anymore.

Thanks for the quick response! I actually never seen such an edge case in the wild before.
I also just realized that in the issue it explains they can still re-request or delete a review request when the PR has already been merged. So if I understand this correctly, this looks like more than just hiding a button when a PR is closed/merged; rather, some backend logic needs to be changed so that users cannot re-request a review on a merged PR. Still kind of fuzzy how closed PRs should be handled, but I'll make this a draft PR and see if I can resolve this.

I believe I found the culprit route handler, that being UpdatePullReviewRequest(). I manually tested the code path by doing what I did in my original description (this time running Gitea in a dev environment rather than having to fire up the containerized app). I got the following logs when trying to re-request a review of a merged PR:
2024/03/25 10:43:00 ...rs/web/repo/issue.go:2417:UpdatePullReviewRequest() [W] UpdatePullReviewRequest: cannot add or remove a review for merged PR gek/test-repo#3
2024/03/25 10:43:00 ...eb/routing/logger.go:102:func1() [I] router: completed POST /gek/test-repo/issues/request_review for <IP_ADDRESS>:56852, 403 Forbidden in 32.2ms @ repo/issue.go:2387(repo.UpdatePullReviewRequest)

I'll make this PR ready-to-review. I do have some questions/comments:

I did not see any tests for UpdatePullReviewRequest(). Should I add one to account for this bug and if so should I add it to issue_test.go?
Is there an api/v1 route I should look out for too? Maybe /api/v1/repo/pull_review.go?
Closed PRs were discussed here. I believe closed PRs can still be reviewed since the HTML template if check I add was just to check if the PR was merged, not if it was closed (which I think we can do $.Issue.IsClosed for that). Then again I am a complete newbie so I may be wrong.

Thanks in advance for your time!

I checked on GitHub, it's not possible to submit a review it either closed or merged states.
BTW, we don't have to blow up this PR with more scope extensions, I could accept it as-is.

I tested changing reviews and GitHub does allow adding/removing review requests in both merged and closed states. Only the "re-review" and "dismiss" options are not possible in those states in GitHub.

I tested changing reviewers and GitHub does allow adding/removing review requests in both merged and closed states. Only the "re-review" and "dismiss" actions are not possible in those states in GitHub.

Thanks for the review! Reading this, it looks like simply throwing a http.StatusForbidden when requesting for a review on a merged PR is not good enough. I guess I need to figure out if we were given a re-review or dismiss action for the reviewer. Looking so far, I'm unsure if this is something the frontend communicates but I'll see if I can find a solution 👍

Most recent change also considers closed PRs when a user is trying to perform a re-review or dismiss request.
Will look into how unit testing works here and how I can cover these code paths.

My recent commit does the following:

Rollback failures in review.AddReviewRequest() instead of committing them
Throw the PR merge checking logic under a if issue.IsPull branch
Fix a UI logic bug
Fix error messages to include both  closed and merged

I have added a couple of unit tests to cover the code paths I've added. Let me know if I should add more or make improvements.
Next I'll scour the API routes looking for any signs of issues similar to this one.

(Sorry for the repeated force-pushes)

Made the necessary changes to the API and from testing using the swagger API client it looks like it works as expected. I haven't seen any tests related to the API code, but let me know if I should add some for my changes.


Extract images from ImageSet.

Consider custom ImageSet resources when extracting image references for vendoring.
Requires https://github.com/gravitational/lens/pull/1, closes https://github.com/gravitational/gravity/issues/676, closes https://github.com/gravitational/gravity/issues/680.
@benarent Please review the docs.

Need more time for lens PR though.

@a-palchikov @knisbet FYI moved everything from the lens PR here. I think it's actually quite nice as we now have a framework in Gravity to add more custom resources should we need to.
I'm going to merge this once CI passes.


Redirect Library OPAC URLs

Redirect OPAC URLs to https://wellcomecollection.org/works/{work_id}
Example URL:
https://catalogue.wellcomelibrary.org/record=b2475299

Link to feature: https://wellcomecollection.productboard.com/feature-board/2024994-current-quarter-by-hierarchy/features/6699469/detail


post context properties to kodahash/render

Closes #5

@vikiival
i can't assign you as reviewer

@daiagi have you tested that on art that hasn't been consistent?

i have

Before

After


Implement SIMD for ARM architectures

Should be possible by using PMULL/PMULL2.

Implemented for aarch64, armv7 left.
Runtime feature detection with nostd if at all possible


Grabber Post-Filtering: Making Advanced Searches Possible Without Additional API Interfacing

@Bionus First post here so thanks for imgbrd-grabber. It works great.
To be as concise as possible, I either 1: Don't understand how post-filtering is supposed to work or 2: Am suggesting change to it which would allow very specific search without. having to deal with complicated API stuff. I believe it's the latter.**
Currently, Grabber can only retrieve several tags at a time due to site-placed restrictions. However, it can retrieve all tags simultaneously. Suggestion: Allow advanced Boolean Logic, done locally, to filter
tags.
For example:
Search for: Cat
Post-Filtering Tags: (Fluffy AND **-**White) AND (FileType:.webm OR FileType:.mp4)

Retrieve all images and tags as usual for the tag "Cat"
Apply blacklist as usual; eliminate blacklisted images from the results/further filtering
Client computer locally filters image tags to display files with tags indicating a Video of Fluffy cats that are not White, without querying sites.

**I'm not sure how post-filtering works, my bad if I've misunderstood it and you can already do this. I'm taking a bit of a stab in the dark saying this, but it seems like the blacklist already works on a local basis? Would it be overly complicated to expand the blacklist function to support Boolean filters?

Hello 👋
I'm not sure about your exact use case but that's indeed what post-filtering does (filtering images "post" loading): first load the images using the search input, then remove the blacklist + all images not matching the post-filter.
It's documented here and it should support parenthesis, "&&" and "||" for complex filters, although I did not try it in a while. Have you tried using those instead of "AND" and "OR"?

I've only gotten post-filtering to work once or twice. To my memory, every time I try to use it, even if I type into the post-filter only 1 tag that I've verified has images in common with the primary search tags, the search results just come back empty. Absolutely nothing.
(I'm still tweaking my filename syntax, so I'm sure it's possible I'm misusing it.)

Indeed, post-filters right now do not support parenthesis or anything, just basic tag or token filtering. Switching to bug report as it ought to be consistent with other conditionals.


SCCM Remote Control - view only 1 of 2 monitors

We are running System Center 2012 R2 Configuration Manager Remote Control version 5.00.7958.1000
When trying to remote into a desktop with two monitor (single dual adaptor) we are only seeing the left side monitor. We manage 450 different type of desktops and so far the issue appears to be arbitrary. All desktops are Windows 10.
Have been using SCCM remote for last 8 years and never saw this.
Any ideas?

Hi @timuze,
Thanks for taking the time to share this with the Configuration Manager documentation team. I’m sorry that you’re having problems, and I want to make sure it gets to the right people that can help. There are a couple of options available to provide the best level of support for this:

Post on Configuration Manager forums. These are moderated by our community MVPs who are very experienced, knowledgeable, and helpful.
Use Microsoft Support to search for the issue, or open a case with professional support.
Contact your Microsoft Premier team, such as a Premier Field Engineer or Technical Account Manager.
I'm closing this issue as there's nothing actionable for docs at this time.

FYI, Configuration Manager current branch version 1902 has improved support for this scenario. For more information, see What's new in version 1902.


[HSEARCH-1059, HSEARCH-1060]; fix resource lookup, desearialization.

@alesj thanks, one patch is in master already, the JGroups one is replaced with a new pull request:
https://github.com/hibernate/hibernate-search/pull/251
Your fix was good but I wanted to change the way we where using JGroups anyway as we have our own serialization now.


screenshare not working properly

My Setup

Operating System:windows 10 pro 64 bit
App Version:2.14.1
Installation type: exe

[x] I have tested with the latest version
[ ] I can simulate the issue easily

Description

i have integrated "Dead simple screen share" but i am not getting the link when i send the message "sharescreen"
Current Behavior

should get a join meeting link
Expected Behavior

nothing
additional info

Data Passed to Integration
{
"token": "s8FsXefttKSnTD2yxGD5Cbbh",
"bot": false,
"trigger_word": "screenshare",
"channel_id": "DruSPcwMZvenC2iAmbKqDsNjHBRTcxEXLc",
"channel_name": null,
"message_id": "NhnLGW83ZoDcbkxHR",
"timestamp": "2018-10-31T10:21:14.433Z",
"user_id": "DruSPcwMZvenC2iAm",
"user_name": "lokesh.nayak",
"text": "screenshare",
"alias": "Lokesh Nayak"
}
Data Passed to URL
{
"token": "s8FsXefttKSnTD2yxGD5Cbbh",
"bot": false,
"trigger_word": "screenshare",
"channel_id": "DruSPcwMZvenC2iAmbKqDsNjHBRTcxEXLc",
"channel_name": null,
"message_id": "NhnLGW83ZoDcbkxHR",
"timestamp": "2018-10-31T10:21:14.433Z",
"user_id": "DruSPcwMZvenC2iAm",
"user_name": "lokesh.nayak",
"text": "screenshare",
"alias": "Lokesh Nayak"
}

Please test if the latest release did solve this. If not please reopen the issue or create a new one and tell us. I will be closing this one for now.
Thank you!


ArrayIndexOutOfBoundsException on lookup in MapDoubleArray

Hi. I am using trie4j to store natural-language dictionary entries. With a particular dictionary index and a particular lookup word I am getting an ArrayIndexOutOfBoundsException on MapDoubleArray.get(). Sample code:
MapTrie<Object> mpt = new MapPatriciaTrie<>();
Files.lines(Paths.get("dicttest-Harrap_s_Business_fran_ais_ang.idx.txt"))
        .forEach(word -> mpt.insert(word, null));
MapDoubleArray<Object> da = new MapDoubleArray<>(mpt);
System.out.println(mpt.get("you")); // OK
System.out.println(da.get("you")); // Throws exception

The test file loaded above is available here: https://gist.github.com/amake/ba10166a7bf59ffac180defcecbec8d4
The lookup word "you" is not contained in the index, so I expect both get() calls to return null; however the second one results in this exception:
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 128198
	at org.trie4j.doublearray.DoubleArray.getNodeId(DoubleArray.java:212)
	at org.trie4j.doublearray.DoubleArray.getTermId(DoubleArray.java:220)
	at org.trie4j.AbstractTermIdMapTrie.get(AbstractTermIdMapTrie.java:154)
	at test.main(Test.java:13)

This occurs with version 0.9.4.

Thanks @amake !
I confirmed the issue and reproduced the problem with following data extracted from yours:
seu o
yen

This issue will be fixed in a day.

0.9.5 https://github.com/takawitter/trie4j/releases/tag/0.9.5 released.
It takes a while to be available at maven central repository.

It is available for me already, and I can confirm that the issue is fixed. Thanks for the very fast response!


Add mapping argument to preprocessor methods for reusing and updating mappings

This pull request introduces a mapping argument to the Preprocessor.sequence() method and other preprocessing methods for different formats.
Problem
Currently, there is no option to reuse an existing mapping. It can be a problem in scenarios where a train-test split is done before preprocessing (e.g. when working with pre-split datasets). In such cases, the mappings created by preprocessor can be inconsistent.
Example
preprocessor = Preprocessor(length=30, timeout=60, NO_EVENT=-1337)
X_train, y_train, label_train, mapping_train = preprocessor.csv("train.csv", verbose=True)
X_test, y_test, label_test, mapping_test = preprocessor.csv("test.csv", verbose=True)
print('Mapping train:', mapping_train)
print('Mapping test:', mapping_test)

output:
Mapping train: {0: 4, 1: 7, 2: 8, ... , 129: 219, 130: 222, 131: -1337}
Mapping test: {0: 0, 1: 4, 2: 6, ... , 129: 218, 130: 219, 131: 222, 132: -1337}

Solution
The proposed change allows the use of a previously created mapper by passing it through the new mapping argument. If any unseen event IDs are encountered, the mapping is updated accordingly.
Example
preprocessor = Preprocessor(length=30, timeout=60, NO_EVENT=-1337)
X_train, y_train, label_train, mapping_train = preprocessor.csv("train.csv", verbose=True)
X_test, y_test, label_test, mapping_test = preprocessor.csv("test.csv", verbose=True, mapping=mapping_train)
print('Mapping train:', mapping_train)
print('Mapping test:', mapping_test)

output:
Mapping train: {0: 4, 1: 7, 2: 8, ... , 130: 222, 131: -1337}
Mapping test: {0: 4, 1: 7, 2: 8, ... , 130: 222, 131: -1337, 132: 6, ... , 137: 155}

Great new feature, thank you for the pull request, I've merged it!


Inject Admincp

For some reason the module can be inject and get the sql details and admincp

Can you include a PoC? I would like to secure my website

Invalid issue


Set up nix-shell dev env

We don't need Haskell language server in CI so we should skip compiling that, hopefully reduce the build times 🤞

We don't need Haskell language server in CI so we should skip compiling that, hopefully reduce the build times crossed_fingers

down to sub 10 minutes, worth the time it took to work out how to do that.


Allow only alphanumerical and hyphen in record

Close #437

Update regex to allow only alphanumeric characters and -
Record name can't start or end with -
Record name can't have consecutive - in the middle

@Genne23v @humphd
Correct, I think is not needed, at least not the part that is limiting the use of _ characters

Cool, let's take that bit out, and fix @Genne23v?

@humphd I think I can just close this PR. The change is limiting underscores in subdomain.

Sounds good.


Support CRDs

To allow Flux to be used with tools such as seldon-core that uses Custom Resource Definitions. This would also facilitate surrounding projects such as kubeflow creating "GitOps" Machine Learning e2e pipelines: https://github.com/kubeflow/kubeflow/issues/971 as well as other examples such as #1038

Hi @cliveseldon can you please post a CR example

A simple example
A more complex example

The proto definition:

docs
raw proto

@cliveseldon Are you expecting WeaveFlux to automatically substitute the images into the manifests and that's why you think WeaveFlux needs to know about CRDs?
The trend (e.g. helm v3 discussions) seems to be that the tool chain should be composable. So my expectation is that people templatize their app using whatever tool they like (ksonnet, kustomize, helm, etc...) and then down stream tools in the app mangement tool chain (e.g. Weave) just consume the fully realized YAML manifests.
Within Kubeflow we have a number of places (e.g. hyperparameter tuning) where we need to substitute parameters into some user defined manifest.
So I think the direction we are headed is creating an API like
getManifests(Git URI)
that could return the manifests based on a URI that could be pointing at an app written in any number of templating solutions (e.g. helm or ksonnet). So a tool like WeaveFlux wouldn't necessarily need to know about either 1. The particular structure of the resources or 2. the particular implementation of the templating solution. It would just need an endpoint that knew how to evaluate one or more templating solutions

@jlewi That makes sense.
Maybe I'm not clear then why you would need WeaveFlux if the image repos and manifests have been updated downstream the remaining functionality is to simply push those new Manifest to the k8s cluster over the k8s API.
Maybe you can show how the API you mention should work with WeaveFlux? I'm not clear:

Whose responsibility it is to update the Deployemnt git repo with new image tags when a Docker image repo is updated
How WeaveFlux knows which images in a Docker Repo connect to a set of Manifests

Hopefully the Weave folks will chime in but my understanding is that WeaveFlux automates applying the manifests which is the functionality I'm looking for.

WeaveFlux will automatically monitor the GitRepo detect when the manifests and cluster are out of sync and roll them out.
WeaveFlux provides additional controls (e.g. a CLI and UI in the premium version) for managing the process

So here's what I have in mind.

We have a ksonnet app here that describes the dev kubeflow deployment (dev.kubeflow.org)
I'd like to use WeaveFlux to keep dev.kubeflow.org in sync with what's ever checked in.
For example:
* Kubeflow contributor submits a PR updating the ksonnet app
* WeaveFlux detects the changes and applies them.

In this case, the  Deployments in the Git Repo are updated manually. I would automate that upstream of WeaveFlux using a combination of  Prow, Cron, and Argo.
So as a specific example, we'd like to automatically push the latest Kubeflow changes to dev.kubeflow.org. Here's how that might work
 * On merging a PR into kubeflow/kubeflow Prow triggers a postsubmit job
 * The postsubmit job triggers an Argo workflow
 * The Argo workflow triggers the scripts we wrote to update the dev.kubeflow.org ksonnet app
 * Argo workflow submits a PR to commit those changes back to kubeflow/testing
 * WeaveFlux detects the changes and applies them
 * WeaveFlux monitors the rollout and if it fails it rolls back to the last known good release.

We could even have our Argo workflow emit the YAML manifests and check those in so that we aren't blocked on lack of ksonnet support in Argo.
/cc @thejaysmith

The UI seems to be more in line with the premium version. There is a fluxctl CLI tool available on the standalone version. I have nearly completed converting my Flux YAMLs to a ksonnet app so we'll see how it works. The Argo integration will be an interesting piece to try out. Interested in hearing the opinion of the WeaveWorks team.

This is a really interesting discussion, and echoes discussions we've had internally about the design and purpose of flux (and latterly, the helm operator).
You can look at flux as being three things, which are munged together for those historical reasons we're always hearing about.

something that applies config in git to a cluster, with some embellishments (like being able to ignore resources)
something which automates the update of image refs in manifests, with an API and command-line tooling
an image metadata database which you can query

Now, 2) and 3) obviously go together quite closely, but 1) is really its own thing that could stand alone. So you can treat the question of supporting CRDs (or Helm, or ksonnet) as having two parts: firstly, can flux apply the given flavour of config to the cluster in a sensible way? Secondly, can image updates be supported for the given flavour of config?
By "sensible" I mean that we can maintain the invariant cluster == git -- usually this means that the application is deterministic and idempotent, or close enough that we can paper over it (which we have to do for the helm operator).
Anyway, my answers thus far look like this:

Flavour
Sensible apply
Support updates

"Regular" Kubernetes YAMLs
Use kubectl apply
Yes, but requires interpretation

Custom resource YAMLs
Use kubectl apply
Needs interpretation per CRD

Helm charts
Needs a representation of "chart release" as a resource that can be actuated (we chose to make a CRD for this, called FluxHelmRelease)
Requires interpretation of the "chart release", and charts to be parameterised along certain conventions

ksonnet
Use ks apply ?
I don't know; maybe if parameterised in the right way

When I use "needs interpretation" above, I mean that flux has to have code that deals specifically with that kind of config -- so for example, we had to bake in code for updating FluxHelmRelease resources, since they necessarily differ from the regular Kubernetes resources.
In the case of ksonnet, and to some extent Helm, the image update automation may be not that important, since they have their own units of deployment which aren't images -- we may have got that slightly squint in the Helm operator design, I'm not sure yet. (In fact it may turn out that automated image updates are a pretty specialised requirement, that we happened to need ourselves for our own infrastructure.)

I am for this. Regardless of what templating solution is used, there needs to be a common YAML. We have a simple python script to convert the YAML into Jsonnet (I used that in part to create the ksonnet package for Flux) so just improving that could be used to revert jsonnet or whatever into a standard k8s friendly YAML.
For the larger issue, it seems like step 1 requires us to still use ksonnet so it forces us to maintain the jsonnet language and translate YAML back to a ksonnet format.

To respond to the original post:

flux will happily attempt to apply any YAML it finds, so if you can kubectl apply something, it most likely can too;
if you want to be able to view and update some kind of resource via flux, it needs to understand how to interpret it as a workload (i.e., something that specifies pods to be run). If it looks sufficiently like a "regular" Kubernetes resource, this isn't too difficult -- a modest extension, so one can say "look here for the container specs". Your examples look like they would be amenable to that approach (assuming they were translated to YAML, or flux gained JSON support.)

For Seldon's CRDs the images will be in contained PodTemplateSpecs which are part of the CRD.
The CRD also contains an OpenAPI validation spec if that is of any help.


Teach mull-cxx to read dynamic libraries from a binary

Currently, there is no way to load additional dynamic libraries via mull-cxx interface.
These files should be automatically derived from the binary.

Closed via #464


What about using MCJIT for crystal eval/run?

Hey guys.
First up, awesome job! :+1:
crystal eval uses tempfiles as it's output and then runs them, I'd like to avoid this, because crystal eval may get called absurdly often.
Same with crystal run.
What about using MCJIT and maybe a fallback interpreter, just like lli, instead of the tempfile way?
It would be a very nice alternative from my point of view.
I toyed around a little in the codebase, sadly I think I need a while to understand what is going on :P
Greetings,
vifino

Did you see this one for getting up to speed? https://github.com/manastech/crystal/wiki/Compiler-internals
Would be great if similar functionality like concepts and macros in Nim could become part of the language and utilize JIT tech or other way of avoiding writing an entire crystal interpreter in order to run macros etc. Disclaimer: I don't know much about the macros yet, they just looked simplistic in the doc..
On a side note: What if temp files where stored in temp ram disk - good enough?

@ozra I think RAM disks are insanely not-portable. If someone wants to use one, I believe they can override the temporary directory Crystal uses and create the disk themselves.

@vifino Hi! We are actually using the MCJIT for some specs, the ones that don't require the full prelude. As far as I remember, we had some issues with it and exception handling. For example you can write this program:
# interpreter.cr
require "compiler/crystal/**"

program = Crystal::Program.new
program.run File.read(ARGV[0])

Compile it:
crystal build interpreter.cr

And give it this file:
# boom.cr
require "prelude"

puts "hello"
if 1 == 1
  raise "OH NO"
end

Now run it:
CRYSTAL_PATH=~/Projects/crystal/src ./interpreter boom.cr

(I'm setting CRYSTAL_PATH to the src directory of the compiler because the default one is just "libs"). This will result in a segfault and I have no idea why. Maybe if we solve this we can switch to use the MCJIT.
BUT... what if the MCJIT doesn't behave exactly the same as an executable? (maybe because of a bug in the JIT). Then we will have to deal with two "implementations" of the language, which makes things harder.
The same applies if we ever decide to interpret part of the language: we will have to maintain two different behaviours.
Now I'll talk a bit about macros :-)
The reason why macros are simple is because they are intended for simple things. If you want to do a complex thing you can use the run macro method, in which you get to program in Crystal, its full set, not just a subset. This means there's only one language and one implementation.
I personally (and this is just my personal opinion, my guess is that @waj and @bcardiff  might disagree with me) don't like abusing macros because then it becomes harder to understand the code. It makes you spend more time thinking about how to make the code cleaner, avoid redundancy and squeeze the compiler to its maximum rather than just focus on the problem at hand. The macros I remember in the std are:

getter, setter, etc., which are just to avoid a small boilerplate
pp: just for nicer debugging
record: to define an immutable struct with some properties (like a record :-)
spawn: to abstract away the fact that a Fiber is created
ecr: to transform a file in another language to crystal (with this you could implement things like haml, slim, etc., although all of these are less powerful than in Ruby)
some macros in Enum to traverse its constants and provide to_s or parse methods
macro defs to_s and inspect: just iterate through the instance variables to show them (they are also used for Struct#== and Struct#hash). def_hash, def_equals, etc. also fall in this category (using the instance variables of a type for something)
json_mapping: which, given a description of a JSON object, basically generates a specific JSON parser, but this is made possible very easily because other things (notably a pull parser) are already implemented. The macro itself is less than 100 lines of code and relies on no other macros. To get to this point we first thought how would we make it without a macro and just then figured out the pattern that needed to be extracted.
marshal: this is in a branch, but @waj was working on this. It basically inspects an object's instance varaibles at compile time to know what needs to be marshalled. This could easily be done without macros, but this way it avoids boilerplate code.

I think macros should be used to avoid small boilerplate. But once it becomes hard to understand the macros themselves I think that's when one is abusing them. But, again, this is just my personal opinion. I also feel that successful (at the industry) languages have no, or a limited form, of macros. I think that makes everyone being capable of understanding what's going on without having to decipher obscure code.

@asterite I see you don't mention metaprogramming. Ruby is said to be strong in this aspect and for a compiled language that don't want to rely on runtime reflection, macros is the best choice in my opinion.

@ozra: Thanks, didn't see that before! :+1:
Hello @asterite!
That is awesome!
I'm thinking that making it optional, with a flag like --mcjit or via environment variables, would be awesome.
While I don't think there is much of a difference since they are going the same code paths, I do see the problem.
Maybe implement it and use it only for crystal eval by default, making it available as a flag for crystal run?
Sadly I can't run the interpreter :(
[vifino:~/code/crystaljit]$ CRYSTAL_PATH=~/code/crystal/src ./interpreter boom.cr
LLVM ERROR: Cannot select: 0x48c4c70: i64 = X86ISD::WrapperRIP 0x48bc2d0 [ORD=2] [ID=6]
  0x48bc2d0: i64 = TargetGlobalTLSAddress<i32* @errno> 0 [TF=10] [ORD=2] [ID=4]
In function: *FileDescriptorIO#unbuffered_write<FileDescriptorIO, Slice(UInt8), Int32>:Int32

@vifino what code are you feeding the interpreter? For example if you always raise instead of doing it conditionally then it crashes, but that's a known issue (but doesn't matter much because you can't trigger it via the compiler).

@asterite: I'm running your example code, compiled with yesterdays git :)

The interpreter example above: does the run method use mcjit? Or did I get that wrong?
Regarding macros (at the risk of OT tainting this issue). There are successful languages in the industry that are homo-iconic, and they're growing in numbers.
Not all code can be understood by all people. In projects I've worked on there's often a "black box" part of the system that is way more advanced - and risky - then the rest, in order to make the rest of the coding effortless to others on the team working "on the surface" of the system.
I find macros very good for facilitating and making development safer and with less boiler plate for a majority, while maybe one or two coders are concerned with the gory intestines, which often can be hardened and then rarely have to be touched once in place. It's great for DSL like things.
Perhaps eval as complement to run should be avail in macros, keeping the "external code" in the macro?
Out of the box crazy idea: Would it be possible to pass an entire function body / block to a macro, send that off as a string via 'run', use the compiler class in the external file (like your example) to parse it, then work on the AST, let the compiler render it to crystal source again, then return it to the calling macro!?
How separated are the compilation stages when using the compiler class?

@vifino Strange, I'll try to reproduce it.
@ozra That's one thing that you can do with run. For example there's this, which basically lets you write a class and it generates (if one day we decide to continue it) a class using Ruby's C API, together with its methods. Here's the macro and here's the implementation.
When using the compiler class all the stages are mixed. You'd use the Program class to parse, infer types and do the codegen if you want to do those separately (the compiler uses it so you can learn how it's done).

@asterite - thanks for the links - that kicks serious butt! Then most everything imaginable is possible :-)

@asterite In case you didn't know, that issue is related to LLVM 3.6.X.
It changed some things :)

I'll close this. We prefer not to use the JIT when doing crystal eval because it might behave different than not using a JIT, so it'll be another point of pain.


Time Validity

Hello @yeojz
Thanks for module.
I am presently using the totp scheme for generating OTPs, Wanted to know what is the validity of the OTP. As in how long the OTP will be valid. There are times when the SMS service is busy and take a few seconds longer. In that case the OTP from totp scheme is invalid and the user needs to generate a new one. So, is there a way I can increase this time out by a few seconds (this is a silly question)?
I am aware that I can use hotp for perpetually valid token but wanted to stick with totp for its security.
Any help is appreciated.
Thanks,
Arvind.

Hey, did you find a way to do it.. I too want to increase the validity

No I couldn't. Ended up using https://www.npmjs.com/package/speakeasy
module.
On Dec 4, 2016 22:56, "mycloudyhopes"<EMAIL_ADDRESS>wrote:
Hey, did you find a way to do it.. I too want to increase the validity
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/yeojz/otplib/issues/4#issuecomment-264717355, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ACcbsIbECcEpQ4ic-2Jjmgb9qVpchbRdks5rEvfYgaJpZM4I9v-f
.

Totally missed this.
@mycloudyhopes
Currently the library does not expose the time interval.
Am a little stretched at work and couldn't give much attention here. I can probably set some time aside over the coming weekend to take a look at it.
Alternatively, you can also go with speakeasy which @arvindr21 suggested, which seems much more recent. Thanks @arvindr21

Thanks @arvindr21 & @yeojz
Your responses really helped me.

@mycloudyhopes @arvindr21 @yeojz As far as I know, this library is very good & also provides time interval for totp (not sure about hotp). I am already using it.
If you look into the code of totp https://github.com/yeojz/otplib/blob/master/src/classes/TOTP.js , you can see step propery, which you can pass during initialization like
var lib = require('otplib');
var totp = lib.totp;
totp.options({ step: 120}); in seconds. Your generated token will be expired after 120 seconds.
@yeojz Please let me know, if I am missing something.

@AmreeshTyagi @mycloudyhopes @arvindr21
Will be doing a clean up of this library + updating all outdated dependencies + refactoring. #5
Shall do a refresh of documentations as well since it seems to be lacking that even I myself missed it.
If there are any more concerns, let me know and I'll take note to include into documentations.
Thanks!


Support Nexus Pro 3

Hi,
io.github.gradle-nexus.publish-plugin fails with http 404, when I try to use it with Nexus Pro 3:
Execution failed for task ':initializeReleaseStagingRepository'.
> Failed to load staging profiles, server at https://<nexusProUrl>/repository/<repoName/ responded with status code 404, body:
  <!DOCTYPE html>
  <html lang="en">
  <head>
    <title>404 - Sonatype Nexus Repository</title>
...

The REST API of Nexus Pro 3 (3.63.0-01) doesn't seem to offer any endpoint for stagingProfiles. Instead, one is supposed to tag artifacts and stage them by moving them to another repository (/v1/staging/move/{destination}?tag=...).
Do I assume correctly, that this Plugin is only compatible with Nexus Pro 2.x?
If this plugin could offer to add a tag to each published artifact, this would help a lot for staging or dropping a set of artifacts.

Nexus 3 is rewritten a piece of software from scratch with completely different API. This plugin supports only Nexus 2 which powers The Central Repository (aka Maven Central). I also haven't heard about the Sonatype plans to switch publishing to TCR to Nexus 3. The main use case it to publish artifacts to TCR.

If this plugin could offer to add a tag to each published artifact, this would help a lot for staging or dropping a set of artifacts.

It looks like the completely new API. I would be probably hard to maintain both APIs (and related workflows) in the same plugin.
Let's keep this request open, but probably it would be easier to write a separate plugin for that (but I haven't analized the problem in depth).

Thanks for the clarification. I wasn't aware, that Nexus 3 is a completely rewritten Software.
I totally agree, that it doesn't make sense to squeeze support for both Nexus versions into this plugin.
What do you think about making the docs clearer on, what Nexus versions are supported?
In case, the Sonatype Enterprise Support shares some best practices with me, how to publish to Nexus Pro 3 from Gradle, I am going to share it here.

What do you think about making the docs clearer on, what Nexus versions are supported?

Good idea.  Would you like to create a PR with the proposed change?

In case, the Sonatype Enterprise Support shares some best practices with me, how to publish to Nexus Pro 3 from Gradle, I am going to share it here.

:+1:


The Dialog seems to have bugs

Describe the bug
When I try to animate the closing of a androidx.compose.ui.window.Dialog using scaleOut, the dialog animates but another ghost of the dialog remains on the app.
I also noticed that the dialog background scrim seems too dark. So maybe multiple instances of it are created?
When I try to take a screenshot of a Dialog in a test, the test fails with the following error:
java.lang.AssertionError: Failed: assertExists.
Reason: Expected exactly '1' node but found '2' nodes that satisfy: (isRoot)
Nodes found:
1) Node #1 at (l=0.0, t=0.0, r=0.0, b=0.0)px
2) Node #2 at (l=0.0, t=0.0, r=1024.0, b=768.0)px
[IsDialog]
	at androidx.compose.ui.test.SemanticsNodeInteraction.fetchOneOrDie(SemanticsNodeInteraction.kt:145)
	at androidx.compose.ui.test.SemanticsNodeInteraction.fetchSemanticsNode(SemanticsNodeInteraction.kt:79)
	at androidx.compose.ui.test.SemanticsNodeInteraction.fetchSemanticsNode$default(SemanticsNodeInteraction.kt:78)
	at androidx.compose.ui.test.OutputKt.printToString(Output.kt:45)
	at androidx.compose.ui.test.OutputKt.printToString$default(Output.kt:41)
       ...

Minimal reproducible code:
import androidx.compose.ui.test.junit4.createComposeRule
import androidx.compose.ui.test.onRoot
import androidx.compose.ui.test.printToString
import androidx.compose.ui.window.Dialog
import org.junit.Rule
import org.junit.Test

class DialogProblemReproducerTest {
    @get:Rule
    val composeTestRule = createComposeRule()

    @Test
    fun `Take screenshot of the dialog`() {
        composeTestRule.setContent {
            Dialog(onDismissRequest = {}, content = {})
        }
        composeTestRule.onRoot().printToString()
    }
}

Affected platforms

Desktop

Versions

Kotlin version*: 1.9.0
Compose Multiplatform version*: 1.5.0
OS version(s)*: Windows 10
OS architecture (x86 or arm64): x86_64
JDK (for desktop issues): Oracle JDK 17.0.7

Thanks for the report. We'll investigate.

This fails on Android in the same way, because both the "regular" root and the dialog itself are considered roots.
So unfortunately we won't be fixing this.
Is the issue with animating the dialog still present? I see you striked-out that part...

Sorry, I cannot try the animation now but the problem was that two separate pieces of code were reacting to a showMyDialog property and two dialogs were displayed. I removed one of the codes and now the scrim is correct too.
So, probably the animation should also have been fixed.

Ok, thanks. I'll close this issue then.

Please check the following ticket on YouTrack for follow-ups to this issue. GitHub issues will be closed in the coming weeks.


[.NET 9]: Refactor and revise OpenAPI docs

We're making some significant changes to the OpenAPI area in .NET 9. Namely:

Adding built-in support for OpenAPI document generation
Updating the web API templates to take advantage of the built-in support

As part of this, we'll want to refactor the docs on OpenAPI to reflect the state of the new universe.
AFAICT, we reference OpenAPI in a few pages in the docs:

OpenAPI support in minimal APIs
Swagger/OpenAPI under controller-based APIs

Includes "Getting started with NSwag" and "Getting started with Swashbuckle" sub-pages

.NET OpenAPI tool reference
Using OpenAPI with gRPC JSON transcoding
Getting started with Swashbuckle and ASP.NET Core sample
Generate a REST API client in a build

As part of this work, I'm recommending a revised header structure for our OpenAPI-related docs.

Controller-based APIs

OpenAPI

Overview
Getting started with Microsoft.AspNetCore.OpenApi
Getting started with Swashbuckle
Getting started with NSwag
Working with OpenAPI at build-time

Minimal APIs

OpenAPI

Overview
Getting started with Microsoft.AspNetCore.OpenApi
Getting started with Swashbuckle
Getting started with NSwag
Working with OpenAPI at build-time

The content under each of these pages should be the same, which conditional headers that allows us to show controller and minimal API specific code when necessary.
We'll also need to go in and revise any code blocks that reference UseSwagger/UseSwaggerUi to reference the new built-in APIs.
We'll probably want to have the first pass of this done to align with the preview4 release coming up in May. I'll create some draft content on the new APIs that we can incorporate here.
cc: @Rick-Anderson @tdykstra @JeremyLikness @mikekistler

@martincostello What are your thoughts on the current Swashbuckle doc page (ref)? I think most of the content on there is accurate but some of the additional resources might be updated.
@RicoSuter How about the NSwag page (ref)? I think the code generation section could use some sprucing. We don't mention the NSwag client generation via VS Connected services at all. And perhaps the screenshots need updating?

@captainsafia Once we ship our 6.6.0 release with net8.0 support (in the next week or so 🤞) I was planning on reviewing the existing content to see what might need tweaking. At a minimum I was going to propose the "doesn't work with .NET 8" notes be removed.

At a minimum I was going to propose the "doesn't work with .NET 8" notes be removed.

That sounds like a good idea to me! What are your thoughts on the amount of content that is on there?
I'm not sure what the right balance between docs on this page and those we would link to externally in the Swashbuckle repo/elsewhere.

I think the level of content there is fine - it seems to cover all the entry-level scenarios with a few troubleshooting hints. It might duplicate stuff that is (or should) be in our README, but I don't think there's anything in there that's out-of-date or incorrect from a quick scroll through.
I did notice there's a link off to https://github.com/domaindrivendev/Swashbuckle.AspNetCore/issues/1920 at the end of the document, which is resolved for 6.6.0, so that can be removed once we ship that too.

@martincostello Great! TYSM for your help here.

@captainsafia and @mikekistler,  I would like to update these two tutorials right away for .NET 9.
Tutorial: Create a minimal API with ASP.NET Core | Microsoft Learn
Tutorial: Create a web API with ASP.NET Core | Microsoft Learn
Question:
For the Visual Studio Code  targeted version of those tutorials for .NET 9, would it be OK to use the Scalar.AspNetCore package for generating UI,  pointing it to the OpenAPI generated .json output?  (This would be enabled for the dev environment only as was also pointed out here:  Work with OpenAPI documents | Microsoft Learn
I could use curl to avoid a UI package, but then the topic gets a fair bit longer as I walk them through how to create those commands.  Using a UI really simplifies these two getting started topic for the VSC versions.
For tutorials targeted at Visual Studio I would continue to use the Endpoints Explorer + .http files.

@wadepickett I think either Scalar or SwaggerUI would be fine to use in the VSCode version of the tutorials. I know @captainsafia prefers Scalar, but .NET developers are very familiar with SwaggerUI and might find that more natural.

@captainsafia prefers Scalar, but .NET developers are very familiar with SwaggerUI and might find that more natural.

For SwaggerUI, they can follow the .NET 8 version , so my vote for .NET 9 is Scalar.

@tdykstra This issue was meant to track refactoring our OpenAPI documentation all up. However, I think the referenced PR was the cherry-on-top for this work and we can close this issue out.
@mikekistler Thoughts?

I think we've addressed the key things in the issue with the restructure that we've just completed. There are a few things we didn't get to, like incorporating the gRPC transcoding stuff in the new OpenAPI section, but I think we can close this out and address those few remaining issues as needed.

Alright! Let's close this and track smaller items individually.


Internet goes out with every download

Describe the bug
I can start downloading assets without issue, but within ~30 seconds my internet stops working with every download. It will eventually start working again within a few minutes if I close the asset manager.
To Reproduce
Steps to reproduce the behavior:

Login to Epic Asset Manager
Find any content in the marketplace that you have in your account
Choose either "Download" or "Add to Project"
Wait ~30 seconds, at which point the internet stops working for both Epic Asset Manager (download hangs) and other applications on my machine

Expected behavior
Internet should not stop working
Desktop (please complete the following information):

OS: Arch 6.9.3 Kernel
Version  Flatpak 3.8.6

Debug logs
run --user --env=RUST_LOG=debug io.github.achetagames.epic_asset_manager
<epic_asset_manager> - [DEBUG] - /app/share/epic_asset_manager
<epic_asset_manager> - [DEBUG] -
<epic_asset_manager> - [DEBUG] - 3.8.6
<epic_asset_manager::application> - [INFO] - Epic Asset Manager (io.github.achetagames.epic_asset_manager)
<epic_asset_manager::application> - [INFO] - Version: 3.8.6 ()
<epic_asset_manager::application> - [INFO] - Datadir: /app/share/epic_asset_manager
<epic_asset_manager::application::imp> - [DEBUG] - GtkApplication<EpicAssetManager>::startup
<epic_asset_manager::tools::epic_web> - [ERROR] - Error parsing json: reqwest::Error { kind: Decode, source: Error("expected value", line: 1, column: 1) }
<epic_asset_manager::tools::epic_web> - [ERROR] - Failed to query EULA status: EulaQuery/hasAccountAccepted: Request failed with status code 401 with response: {"errorCode":"errors.com.epicgames.common.authenticat
ion.authentication_failed","errorMessage":"Authentication failed for /api/public/agreements/unreal_engine/identities/b12afdb85fef4abc9d8a92c15b769477","messageVars":["/api/public/agreements/unreal_engine/identitie
s/b12afdb85fef4abc9d8a92c15b769477"],"numericErrorCode":1032,"originatingService":"com.epicgames.eulatracking.public","intent":"prod","errorStatus":401}
<egs_api::api::types::download_manifest> - [ERROR] - No header magic
<egs_api::api::types::download_manifest> - [ERROR] - No header magic
<egs_api::api::types::download_manifest> - [ERROR] - No header magic
<egs_api::api::types::download_manifest> - [ERROR] - No header magic
<egs_api::api::types::download_manifest> - [ERROR] - No header magic
<egs_api::api::types::download_manifest> - [ERROR] - No header magic
<egs_api::api::types::download_manifest> - [ERROR] - No header magic
<egs_api::api::types::download_manifest> - [ERROR] - No header magic
<epic_asset_manager::ui::widgets::download_manager::asset> - [ERROR] - Failed to start chunk download, trying again later: error sending request for url (https://download.epicgames.com/Builds/Rocket/Automated/Anci
entG02ff5dddb8ffV1/CloudDir/ChunksV3/95/3856DC2135A0C0E5_5AC5D8314EA536D53A8507AC14F92F63.chunk): error trying to connect: dns error: failed to lookup address information: Temporary failure in name resolution
<epic_asset_manager::ui::widgets::download_manager::asset> - [ERROR] - Failed to start chunk download, trying again later: error sending request for url (https://fastly-download.epicgames.com/Builds/Rocket/Automat
ed/AncientG02ff5dddb8ffV1/CloudDir/ChunksV3/72/E48727E27E7E96A6_D52808DA4F5955B3F96A83B19F31547F.chunk): error trying to connect: dns error: failed to lookup address information: Temporary failure in name resoluti
on
....

This message repeats until I Ctrl + C

Some anecdotal evidence I'll continue testing, but I disabled my wireless adapter (I don't use wifi on this machine) and downloads seem to be working much more reliably now.
I realized that systemctl status NetworkManager showed the following:
Jun 16 23:45:46 archlinux NetworkManager[612]: <info>  [1718606746.9674] device (wlan0): set-hw-addr: set MAC address to 8E:B4:B1:54:05:49 (scanning)
Jun 16 23:45:47 archlinux NetworkManager[612]: <info>  [1718606747.0040] device (wlan0): supplicant interface state: inactive -> disconnected
Jun 16 23:45:47 archlinux NetworkManager[612]: <info>  [1718606747.0040] device (p2p-dev-wlan0): supplicant management interface state: inactive -> disconnected
Jun 16 23:45:47 archlinux NetworkManager[612]: <info>  [1718606747.0078] device (wlan0): supplicant interface state: disconnected -> inactive
Jun 16 23:45:47 archlinux NetworkManager[612]: <info>  [1718606747.0078] device (p2p-dev-wlan0): supplicant management interface state: disconnected -> inactive
Jun 16 23:52:39 archlinux NetworkManager[612]: <info>  [1718607159.9510] device (wlan0): set-hw-addr: set MAC address to D6:29:71:62:A6:E8 (scanning)
Jun 16 23:55:36 archlinux NetworkManager[612]: <info>  [1718607336.9659] manager: NetworkManager state is now CONNECTED_SITE

So I disable wlan0 and I haven't had issues since.

That is a strange thing to have happened, I did use this with dual nic machine before and had no issues. I need to expose the amount of download threads as a config option, that might help mitigate or at least investigate the issue.

I was wrong, this is still happening, but is clearly an issue with my machine so I'll close this.


Update mobile-deep-links.md

providing an example of a model-driven app deeplink as customers tend to keep https:// in the orgurl

Closing public pr and reopen in private repro


fix: Update output.tf to output Neos tutorial url

This pull-request fixes https://github.com/GoogleCloudPlatform/terraform-ecommerce-microservices-on-gke/issues/10.
We may need to update the value later (in case the URL changes).

The CI failure is due to a known issue (#3):
module.ecommerce_microservices_on_gke.google_gke_hub_feature.multi_cluster_ingress_feature: Destroying...
...
Error: Error deleting Feature: googleapi: Error 400: Feature has associated resources that should be cleaned up before deletion. Check the Feature state details for more information. This check can be overridden by setting force=true.

Merging.


Problem with La.parquet size

Demo_Data.yaml references a lot of files that aren't published anywhere, and so we decided to edit it a little and create this version:
IMAGE_TEXT: # Group name should be in [IMAGE_TEXT, TEXT_ONLY, IMAGE_TEXT_IN_CONTEXT]
  LADD: # LLaVA Detailed Description, dataset name can be assigned at any name you want
      mimicit_path: otter_data/json/LA/LADD_instructions.json # Path of the instruction json file
      images_path: otter_data/Parquets/LA.parquet # Path of the image parquet file
      num_samples: -1 # Number of samples you want to use, -1 means use all samples, if not set, default is -1.
  LACR_T2T:
    mimicit_path: otter_data/json/LA/LACR_T2T_instructions.json
    images_path: otter_data/Parquets/LA.parquet
    num_samples: -1

and running the finetuning script on it leads to  this error:

pyarrow.lib.ArrowCapacityError: array cannot contain more than<PHONE_NUMBER> bytes, have<PHONE_NUMBER>

Please refer the updated code here:
https://github.com/Luodian/Otter/blob/8b386816ec67b15833cde3dcd1d7ca6a752d2451/pipeline/mimicit_utils/mimicit_dataset.py#L222C27-L229

Nice, that helped, thanks!

You should add something like import pyarrow.parquet as pq at some point, otherwise the interpreter gets confused


Use custom fetch(er) for the web client

Use case
By default the web client uses the global fetch. The idea is to configure the client with a custom fetch(er) that may decorate the fetch function with additional functionality. A use case for this could be -
Using mTLS: Cloudflare Workers can use mTLS with added bindings, the binding exposes a fetch(er) with mTLS pre-configured
Describe the solution you'd like
  const client = createClient({
    // A custom fetcher with decorated functionality, in this case a Cloudflare Worker binding with a fetcher preconfigured with mTLS
    fetcher: cloudflareEnv.MY_CERT.fetch
  })

Additional context
https://developers.cloudflare.com/workers/runtime-apis/bindings/mtls

@arfath-linklet, would you like to contribute to the project and create a PR for it? The implementation is rather trivial:
Add fetch? to
export type WebClickHouseClientConfigOptions =
  BaseClickHouseClientConfigOptions & {
    fetch?: typeof fetch
  }

You'll only need to double-check if the signature is compatible with that cloudflareEnv.MY_CERT.fetch like that right away. Then it just needs to be passed to the WebConnection and used instead of the default one there, if it is defined (see WebImpl in the same config.ts file).
If you have more questions, you can always DM me in the community Slack or comment here.


Update description to render as html

It looks like the description is being rendered as text, but may come from the source with HTML formatting
https://github.com/trevinflick/ArtPOD/blob/1b3d8699a4d284a26b6079aa940805025f7a67e2/app.py#L51-L54
https://github.com/trevinflick/ArtPOD/blob/1b3d8699a4d284a26b6079aa940805025f7a67e2/app.py#L112-L116
EX:

You could update this so that the output is rendered as HTML as described at https://shiny.posit.co/py/docs/ui-html.html
from shiny import ui
ui.HTML("
Hello world!
")

similar to how you are doing the artist name
https://github.com/trevinflick/ArtPOD/blob/1b3d8699a4d284a26b6079aa940805025f7a67e2/app.py#L122

Good catch! Thanks for the assistance. I noticed the tesserae but wasn't sure of a fix right away. I think 6bcd05dbfed6be4a8a6da00d54f1ecb22440deea should fix this so the description renders HTML in the future.


The file for ModelValidatorProviderContext is misnamed

It's missing a 'd':
https://github.com/aspnet/Mvc/blob/dev/src/Microsoft.AspNetCore.Mvc.Abstractions/ModelBinding/Validation/ModelValiatorProviderContext.cs

Could you send a PR for this?

Sure thing


Why "id" attribute is being generated on heading elements?

I see that this line of markdown:
#This is a test of GFM for javascript
once converted to HTML becomes this:

This is a test of GFM for javascript

Which option is responsible for auto-generating id in heading elements and how can this be turned off?

I thought this was  standard markdown. https://github.com/gitlabhq/gitlabhq/blob/master/doc/markdown/markdown.md#headers

I did a little more digging. I was always under the assumption this was standard markdown behavior. Grunt markdown uses https://github.com/chjj/marked It might be part of the GFM option. You can pass options to marked using the markdownOptions of in your grunt file

I don't see any option for disabling auto generation of id attributes on headings, do you?

No I don't you might not be able to disabled it.
https://github.com/chjj/marked/blob/master/lib/marked.js#L788
Thats where they are adding the ID. I don't seen any obvious way of disabling that behavior.

I don't think this is part of standard markdown. John Gruber's markdown samples and his Dingus app do not generate such id on headings when converting markdown to html, nor does Github.

Yea not sure where this came from. Marked is doing it and since i am using marked there is little I can do about it.
Looks like there is some discussion on this thread https://github.com/chjj/marked/issues/515

Others have also complained about this:
https://github.com/chjj/marked/issues/420
Marked's development seems to be stalled with 180 open issues.

Have you looked into this? https://github.com/evilstreak/markdown-js

Does it support GFM? GFM is really important for a lot of the common uses of grunt markdown.

Not yet, but he is working on it. You can voice your support for it here:
https://github.com/evilstreak/markdown-js/issues/41

If you really want to remove the ids you could do it in postCompile. I think thats all I can do for right now since marked doesn't expose a setting.

I think I can live with it. Thanks for your help.

Wouldn't you be able to override marked's Renderer and override it's heading handler?

@dnutels yes I could but right now I don't extend marked at all. If I start to extend marked the plugin will become coupled to marked. I like to keep the coupling loose so if I find a better markdown parser I should be able to swap it easily.

at some point one has to give up :) and understand that marked is the de-facto standard
you can abstract that setting - since any "better" markdown plugin should have the ability to alter the way headings are generated (otherwise you'd be trading down, IMO):

{
    ...
    markdown: {
        heading: function() {...}
    }
}

it looks like id is being added to headings only. Use a renderer to override this as follow:
var marked = require('marked');
var renderer = new marked.Renderer();
renderer.heading = function (text, level) {
  return `<h${level}>${text}</h${level}>`;
};


RaidEmulator - Elapsed Time display incorrect when rolling over minutes

Description
When RaidEmulator's elapsed time rolls over a minute, e.g. from 1:59/11:00 to 2:00/11:00, it instead displays 1:00/11:00 for the first second of the second minute, etc.
Additional information
It's the most minor things that I notice when I'm sitting there waiting for custom triggers/timeline entries to test...

Thanks for the bug report.  I like the raidemulator's real time playback, but as a side thought I also wonder if it'd be good to have something that could read a log and instantly just print out all of the triggers that fired and what they printed.

I've thought a few times about some sort of analysis of pulls, like a list of triggers and you can select a trigger to get info about what caused it and what it displayed to who. Also thought about taking a stab at raidemulator pulling the current set of encounters from ACT instead of having to re-import each time (by looking at start time/end time of encounter history and doing a re-import of those lines specifically from the file). Lots of thoughts, but not really much from them yet.

I think the "reimport" issue could be fixed with #853.  If you could store more fights (or even one full TEA log), then it's much easier to reload and iterate without the reimport step.
Not to say that loading from ACT wouldn't also be an option, but just saying I think there's already half a solution there, it just needs to be finished.

This issue is now irrelevant with the redesigned raidemulator.