Add LTP build for sim and allow long arguments for archive

Summary
This adds a build configuration for sim that includes the LTP posix tests. Because this test suite is very large there are a couple build system related changes that were also needed:

Include the relative path in the archive. Without this we overwrite the files in the archive if they have two object names that are the same.  (P argument on ar)
Add a MAX_ARGS Makefile macro that helps split up calls that include a large number of augments.  Here this is used with ARCHIVE_ADD so batches of 100 objects are added at a time, keeping us under the limit for max command line arguments in a normal situation.

Impact
The simulator can now run the LTP tests.  Note that many of the fail including causing segfaults this will need to be iterated on.
Testing
Local testing as well as CI
NOTE:  We will need to adjust the app repo so that the expected compiler warnings are allowed for this application.  This needs to be done prior to merging this.
NuttShell (NSH) NuttX-10.0.1
nsh> ltp_interfaces_timer_settime_9_2
Test for value<PHONE_NUMBER> sec 735625645 nsec
Timer expired<PHONE_NUMBER> sec 735725645 nsec
Test for value<PHONE_NUMBER> sec 735725645 nsec
Timer expired<PHONE_NUMBER> sec 735825645 nsec
Test for value<PHONE_NUMBER> sec 765825645 nsec
Timer expired<PHONE_NUMBER> sec 765925645 nsec
Test for value<PHONE_NUMBER> sec 765925645 nsec
Timer expired<PHONE_NUMBER> sec 766025645 nsec
Test for value<PHONE_NUMBER> sec 766030645 nsec
Timer expired<PHONE_NUMBER> sec 766225645 nsec
Test for value<PHONE_NUMBER> sec 766225650 nsec
Timer expired<PHONE_NUMBER> sec 766425645 nsec
Test PASSED
nsh> ltp_interfaces_pthread_once_1_1
Test PASSED
nsh> ltp_interfaces_pthread_sigmask_10_1
FAIL: SIGKILL was added to the signal mask
Test FAILED
nsh> ltp_stress_signals_sigismember_stress_1
nsh> ltp_mq_unlink_speculative_7_2
Test PASSED
nsh>

Partial list:
Builtin Apps:
  hello
  ltp_behavior_timers_1_1
  ltp_behavior_timers_2_1
  ltp_clock_settime_speculative_4_3
  ltp_clock_settime_speculative_4_4
  ltp_definitions_errno_h_3_2
  ltp_definitions_errno_h_4_1
  ltp_definitions_mqueue_h_1_1
  ltp_definitions_sched_h_10_1
  ltp_definitions_signal_h_26_1
  ltp_functional_mqueues_send_rev_2
  ltp_functional_semaphores_sem_conpro
  ltp_functional_semaphores_sem_philosopher
  ltp_functional_semaphores_sem_readerwriter
  ltp_functional_semaphores_sem_sleepingbarber
  ltp_interfaces_aio_suspend_5_1
  ltp_interfaces_asctime_1_1
  ltp_interfaces_clock_1_1
  ltp_interfaces_clock_2_1
  ltp_interfaces_clock_getcpuclockid_1_1
  ltp_interfaces_clock_getcpuclockid_1_2
  ltp_interfaces_clock_getcpuclockid_5_1
  ltp_interfaces_clock_getcpuclockid_6_1
  ltp_interfaces_clock_getres_1_1
  ltp_interfaces_clock_getres_3_1
  ltp_interfaces_clock_getres_5_1
  ltp_interfaces_clock_getres_6_1
  ltp_interfaces_clock_getres_6_2
  ltp_interfaces_clock_gettime_1_1
  ltp_interfaces_clock_gettime_1_2
  ltp_interfaces_clock_gettime_2_1
  ltp_interfaces_clock_gettime_3_1
  ltp_interfaces_clock_gettime_7_1
  ltp_interfaces_clock_gettime_8_1
  ltp_interfaces_clock_gettime_8_2
  ltp_interfaces_clock_nanosleep_11_1
  ltp_interfaces_clock_nanosleep_1_1
  ltp_interfaces_clock_nanosleep_13_1
  ltp_interfaces_clock_nanosleep_2_1
  ltp_interfaces_clock_nanosleep_3_1
  ltp_interfaces_clock_settime_1_1
  ltp_interfaces_clock_settime_17_1
  ltp_interfaces_clock_settime_17_2
  ltp_interfaces_clock_settime_19_1
  ltp_interfaces_clock_settime_20_1
  ltp_interfaces_clock_settime_4_1
  ltp_interfaces_clock_settime_4_2
  ltp_interfaces_clock_settime_5_1
  ltp_interfaces_clock_settime_5_2
  ltp_interfaces_clock_settime_6_1
  ltp_interfaces_ctime_1_1
  ltp_interfaces_difftime_1_1
  ltp_interfaces_fsync_4_1
  ltp_interfaces_fsync_5_1
  ltp_interfaces_fsync_7_1
  ltp_interfaces_gmtime_1_1
  ltp_interfaces_gmtime_2_1
  ltp_interfaces_kill_2_1
  ltp_interfaces_kill_2_2
  ltp_interfaces_kill_3_1
  ltp_interfaces_killpg_1_1
  ltp_interfaces_killpg_2_1
  ltp_interfaces_killpg_4_1
  ltp_interfaces_killpg_5_1
  ltp_interfaces_killpg_6_1
  ltp_interfaces_killpg_8_1
  ltp_interfaces_localtime_1_1
  ltp_interfaces_mktime_1_1
  ltp_interfaces_mlock_10_1
  ltp_interfaces_mlock_5_1
  ltp_interfaces_mlock_8_1
  ltp_interfaces_mlockall_13_1
  ltp_interfaces_mlockall_13_2
  ltp_interfaces_mlockall_3_6
  ltp_interfaces_mlockall_3_7
  ltp_interfaces_mlockall_8_1
  ltp_interfaces_mmap_10_1

Looks like llvm-ar does not support the P modifier so we will need to use a different solution here. Probably the right thing is to generate the suffix based on the source file directory instead of the makefile directory. That way we will have unique object file names.

@btashton could you explain a little bit about the problem resolved P option?

@btashton could you explain a little bit about the problem resolved P option?

Sure, the main issue is that our object files look something like myfile.a.top.dir.o but we have that same object name if the object is in dir/1/myfile.c and dir/2/myfile.c this means we overwrite the object files adding them to the archive.  The P option was a ugly solution because it means that the archive has dir/2/myfile.a.top.dir.o and dir/1/myfile.a.top.dir.o instead of just myfile.a.top.dir.o.
I think the correct solution is that the dirname() of the object file needs to be included in the object file name not just the CWD.
This is also technically an issue in the Make.dep file that gets generated as well.  I have a solution for that, but the Makefile side is a little more complicated to fix.

@btashton could you explain a little bit about the problem resolved P option?

Sure, the main issue is that our object files look something like myfile.a.top.dir.o but we have that same object name if the object is in dir/1/myfile.c and dir/2/myfile.c

We need change CSRCS=myfile.c to CSRCS=dir/1/myfile.c. Application.mk already contain the special process to ensure the unique object file name in this case. Here is the related patch:
https://github.com/apache/incubator-nuttx-apps/commit/e48a74f358f119087100cb124707e96e74f81eef

this means we overwrite the object files adding them to the archive. The P option was a ugly solution because it means that the archive has dir/2/myfile.a.top.dir.o and dir/1/myfile.a.top.dir.o instead of just myfile.a.top.dir.o.
I think the correct solution is that the dirname() of the object file needs to be included in the object file name not just the CWD.
This is also technically an issue in the Make.dep file that gets generated as well. I have a solution for that, but the Makefile side is a little more complicated to fix.

Since all source files under apps archive into one library(libapps.a), it is very bad practice to just list the file name and supply the path by VPATH/DEPPATH.

We need change CSRCS=myfile.c to CSRCS=dir/1/myfile.c. Application.mk already contain the special process to ensure the unique object file name in this case. Here is the related patch:
apache/incubator-nuttx-apps@e48a74f

I know this. but it is not enough because $(SUFFIX) ends up just being home.bashton.nuttx.apps.testing.ltp.o  So in this case
in apps/testing/ltp/ we have  ltp/testcases/open_posix_testsuite/conformance/interfaces/clock/1-1.c and ltp/testcases/open_posix_testsuite/conformance/interfaces/ctime/1-1.c  This object will be generated 1-1.home.bashton.nuttx.apps.testing.ltp.o for both on these files.
So we get
ar rcs libapp.a ltp/testcases/open_posix_testsuite/conformance/interfaces/clock/1-1.home.bashton.nuttx.apps.testing.ltp.o ltp/testcases/open_posix_testsuite/conformance/interfaces/ctime/1-1.home.bashton.nuttx.apps.testing.ltp.o
This will result in the second object overwriting the first one in the archive.
I am suggesting that SUFFIX is not good enough using CWD of the Makefile.  It instead should be something like $(CWD).$(subst $(DELIM), ., $(basename $(CSRC))).o

let's close it since ltp is built with ci now.


First working version

Just like https://github.com/testiumjs/testium-driver-wd/pull/1 but for sync. Doing both in parallel to get a feeling for what works with both.

README.md?

@khoomeister I think it's too early for that. This is just the plumbing and the actual interface is in flux. I'll copy the docs as I copy the actual implementations. E.g. all browser.* methods are supposed to be in separate files like in testium. I just hacked in the implementations needed for a first POC test.

Now with the complete integration test suite from testium.

Now with eslint (based on airbnb styles) & fixed lint errors.

Alright! Moved testium-example-app into its own repo so we can reference it from different packages and added a README. Ready for review.

No blockers :shipit:

Added the outstanding things to https://github.com/groupon/testium/pull/171, releasing a 1.0 since the remaining points are all for internals.


avro-python3 1.9.2 is broken

Description

https://jira.apache.org/jira/browse/AVRO-2737
Motivation and Context

Have you tested this? If so, how?

They are working on a fix, so maybe this would not be needed.

fixed in <IP_ADDRESS>


FR: find user by phone number

With the release of phone number authentication many apps will not have email address as a unique key.  It would be very useful to have a 'findUserByPhoneNumber()` call in the admin SDK to match the email one.

Related to this: https://github.com/firebase/firebase-admin-node/issues/34

I was looking into creating a PR for this but it seems like the getAccountInfo endpoint of the auth API doesn't yet allow to search users by phoneNumber, only email and localId.
Feel free to comment here if/when that's added and I'll push the PR :)
P.S.: the documentation for the REST API should probably include the available request properties, in the same way the old documentation did.

Thank you for your help but I plan to implement this when the Auth backend supports it. It is not yet implemented. It is also part of a bigger change which features new API.

Related to this issue and #34, it would also be useful if we could change a phone number via tha admin sdk. I guess this will also be part of the new API you were referring to? Is there anything we can do to help?

Hang tight, it's all coming soon.

This was released today: https://firebase.google.com/support/release-notes/admin/node#5.1.0


NLog ImpersonatingWrapper Example

Hi, I am attempting to implement error logging using NLog on wfp application, but need write files to a central server/shared drive. I have seen the ImpersonatingWrapper as a potential way I could go about it, but I couldnt find detailed documentation on how to do this. Is there an example out there i could use or anyone who has implement this target type, and how? Please assist. Thanks

You can check the unit tests
https://github.com/NLog/NLog/blob/master/tests/NLog.UnitTests/Targets/Wrappers/ImpersonatingTargetWrapperTests.cs

Has you question being answered?

Not really, I stopped working on this for a bit as i was working on other more time sensitive tasks. But I will be getting back to this shortly. So, what i was looking for was a good example with a detailed configuration that is ready to work, or step by step of how to integrate this target type. I have used the file target type before, which is pretty straightforward and has numerous examples online, but the ImpersonatingWrapper dont seem to have as much. Thank you.

It's true that the docs aren't really detailed on this topic. If you get things working, it would be great to extends the docs (wiki)
If you need some more help on this, let us know. I'm closing this issue for now, as we cannot do anything with this.


feat(starter-web): disable warning logs on failing to create tomcat work directory

Unfortunately, the warn log is still there, see the test deployment.

Unfortunately, the warn log is still there, see the test deployment.

Should I try adding the property to cookie-cutter template and see if it works when we create new services?


Setting db to django mysql backend.

Sets database to use django mysql backend explicitly.  Removes try...catch block.
Additions

Removals

Changes

Uses django mysql database backend

Testing

Review

@rosskarchner @chosak @willbarton @vccabral @m3brown

Screenshots
Notes

Todos

Checklist

[ ] Changes are limited to a single goal (no scope creep)
[ ] Code can be automatically merged (no conflicts)
[ ] Code follows the standards laid out in the front end playbook
[ ] Passes all existing automated tests
[ ] New functions include new tests
[ ] New functions are documented (with a description, list of inputs, and expected output)
[ ] Placeholder code is flagged
[ ] Visually tested in supported browsers and devices
[ ] Project documentation has been updated (including the "Unreleased" section of the CHANGELOG)

Can this go directly into the DATABASE dict?

LGTM except maybe it doesn't need to be a variable at all now.

@vccabral   The variable, MYSQL_ENGINE, is used in the following places, so it could go directly in the dictionary, but it would create more code than reduce:
https://github.com/cfpb/cfgov-refresh/blob/master/cfgov/cfgov/settings/local.py#L14
https://github.com/cfpb/cfgov-refresh/blob/master/cfgov/cfgov/settings/production.py#L40
https://github.com/cfpb/cfgov-refresh/blob/master/cfgov/cfgov/settings/base.py#L159

@vccabral   I suppose I could refactor those lines of code also, to reduce the redundancy.  What do you think?

@vickumar1981 thanks. Your comment address my concern.

@vickumar1981 what does this change improve?
I suspect it will break our current deploy, for any environment that uses the fix_mysql flag
We could eliminate fix_mysql if we build the mysql-python wheel on the destination server, instead of in Jenkins.

@rosskarchner  Sorry, should have put the issue on this PR:
https://jira.demo.cfpb.gov/browse/DEV-2042

@rosskarchner If I run 'collectstatic' in sqllite mode, instead of using mysql, would that still build using drama-free and the fix_mysql flag?
B/c that's what this PR attempts to do here: https://github.com/cfpb/cfgov-refresh/pull/2465
I would like to not have a fix_mysql flag if it is possible.

@rosskarchner   k, so this branch fails the build.
https://zusa.demo.cfpb.gov/view/Cf.gov/job/cf.gov-frontend-build/245/console
http://zusa.demo.cfpb.gov/view/Cf.gov/job/cf.gov-deploy-django/194/console
If you look at the deploy django job it states:
out: django.core.exceptions.ImproperlyConfigured: Error loading MySQLdb module: libmysqlclient_r.so.16: cannot open shared object file: No such file or directory

I have a few simple observations here before I close this PR:

The cf.gov-deploy-django job while reporting the mysql error, also incorrectly reports that the job has succeeded.  Unless you actually look at the stack trace, you would have no idea there was even an error, meaning it could be easily overlooked.

If you look at this PR, there's a green checkbox that says All checks have passed.  I strongly feel it would be a step forward, if we could ensure that this green checkbox also means that we have a high degree of confidence that a build will pass and deploy correctly.  Currently, that is not the case.

The source of the problem seems to be [GHE]/CFGOV/website/blob/master/fabfile.py#L62. I'm not entirely sure I follow what's supposed to happen there.

If you look at this PR, there's a green checkbox that says All checks have passed. I strongly feel it would be a step forward, if we could ensure that this green checkbox also means that we have a high degree of confidence that a build will pass and deploy correctly. Currently, that is not the case.

Do you have a suggestion of how we could catch a problem with the fabfile/deployment scripts in travis?

@willbarton  I'm open to ideas here.  But, I think we could perhaps open an issue about it.

please merge https://github.com/cfpb/cfgov-refresh/pull/2465 instead.


Sentinel lab refresh - updated module6 - watchlist

Required items, please complete
Change(s):

updated module6 - watchlist: in exercise 2 Added a first step with the need to install the analytic rule High count of connections by IP from content hub

Reason for Change(s):

Sentinel lab Refresh

Version Updated:
Testing Completed:

yes

Checked that the validations are passing and have addressed any issues that are present:

Looks good to me. Thanks @miguel-pgomes !
@v-atulyadav please merge


Orange Cameroon - new number range

Orange Cameroon is using mobile number prefix +237-68

Thanks for the report!
Can you please reference official government or telco docs as evidence for the change?

This will be included in the next release.

Thanks! This is in libphonenumber 7.0.7.


Fix .gitignore,

fix committed merge conflict #150
support carthage submodules
update gitignore to https://github.com/github/gitignore/blob/master/Swift.gitignore

@matthewpalmer can you review this please?


VASCAN ClassificationIds suddenly missing

The ClassificationIds key is not returned anymore for DataSource VASCAN (147) but is still returned by other sources like Catalogue of Life (1) and ITIS (3).
Query example:
https://verifier.globalnames.org/api/v1/verifications/salix?capitalize=true&all_matches=true&data_sources=1|3|147

thanks for letting me know @BenMerSci, I'll double check what could be a problem, hopefully will be fixed soon

VASCAN has both flat and tree hierarchies, and flat was default. Now, if tree hierarchy exists, it has a priority.

Hm, I just noticed that VASCAN is set as not curated, which actually means 'curation is unknown' I think it shoud be changed to 'curated'. It will be fixed next time database is updated

Good to know!
Again, thank you so much @dimus !

you very welcome @BenMerSci, please do not hesitate to let me know if something else will appear, or if  there is a feature that is important but missing


Create Events Page for the application

I would like to work on this.

@himanshusharma89 someone is already  being assigned this


focused node always white (foreground) in debug window

It seems that, while in debug mode, vscode set foreground for the focused item in list to #ffffff, and I didn't find any way to change that behavior.
This behavior leads some "bug" for material-like design, in which the focused item doesn't have background.
See https://github.com/chuling/vscode-theme-github-light/issues/4.
And the popular material theme

It's wired that list.activeSelectionForeground doesn't have effect in debug mode, for any focused item in debug window, their foreground was set to #ffffff.

update
I check vscode's src, it seems that listFocusAndSelectionBackground and listFocusAndSelectionForeground are controlling this behavior.
https://github.com/microsoft/vscode/blob/ab0eb6de2d5fb4563385d98f302f78caa1767efd/src/vs/base/browser/ui/list/listWidget.ts#L872-L873
but there is no registered entry in src/vs/platform/theme/common/colorRegistry.ts
https://github.com/microsoft/vscode/blob/ab0eb6de2d5fb4563385d98f302f78caa1767efd/src/vs/platform/theme/common/colorRegistry.ts#L230-L247

Please file this against the theme extension you are using. Since the colors they are using seem to cause this.

I think this was not caused by the extension, because ANY THEME EXTENSION, INCLUDING THE BUILDIN THEME, their focused item in list was colored with pure white foreground.
Extension I am using: https://github.com/chuling/vscode-theme-github-light/
https://github.com/material-theme/vsc-material-theme and many others...
Just download any theme you name, and begin debugging, you will see this problem...

I can not reproduce this with the Github-light vscode theme that I use. Neither with our builtin theme.
Please try reproducing with vscode insiders  https://code.visualstudio.com/insiders/
I thikn you have some extension which breaks this.

@isidorn Sorry, maybe I am not express clearly :sob: :sob: :sob:, you have to focus your mouse on the list to see this problem (while debugging) ... I will post some screenshot later :joy: :joy: :joy:, and thanks for replying so quickly.

material theme lighter

Github-light vscode theme you use

Just confused that why these node respect list.activeSelectionBackground but not respect list.activeSelectionForeground...:confused::confused::confused:
So the only solution is to add background for these active selected node...
Anyway, thanks for replying.

So it does look like there is a bug, we explicitly set the foreground color in:
https://github.com/microsoft/vscode/blob/2d87c95e53471539a1fab25a5367bb3eeffc9456/src/vs/workbench/contrib/debug/browser/media/debug.contribution.css#L175-L179
I'm going to change this to make it inherit, so that it will properly themes when you set list.activeSelectionForeground:

@isidorn we should also take a look at the hovers of these list items as those also run into a similar problem where there are no color tokens:

Should we apply the same treatment?

@misolori yes we could introduce token colors for this, I am not against this, we just never did it. If you would like to add them let me know and I can provide pointers.


Typo fix in sonarqube/README.md

Some sentences miss the "." at the end.
This PR is a recreation of PR #4958, because of conflicts in yaml that caused by other PR.
Lucky to meet the conflicts two times. :(

/assign
/ok-to-test

/lgtm
Third time a charm


[Bug]: VSCode shows irrelevant error in ballerina code with annotations on tuple-type members

Description
VSCode plugin shows an error even though it works well with the bal run command.
Note: This is related to tuple-member annotations

Steps to Reproduce

Locally build this branch https://github.com/ballerina-platform/ballerina-lang/tree/annot-tuple-members
Change VSCode workspace/user settings pointing to this local build and enable plugin-dev mode
Copy and paste the following code snippet to a new ballerina file

type T1 [int, @member int, string];

function testTupleAnnot() {
    T1 thisOne = [1, 2, "A"];
}

const annotation member on field;

Look for the error messages

Affected Version(s)
No response
OS, DB, other environment details and versions
No response
Related area
-> Editor
Related issue(s) (optional)
No response
Suggested label(s) (optional)
No response
Suggested assignee(s) (optional)
No response

Reinstalling Ballerina Extension fixed the issue.


The slider movement to be based on keyboard keys

Currently based on the mouse pointer the slider moves, it should incorporate movement via keyboard keys as well

I fixed issue and made pr. plz check.

Thanks, merged 🤝


Output ends with ESCAPED_SOURCE_END_CLEAN_CSS

I have a vendor-provided CSS file which I am using grunt-contrib-cssmin (^1.0.1) to strip out comments and minify.
The file is processed, and the output ends with __ESCAPED_SOURCE_END_CLEAN_CSS__.
A sample of the vendor file I am processing is attached (had to change file ext. to .txt so it would upload, but I am dealing with the same file with ext. .css).
Is there anything in this sample file that would cause the processing to add this output?
Cheers :D
player.txt

I can see (with a hex editor) that this file ends with 0x1a character which seems to be a Windows Ctrl-Z control code. Quick fix is just to remove it but it'd be good to find out why it results in such behavior.
http://stackoverflow.com/questions/17024436/what-is-the-unicode-u001a-character-aka-0x1a

True, it does, and that char is also added during processing. It's less of a concern than the __ESCAPED_SOURCE_END_CLEAN_CSS__ that is also inserted however.

Thanks to the new tokenizer this is no longer an issue. Added a test case and marking it as fixed on master. Thanks @davewallace!

@davewallace I just reproduced the error and it seems to only happen on css files with errors.  For example, if you add body { background-color: orange; } in one file, then misspell background in another file, the second file would have ESCAPED_SOURCE_END_CLEAN_CSS added to the end.  Is this the same behavior happening on your system?

For my part, I was dealing with vendor contributed files which we did not want to touch anyway, so whether it was a parsing error which resulted in the __ESCAPED_SOURCE_END_CLEAN_CSS__  or not, the point is moot as we wouldn't change a given error to avoid the problem apparent in this issue.
Thinking back though, I seem to recall removing some comments via minification which removed the error... this is a while ago for me now, done a lot of things since, so I may be incorrect there.


Get-PnPFile not releasing local file resource when loading file on Mac

PSVersion 7.2.1
PSEdition Core
OS: macOS Catalina
Pnp.PowerShell 1.9.0
Issue Summary
Using Get-PnPFile to download a file to the local disk causes file access issues when later used in Add-PnPFileToSiteTemplate throwing error "The process cannot access the file 'filename' because it us being used by another process".
At a guess on MacOS the stream that was used to create the file is not getting closed / recycled.
Working fine on windows, not working on Mac
Expected behavior
Should download file and allow adding to site template
Actual behavior
Throws file access error on Mac OS
Steps to reproduce behavior
# get temporary files path
$TemplatePath = [System.IO.Path]::GetTempPath()

# connect to sharepoint site
Connect-PnPOnline -Url https://yourdomain/sites/yourexistingsite -Interactive -LaunchBrowser

# download site template
Get-PnPSiteTemplate -Out $TemplatePath + '/mysitetemplate.xml' -IncludeAllClientSidePages -PersistBrandingFiles -PersistPublishingFiles -IncludeNativePublishingFiles -Force

# download a file that exists on you site to the local disk
Get-PnPFile -ServerRelativeUrl '/sites/yourexistingsite/your document library/yourfile.txt' -Path $TemplatePath -FileName yourfile.txt -AsFile

# try and add file to site template
Add-PnPFileToSiteTemplate -Path $TemplatePath + '/mysitetemplate.xml' -Source $TemplatePath + '/yourfile.txt' -Folder '/your document library' -FileLevel Published

What is the version of the Cmdlet module you are running?
Pnp.PowerShell 1.9.0
Which operating system/environment are you running PnP PowerShell on?
macOS Catalina

@gingemonster - I'm not very familiar with the [System.IO.Path]::GetTempPath() but trying on a Mac it's already giving me some issues. Doing a bit of research, I have found this closed issue which is what I get on MacOS Monterey.
And looking at the Microsoft documentation, I don't see MacOS. But once again, I'm not familiar with it.
Does it still download the file when you run the script? (without the last line Add-PnPFileToSiteTemplate......)

Working on a fix right now, will hopefully be in tomorrow.

@brannmar - thanks for looking into this. Will await your PR :)
Just curious, are you able to repro it on Windows as well ? Is it PS 5 or 7 or both ?
@veronicageek - is it not working on Mac ?

I added a PR https://github.com/pnp/powershell/pull/1569 that solves the issue where Add-PnPFileToSiteTemplate throwed The process cannot access the file 'filename' because it us being used by another process
@gautamdsheth Did you close the issue by accident?
I was able to reproduce it in windows using PS 7.

@brannmar  - yeah my bad, sorry.
Looking into your PR , how is it fixing Get-PnPFile  which is what was raised here ? Seems like a separate issue compared to this one.

@brannmar - re-reading this, looks like I am mistaken again. Sorry 😟

Closing this now, the PR related to this has been merged.
It will be available in tomorrow's nightly , please do reopen the issue if it persists :)
Thank you @gingemonster for raising the issue and @brannmar  for fixing it :)


fix broken macos github action

found the source of the issue: the source is process forking on macOS after High Sierra. Solution: https://github.com/rails/rails/issues/38560#issuecomment-1159834445
Asmir, feel free to just merge this one upon approval

wat...
this is a fun one...


Issue using locally generated Swashbuckle.AspNetCore.Cli package

I'm trying to generate a local, test package of Swashbuckle.AspNetCore.Cli.
After creating the .nupkg with a new version (no code modifications yet) and correctly restoring it, I'm getting the following error when using it:
  Unhandled Exception: System.IO.FileLoadException: Could not load file or assembly 'Swashbuckle.AspNetCore.Swagger, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=62657d7474907593'. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
     at Swashbuckle.AspNetCore.Cli.Program.<>c.<Main>b__0_3(IDictionary`2 namedArgs)
     at Swashbuckle.AspNetCore.Cli.CommandRunner.Run(IEnumerable`1 args)
     at Swashbuckle.AspNetCore.Cli.CommandRunner.Run(IEnumerable`1 args)
     at Swashbuckle.AspNetCore.Cli.Program.Main(String[] args)

Package generation commands tried (run at solution level):
dotnet pack -c Release --include-symbols --include-source -o  C:\local-nuget-repo --version-suffix alpha
dotnet pack -c Release --include-symbols --include-source -o  C:\dev\prd\src\local-nuget-repo /p:Version=3.1.0-alpha

@eduherminio were you able to resolve your issue? I'm seeing a similar issue.

I did manage to solve it:
Try generating packages of all projects and using generated ones together with your version of Swashbuckle.AspNetCore.Cli.
Let me know if that works for you!


Korifi API users should be able to consume any managed service credential

Blockers/Dependencies
Blocked by managed service support!
Background
As of today korifi is assuming that service credentials are key-value string mappings. However according to the v3 api spec the credential can be a more complex json object.
Here is an example of a service binding secret using more complex credentials:
{
  ".metadata": '{"credentialProperties":[{"name":"foo","format":"json"},{"name":"bar","format":"text"}],"metaDataProperties":[{"name":"md1","format":"text"},{"name":"md2","format":"json"}]}',
  "foo": '[{"a": "b"}]',
  "bar": "yes, please",
  "md1": "first metadata",
  "md2":'[{"md2": "md2value"}]'
}

Korifi should not assume a flat map as the structure of the service binding credentials
Action to take
Think of a better way of modeling thet service binding credentials. A map[string]any sounds like a better option.

FWIW, here's some OSBAPI client code for binding responses. Credentials should be a map of string to interface.

I left a comment on Slack about this as well:
https://cloudfoundry.slack.com/archives/C0297673ASK/p1680190415283189
Basically Cloud Controller in CF for VMs is storing credentials as serialized JSON in a TEXT column in the database, so any valid JSON object type should be supported. One thing I think we could do is actually store the original input off on the UPSI Secret under a key like KORIFI_SERIALIZED_CREDENTIALS or CF_SERIALIZED_CREDENTIALS (or something) and use that when constructing VCAP_SERVICES. We should still break out the credentials into individual keys/values on the Secret for K8s Service Binding support, but having what was originally provided through cf cups SERVICE_NAMED -p '{"some-cred": "some-value", "other-cred": 9000"}' feels like what we need to do here.

It would be better to defer the modeling of the service binding credentials to when we start writing the managed service proposal


1st_version of the github_auto_add

Hi @haesleinhuepf, would you mind helping me review it?
This PR relates to #185
This PR closes #185

Why can I not write this link like you did "https://github.com/NFDI4BIOIMAGE/training/blob/2f17431f8fbd1ee4ffb205eb6fe26474a04ec41a/scripts/auto-add-zenodo-entry.py#L1"?

Hi @SeverusYixin ,
awesome, thanks for this! Two things:

Before sending a PR, please check if the tool works as you describe it in the documentation.
I just tested it and received an error message. That's the command I ran:

python scripts/auto-add-github-resources.py nfdi4bioimage/training 236

And that's the error:
-> get_conversation_on_issue(nfdi4bioimage/training, 236)
Request GET /repos/rmassei/ymia_python_event_series_material/collaborators?per_page=1 failed with 403: Forbidden
Traceback (most recent call last):
  File "C:\structure\code\training\scripts\auto-add-github-resources.py", line 139, in <module>
    main()
  File "C:\structure\code\training\scripts\auto-add-github-resources.py", line 34, in main
    github_data_dict = complete_github_data(github_repo_url)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\structure\code\training\scripts\auto-add-github-resources.py", line 97, in complete_github_data
    if collaborators.totalCount > 0:
       ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\rober\miniforge3\envs\devbio-napari-env\Lib\site-packages\github\PaginatedList.py", line 192, in totalCount
    headers, data = self.__requester.requestJsonAndCheck(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\rober\miniforge3\envs\devbio-napari-env\Lib\site-packages\github\Requester.py", line 550, in requestJsonAndCheck
    return self.__check(*self.requestJson(verb, url, parameters, headers, input, self.__customConnection(url)))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\rober\miniforge3\envs\devbio-napari-env\Lib\site-packages\github\Requester.py", line 715, in requestJson
    return self.__requestEncode(cnx, verb, url, parameters, headers, input, encode)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\rober\miniforge3\envs\devbio-napari-env\Lib\site-packages\github\Requester.py", line 812, in __requestEncode
    status, responseHeaders, output = self.__requestRaw(cnx, verb, url, requestHeaders, encoded_input)
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\rober\miniforge3\envs\devbio-napari-env\Lib\site-packages\github\Requester.py", line 846, in __requestRaw
    response = cnx.getresponse()
               ^^^^^^^^^^^^^^^^^
  File "C:\Users\rober\miniforge3\envs\devbio-napari-env\Lib\site-packages\github\Requester.py", line 198, in getresponse
    r = verb(
        ^^^^^
  File "C:\Users\rober\miniforge3\envs\devbio-napari-env\Lib\site-packages\requests\sessions.py", line 602, in get
    return self.request("GET", url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\rober\miniforge3\envs\devbio-napari-env\Lib\site-packages\requests\sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\rober\miniforge3\envs\devbio-napari-env\Lib\site-packages\requests\sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\rober\miniforge3\envs\devbio-napari-env\Lib\site-packages\requests\adapters.py", line 667, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "C:\Users\rober\miniforge3\envs\devbio-napari-env\Lib\site-packages\urllib3\connectionpool.py", line 934, in urlopen
    retries = retries.increment(method, url, response=response, _pool=self)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\rober\miniforge3\envs\devbio-napari-env\Lib\site-packages\github\GithubRetry.py", line 187, in increment
    raise Requester.createException(response.status, response.headers, content)  # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
github.GithubException.GithubException: 403 {"message": "Must have push access to view repository collaborators.", "documentation_url": "https://docs.github.com/rest/collaborators/collaborators#list-repository-collaborators", "status": "403"}

Let me know if I should try again!
And thanks again for working on this. This will be cool to have!
Best, Robert

The problem is that I can not test it, it told me the scads.ai clould refuse me XD

What is "scads.ai clould" in this context? What was the exact error message?

The error above is not about our repository:

Request GET /repos/rmassei/ymia_python_event_series_material/collaborators?per_page=1 failed with 403: Forbidden
...
Must have push access to view repository collaborators.

The error happens when accessing the other repository. I'd say we will not be able to read out collaborators from remote repositories.

The error above is not about our repository:

Request GET /repos/rmassei/ymia_python_event_series_material/collaborators?per_page=1 failed with 403: Forbidden
...
Must have push access to view repository collaborators.

The error happens when accessing the other repository. I'd say we will not be able to read out collaborators from remote repositories.

Yes, I used the code snippet to check locally that the link to our repository is no problem, it show me as PS C:\Users\huang\Documents\GitHub\training\scripts> & 'c:\Users\huang\AppData\Local\Programs\ Python\Python312\python. exe' 'c:\Users\huang.vscode\extensions\ms-python.debugpy-2024.10.0-win32-x64\bundled\libs\debugpy\adapter/ ... /... \debugpy\launcher' "51870" "--" "C:\Users\huang\Documents\GitHub\training\scripts\test.py"
Token found: githu
Repository successfully accessed: NFDI4BIOIMAGE/training
PS C:\Users\huang\Documents\GitHub\training\scripts>
I'm trying to see if this can be avoided, but nothing I've seen so far seems to work! Or maybe not with github token, I'm not sure

nothing I've seen so far seems to work!

Yes, it is possible that this will not work. Do you have an overview about which meta data we can access and which not?

nothing I've seen so far seems to work!

Yes, it is possible that this will not work. Do you have an overview about which meta data we can access and which not?

At the moment, it looks like the request will be denied when you try to access it with github token.

Can you please try "repo.get_contributors()" instead of "repo.get_collaborators()"?

Can you please try "repo.get_contributors()" instead of "repo.get_collaborators()"?

Let me try

Did it work?

Did it work?

no

Please do not merge. There are still some details to discuss :)

Currently it looks like #238, so again, what should the name be, "ymia_python_event_series_material" or "YMIA - Python-Based Event Series Training Material"?

and for tags, do we need this "tags: No tags available"? or this "tags:   "

"ymia_python_event_series_material" or "YMIA - Python-Based Event Series Training Material"?

I would use whatever the github returns as repo.name

2. and for tags, do we need this "tags: No tags available"? or this "tags:   "

If there are no tags, we should not have  a "tag:" in our yaml.

no

Consider posting an error message, in case you wish that I assist you.

Consider posting an error message, in case you wish that I assist you.

Already solved, thank you if I need that I will post a error message. Thank you again :)

Are there any news here @SeverusYixin ? Shall I test it agian?

Are there any news here @SeverusYixin ? Shall I test it agian?

Seems like you didn't see issues #185.
I asked you how should we define the type and tag for a github repositorie. Except for type and day, all other functions are implemented.

write something for the tags like "Dear users, please add a tags"

And never have empty strings

Hi @haesleinhuepf, please review again, if everything is ok then we can close and merge this.
Thank you :)


[Bug]: Direct Runner sometimes rises RuntimeError: dictionary changed size during iteration

What happened?
Apache Beam: 2.46.0
Direct Runner
We are running some parts of our pipelines using Direct Runners, since we are processing small batches of data there. The rest is using Dataflow runner, which has some auto-recovery. For direct runners it fails the run and triggers alerts for SRE team.
I couldn't reproduce the issue myself, the issue is non-deterministic.
The part of our company's proprietary system sometimes (not very often ~100 runs) fails with:

apache_beam/runners/worker/sdk_worker.py in shutdown_inactive_bundle_processors at line 585

Exception in thread Thread-37:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/threading.py", line 980, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.9/site-packages/sentry_sdk/integrations/threading.py", line 72, in run
    reraise(*_capture_exception())
  File "/usr/local/lib/python3.9/site-packages/sentry_sdk/_compat.py", line 60, in reraise
    raise value
  File "/usr/local/lib/python3.9/site-packages/sentry_sdk/integrations/threading.py", line 70, in run
    return old_run_func(self, *a, **kw)
  File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/data_plane.py", line 228, in run
    self._function(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 585, in shutdown_inactive_bundle_processors
    for descriptor_id, last_access_time in self.last_access_times.items():
RuntimeError: dictionary changed size during iteration

Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
Issue Components

[X] Component: Python SDK
[ ] Component: Java SDK
[ ] Component: Go SDK
[ ] Component: Typescript SDK
[ ] Component: IO connector
[ ] Component: Beam examples
[ ] Component: Beam playground
[ ] Component: Beam katas
[ ] Component: Website
[ ] Component: Spark Runner
[ ] Component: Flink Runner
[ ] Component: Samza Runner
[ ] Component: Twister2 Runner
[ ] Component: Hazelcast Jet Runner
[ ] Component: Google Cloud Dataflow Runner

we should be able to fix this by adding a lock same as https://github.com/apache/beam/blob/41e6628498c6ebdfc2932c61b68bcbde4da1a47e/sdks/python/apache_beam/runners/worker/sdk_worker.py#L470 .

Hi! It's my first time contributing and I would love to give this a shot!

.takeissue

great, thanks a lot, @Anneysha7

Hey @Anneysha7 , just wondering if you are still interested or need help. Thanks!

Hi, sorry for the late reply, still interested! I'll come back to you asap
if I need help.
On Tue, 8 Aug 2023, 12:20 tvalentyn, @.***> wrote:

Hey @Anneysha7 https://github.com/Anneysha7 , just wondering if you are
still interested or need help. Thanks!
—
Reply to this email directly, view it on GitHub
https://github.com/apache/beam/issues/27501#issuecomment-1668811225, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AEWUSWJCTS3535W7GYL2QPLXUGO73ANCNFSM6AAAAAA2J7D2LU
.
You are receiving this because you were mentioned.Message ID:
@.***>

Hi @Anneysha7 i'd like to have this issue fixed in next version of Beam esp since it should be a simple fix.  I suggest  we unassign it for now in case someone else has time to take a look - no worries you can always pick this or any other issue when you have time to work on Beam.

Hey, would like to give this a go. Might take some time since it would be my first time contributing to apache beam. I'll assign it to myself, feel free to reassign if the criticality requires faster fix @tvalentyn. Thanks

.take-issue

Realised I am having bandwidth issues, incase issue still open post couple of week, I'd like to give it a try.

.take-issue

Hi, I have set up my dev environment and will give it a try. It is my first time contributing too, so I am grateful for any guidance. For now, I'll add a lock in the required section

@lrakla Sounds like a plan, thanks for your help. We also have a dev wiki at https://s.apache.org/beam-python-dev-wiki, which you might find helpful.


haskellPackages.hledger*: refactors

Description of changes
Improves situation for latest hledger and cleans up some bits and bops. Best reviewed commit by commit.

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

@ofborg build hledger hledger-web hledger-ui haskellPackages.hledger_1_30_1 haskellPackages.hledger-web_1_30

Excellent!


Changed searching for guess

Changed searching returned list to searching database.
Have not verified changes work. Code comes from documentation -> https://pymongo.readthedocs.io/en/stable/tutorial.html#counting

Hey @CaptainChris425 thanks for opening a PR, I am not sure this would work as I am using the motor lib. I believe to use this logic, it would need to be awaited motor docs
guessed = await collection.count_documents({"lhguess": str(guess).lower()})
if guessed > 0:

Something like this I believe.

Hi again @CaptainChris425 I went ahead and made the change myself. Thanks for pointing this out! 😄
https://github.com/alexraskin/lhbot/blob/81f37c5774f08b205cad0815bd7c2c0bc7c6fee6/cogs/lhguess.py#L71


rootless container

Support rootless docker container

discard this PR since the rootless container is implemented by PR #14


New Game: Parade Chess Solitaire

Neighbour solitaire with a different move mechanic: pieces pick up an opposite colour piece from an attacked square and place it on another attacked square.

Abandoning, because it's too similar to Chess Golf and also a bit dry.


Updated reveal.js to v 4.10

Purpose
Update reveal.js to v4.10 and include some additional useful plugints, addresses #1802
Update also adds chalkboard and menu plugins.
Documentation
Please note while this update works, it requires the following:
1- testing that mathjax and speaker notes work (currently works on my setup, mac + safari)
2- I think it is possible to make presentations configurable from front matter of slides. I am not familiar with hugo templating to make this work, and hope someone can give guidance or assist with this.

Very excited about this. I'll try to test it as much as I can

I made a quick test of this and it seems shortcodes don't work. I tried to add a figure in the example section.md and fine, but if I add the same figure using the same code I see nothing in the slide. I also tried with a Youtube video and a custom section.html shortcake I had and nothing, that's why I think it has to do with shortcodes in general. Could you please take a look @koutbo6? Thanks a lot for this contribution!

@rodrigoalcarazdelaosa just to be clear ... are shortcodes broken in this pr? or shortcode dont work in slides only?
Can you confirm that shortcodes did work in slides before? can you give an example broken shortcode to make sure I view the same thing that you do

@rodrigoalcarazdelaosa just to be clear ... are shortcodes broken in this pr? or shortcode dont work in slides only?
Can you confirm that shortcodes did work in slides before? can you give an example broken shortcode to make sure I view the same thing that you do

I added a figure in the example section.md of the test site and it did work. I added the same shortcode in a slide (using the reveal4 branch) and it didn't work (I saw nothing and inspecting the HTML I couldn't see the figure). This is the live version of the slide I was testing (this one is using reveal 3 and you see images and videos working fine).

@rodrigoalcarazdelaosa I can confirm youtube and icon shortcodes work in slides on my end ... would appreciate a concrete example that fails on your end.
here is the index.md that I am using which works perfectly .... how can we debug this and make sure we have the same configuration? .....

let me create a new clone and site from the pr and try to replicate what you did ... if u can share some slide source that u know doesnt work .. that would help

@rodrigoalcarazdelaosa I can confirm youtube and icon shortcodes work in slides on my end ... would appreciate a concrete example that fails on your end.
here is the index.md that I am using which works perfectly .... how can we debug this and make sure we have the same configuration? .....

Are you using the test site shipped with Wowchemy?

@rodrigoalcarazdelaosa I did the following and it worked for me:
1- Cloned reveal4 branch from koutbo64
2- Cd to test directory, then created slides/example/ directories
3- Created my index.md for slides under example directory
4- started the hugo server and opened path slides/example ... and it worked
I noticed that if I dont follow this directory structure for slides where I place them under the root "slides" directory .. then slides will not render correctly
But everything else seems to work fine .. for math and diagrams you have to enable math: true and diagrams: true in front matter or set them to true in params.toml
I'm using latest hugo version on a mac .... any feedback welcome

just pushed the test slides I used under test:
/test will show slides that dont render correctly
/slides/example works fine

I've also been looking into shortcodes. So far only been looking at fragments. That one is absolutely broken!
The markdown parser must have changed, because it seems that raw html is simply ignored now. I'm looking at the slides used in the academic-starter repo, ./exampleSite/content/slides/example/index.md. Looking at slide 6, the rendered html source is:

Fragments

Make content appear incrementally

{{% fragment %}} One {{% /fragment %}}
{{% fragment %}} **Two** {{% /fragment %}}
{{% fragment %}} Three {{% /fragment %}}

Press Space to play!

One

Two

Three

Looking at the output of the fragment shortcode:

  {{ .Inner }}

Which is far from what reveal is now expecting (https://revealjs.com/markdown/#element-attributes):

This suggests that all shortcodes that simply returns html is no longer usable! And so, we would need to rewrite all current "slides only" shortcodes and make special slides shortcodes for things like figures, which are not a slides specific shortcode.

Other shortcodes I've checked out:

speaker_note shortcode still works.
slide shortcode is broken.

Long code blocks can't be scrolled without also moving through slides.
We need a to add data-prevent-swipe to code blocks.

    Can't change slides by swiping across this element.

Relevant: https://revealjs.com/markdown/

This suggests that all shortcodes that simply returns html is no longer usable! And so, we would need to rewrite all current "slides only" shortcodes and make special slides shortcodes for things like figures, which are not a slides specific shortcode.

If only there was a way to detect if a shortcode is being called from inside a slide, we could then add slide-specific code

This suggests that all shortcodes that simply returns html is no longer usable! And so, we would need to rewrite all current "slides only" shortcodes and make special slides shortcodes for things like figures, which are not a slides specific shortcode.

If only there was a way to detect if a shortcode is being called from inside a slide, we could then add slide-specific code

Probably using the .Parent variable

This suggests that all shortcodes that simply returns html is no longer usable! And so, we would need to rewrite all current "slides only" shortcodes and make special slides shortcodes for things like figures, which are not a slides specific shortcode.

If only there was a way to detect if a shortcode is being called from inside a slide, we could then add slide-specific code

Probably using the .Parent variable

Why not use jquery in baseof.html to inject these directives to relevant tags . ...
Think we can automate testing for this issue? at the very least, let's have a unifide testcase that shows all these issues so we can manually check, and maybe a checklist of what is broken?

Other shortcodes I've checked out:

speaker_note shortcode still works.
slide shortcode is broken.

can confirm that as well ..
Btw .. i created a copy of the starter slides under /test/
http://localhost:1313/slides/example/
I'm going to modify slightly to illustrate all the issues we know so far
If you come across another issue not there ... please let me know or assist by updating the slides to reflect it
Consider the slides our checklist for now for what needs to be fixed
Thank you all for the feedback

Long code blocks can't be scrolled without also moving through slides.
We need a to add data-prevent-swipe to code blocks.

    Can't change slides by swiping across this element.

maybe this is related to enabling mousewheel scrolling? can u try disabling it (see front matter configuration in test slide) and see if that helps
would appreciate a copy of what you are testing to make sure we are on the same page ... is the code lengthy on vertical or horizontal axes?

@rodrigoalcarazdelaosa @Chrede88
managed to solve the omitted HTML problem by adding the following to config.yaml for the test site:
markup:
  goldmark:
    renderer:
      unsafe: true

more information available here:
https://stackoverflow.com/questions/63198652/hugo-shortcode-ignored-saying-raw-html-omitted
I also noticed that this resulted in callout shortcode working fine for me for regular pages, which didnt work before
Note sure whether there is a security concern or not there.

fragements worked in slides after this change

@rodrigoalcarazdelaosa @Chrede88
managed to solve the omitted HTML problem by adding the following to config.yaml for the test site:

markup:

  goldmark:

    renderer:

      unsafe: true

more information available here:
https://stackoverflow.com/questions/63198652/hugo-shortcode-ignored-saying-raw-html-omitted
I also noticed that this resulted in callout shortcode working fine for me for regular pages, which didnt work before
Note sure whether there is a security concern or not there.

That's the default configuration for normal templates, not sure why @gcushen didn't include it in the test site

code code highlighting .. html seems to be generated correctly to highlight code
However, it is then escaped ... afraid I'll have to stay away from this for a few days
if anyone can suggest the cause of why html is being escaped and can suggest at least where to look to fix this... would help greatly

@rodrigoalcarazdelaosa @Chrede88
managed to solve the omitted HTML problem by adding the following to config.yaml for the test site:
markup:
  goldmark:
    renderer:
      unsafe: true

more information available here:
https://stackoverflow.com/questions/63198652/hugo-shortcode-ignored-saying-raw-html-omitted
I also noticed that this resulted in callout shortcode working fine for me for regular pages, which didnt work before
Note sure whether there is a security concern or not there.

Further testing with the latest version. Now I can see Youtube videos and figures but for example I'm no seeing , try this one (which works with the current version).

A custom shortcode for sections that I had is not working either.

ok i think the issue is not with the new code but the configuration of the test server
I managed to get @rodrigoalcarazdelaosa iframe code to work on my academic starter website perfectly .. and all the issues with the code highlighting are resolved there without making any changes
to confirm this try the following:
run the iframe example on test site shipped with wowchemy on master branch and on reveal4 branch
You will notice both will not work .. this tells us it is not to do with our code but something else in the configuration
To further confirm this, create a new academic starter template site and point the go.mod to the reveal4 branch of wochemy and test slides there ... everything works perfectly
Its clear to me now that the issue is likely hugo related rather than reveal .. we just need to flip the config switches that will fix our problems
I'll keep u posted

ok i think the issue is not with the new code but the configuration of the test server
I managed to get @rodrigoalcarazdelaosa iframe code to work on my academic starter website perfectly .. and all the issues with the code highlighting are resolved there without making any changes
to confirm this try the following:
run the iframe example on test site shipped with wowchemy on master branch and on reveal4 branch
You will notice both will not work .. this tells us it is not to do with our code but something else in the configuration
To further confirm this, create a new academic starter template site and point the go.mod to the reveal4 branch of wochemy and test slides there ... everything works perfectly
Its clear to me now that the issue is likely hugo related rather than reveal .. we just need to flip the config switches that will fix our problems
I'll keep u posted

Interesting. Perhaps @gcushen wants to revise the test site configuration so the tests we perform there are meaningful and compare well to real life situations. I'll make sure to test your branch with a starter-academic template

Give me a few minutes ... I think I'll have a better working test side this time

Give me a few minutes ... I think I'll have a better working test side this time

Great. I'll also test my already created slides to check nothing weird appears

Long code blocks can't be scrolled without also moving through slides.
We need a to add data-prevent-swipe to code blocks.

    Can't change slides by swiping across this element.

maybe this is related to enabling mousewheel scrolling? can u try disabling it (see front matter configuration in test slide) and see if that helps
would appreciate a copy of what you are testing to make sure we are on the same page ... is the code lengthy on vertical or horizontal axes?

It has to do with touch navigation: https://revealjs.com/touch-navigation/
The problem comes up when code blocks extend beyond the page bounds horizontally.

@Chrede88 check the front matter .. i think u can disable it there to fix this behavior
just a flag .. I'll check it in my next update to the test site

@Chrede88 check the front matter .. i think u can disable it there to fix this behavior
just a flag .. I'll check it in my next update to the test site

Right! I think touch navigation is important:) I don't want to get rid of it, but just make sure code blocks are scrollable ;)

ok this part was the culprit from config file (im using toml here not yaml):
  [markup.highlight]
    codeFences = false

ok please add this to config.yaml to fix all previously mentioned issues:
markup:
  defaultMarkdownHandler: goldmark
  goldmark:
    renderer:
      unsafe: true
  highlight:
    codeFences: false

should fix:

iframe
code highlighting
slide shortcode

I'll update the code to also add touch and loop frontmatter options and correct the config file

please check the updated test site
improvements include adding loop and touch configurations to slide front matter
also please visit /test when u run the test site
you can see all the shortcodes that work in regular pages, but not slides even in older versions of wowchemy
Do we want to fix those?
I would like to see diagrams and charts work ... any thoughts on how to achieve that?
maybe another pr after this one is tested and merged
Thanks for the supprt!

@Chrede88 check the front matter .. i think u can disable it there to fix this behavior
just a flag .. I'll check it in my next update to the test site

Right! I think touch navigation is important:) I don't want to get rid of it, but just make sure code blocks are scrollable ;)

@Chrede88 , would surrounding long code with the following resolve the issue as the link you posted suggests?

// code here

@Chrede88 check the front matter .. i think u can disable it there to fix this behavior
just a flag .. I'll check it in my next update to the test site

Right! I think touch navigation is important:) I don't want to get rid of it, but just make sure code blocks are scrollable ;)

@Chrede88 , would surrounding long code with the following resolve the issue as the link you posted suggests?

// code here

Yes:)

add the following slides from matter configurations:
diagram: to enable or disable mermaid diagrams
diagram_theme: to set the preferred diagram theme
enjoy

mermaid diagram works in slides now
Problem was lazy loading of slide contents, there is no event fired when slide is loaded, so added some code to sense if we are close to a slide that is already loaded and will render mermaid
Noticed the following:
1- printing pdfs will print the mermaid code rather than diagram, unless you view the slides containing the diagrams to render them
2- mermaid code is now sensitive to whitespace, empty lines will result in syntax error
3- Flowcharts sometime do not render nodes correctly, and this seems to be an issue with mermaid.

mermaid diagram works in slides now
Problem was lazy loading of slide contents, there is no event fired when slide is loaded, so added some code to sense if we are close to a slide that is already loaded and will render mermaid
Noticed the following:
1- printing pdfs will print the mermaid code rather than diagram, unless you view the slides containing the diagrams to render them
2- mermaid code is now sensitive to whitespace, empty lines will result in syntax error
3- Flowcharts sometime do not render nodes correctly, and this seems to be an issue with mermaid.

I see some issues with some of the diagrams but that's probably caused by the diagram itself? Or probably due to mermaid not playing nice with reveal (upstream then).

Testing one of my already created slides I see that now the slide, when we reach the end, goes back to the beginning again. This also happens with sections (moving vertically), which is a bit weird but probably default behavior with the new version?

@rodrigoalcarazdelaosa
Most likely its a configuration issue for everything you mentioned .. and the rendering is mermaid related
You might want to try the new directives feature in mermaid:
{{% diagram %}}
%%{init: { 'logLevel': 'debug', 'theme': 'neutral' } }%%
graph TD;
%%{config: { 'fontFamily': 'Menlo', 'fontSize': 18, 'fontWeight': 400} }%%
    A-->B;
    A-->C;
    B-->D;
    C-->D;
{{% /diagram %}}

Try changing the configuration of the diagram see if you can get better results.
The issue so far is with the flowdiagram only
For the loop back, can you share the code to the presentation? I need to experience the error

For the loop back, can you share the code to the presentation? I need to experience the error

You can try any of the slides I have here. You'll need images and a custom section.html shortcode.

@gcushen seems like Hugo modifies the case used in front matter configuration keys.
For example, slideNumber will be lower cased to slidenumber when accessed from the template parameter context variable.
Is there a way to maintain case? this is breaking configuration for any js plugins that uses camel case to name configuration keys.
Only solution I can think of is to use snake case and convert them into camel case.

lots of changes in this one:

menu now included from CDN, but not chalkboard
configurations in front matter changed:

added mermaid_enabled under slides to enabled/disable mermaid, with default being disabled
added menu_enabled and chalkboard_enabled under reveal_options to enable/disable each feature, with default being enabled
configurations under reveal_options are converted into a js object, with keys that are snake case converted into camel case since this is what reveal.js uses. All plugins can also be configured by including their key and exactly like their documentation states. see the menu example.

we now have a rudimentary way of adding reveal plugins. chalkboard can be used as an example to demonstrate how it works. By including js distribution under assets. Another options is to link to a CDN. Configuration could be passed from front matter. The only problem is how to initialize reveal. I'm not sure if the initialize method could be called multiple times or just a single time. But its something I have no urgency to fix now and could tackle in the future when I have time.
Note that issues related to rendering diagrams seem to be relation to transitions. Slide and fade transitions work best with mermaid.
Caught a few bugs, like mermaid would not render if you dont use transitions. Now it works with no transitions also. Cant remember the rest.

@gcushen incorporated your last suggestions and cleaned up what I can find
mainly:
1- id=theme was removed along with code that reacted to it in wowchemy-slides.js
2- used the diagram=true variable and added diagram_options to pass on mermaid configuration.
I see you are refactoring fixMermaid function so I didnt mess with that. I'll leave that to you.

@gcushen synced with upstream and now using utils files.

@gcushen need to clean up the commit history .. please do not merge

@gcushen fixed the PR and its commit history. Managed to duplicate all commits
now using fixMermaid from utils
I think we are set with this PR from my end

PR should be now in sync with upstream

Todo
Remaining todo includes:

[ ] Remove Mermaid security param from user options - https://github.com/wowchemy/wowchemy-hugo-modules/pull/2018/files#r546721818
[ ] Add Mermaid security param back to Wowchemy Theming: https://github.com/wowchemy/wowchemy-hugo-modules/pull/2018/files#r550828149
[ ] Refactor to use IF statement: https://github.com/wowchemy/wowchemy-hugo-modules/pull/2018/files#r550828354
[ ] Remove jQuery: https://github.com/wowchemy/wowchemy-hugo-modules/pull/2018/files#r546876161
[ ] Why are you proposing to send Reveal CDN URL as a JS param? Remove/refactor unless it's really needed: https://github.com/wowchemy/wowchemy-hugo-modules/pull/2018/files#r545006141
[ ] Remove the duplicate config file you propose adding: https://github.com/wowchemy/wowchemy-hugo-modules/pull/2018/files#diff-3955202dbb75fab5f35e0092bdaf9dbd06f5561b74764b09d08287067b47d321
[ ] Remove test/content/test/index.md - https://github.com/wowchemy/wowchemy-hugo-modules/pull/2018/files#diff-55d0f2cfdfcec540460113e76b317ee121205ba37269f4f774a6f11fe605856e
[ ] Revert changes to test/go.mod and test/go.sum
[ ] Revert changes to test/content/home/section.md

Todo
Remaining todo includes:

[ ] Remove Mermaid security param from user options - https://github.com/wowchemy/wowchemy-hugo-modules/pull/2018/files#r546721818
[ ] Add Mermaid security param back to Wowchemy Theming: https://github.com/wowchemy/wowchemy-hugo-modules/pull/2018/files#r550828149
[ ] Refactor to use IF statement: https://github.com/wowchemy/wowchemy-hugo-modules/pull/2018/files#r550828354
[ ] Remove jQuery: https://github.com/wowchemy/wowchemy-hugo-modules/pull/2018/files#r546876161
[ ] Why are you proposing to send Reveal CDN URL as a JS param? Remove/refactor unless it's really needed: https://github.com/wowchemy/wowchemy-hugo-modules/pull/2018/files#r545006141
[ ] Remove the duplicate config file you propose adding: https://github.com/wowchemy/wowchemy-hugo-modules/pull/2018/files#diff-3955202dbb75fab5f35e0092bdaf9dbd06f5561b74764b09d08287067b47d321
[ ] Remove test/content/test/index.md - https://github.com/wowchemy/wowchemy-hugo-modules/pull/2018/files#diff-55d0f2cfdfcec540460113e76b317ee121205ba37269f4f774a6f11fe605856e
[ ] Revert changes to test/go.mod and test/go.sum
[ ] Revert changes to test/content/home/section.md

@dralmarzouq thanks! Please can you push your changes to the PR so that we can see them?

@dralmarzouq thanks! Please can you push your changes to the PR so that we can see them?

@dralmarzouq thanks! Please can you push your changes to the PR so that we can see them?

though I already did .. had an issue with authentication
should be pushed now

@dralmarzouq thanks! Please can you push your changes to the PR so that we can see them?

though I already did .. had an issue with authentication
should be pushed now

@dralmarzouq thanks! Please can you push your changes to the PR so that we can see them?

though I already did .. had an issue with authentication
should be pushed now

@dralmarzouq thanks! Please can you push your changes to the PR so that we can see them?

though I already did .. had an issue with authentication
should be pushed now

@dralmarzouq thanks! Please can you push your changes to the PR so that we can see them?

though I already did .. had an issue with authentication
should be pushed now

test/go.mod and test/go.sum still haven't been reverted in the PR - they should not appear in the "Files changed" tab...

@dralmarzouq thanks! Please can you push your changes to the PR so that we can see them?

though I already did .. had an issue with authentication
should be pushed now

test/go.mod and test/go.sum still haven't been reverted in the PR - they should not appear in the "Files changed" tab...

@dralmarzouq thanks! Please can you push your changes to the PR so that we can see them?

though I already did .. had an issue with authentication
should be pushed now

test/go.mod and test/go.sum still haven't been reverted in the PR - they should not appear in the "Files changed" tab...

ok I think its done correctly now ... files changed dropped from 11 to 9
I also synced with upstream .. without duplicating the commit history this time

@dralmarzouq thanks! Please can you push your changes to the PR so that we can see them?

though I already did .. had an issue with authentication
should be pushed now

test/go.mod and test/go.sum still haven't been reverted in the PR - they should not appear in the "Files changed" tab...

ok I think its done correctly now ... files changed dropped from 11 to 9
I also synced with upstream .. without duplicating the commit history this time

please take note I'm fixing the mess that it my development pc settling on the malmarz username and profile

please take note I'm fixing the mess that it my development pc settling on the malmarz username and profile

improvement done and synced with upstream

improvement done and synced with upstream

@gcushen any further updates to pr? It is it ready for merging?

@gcushen any further updates to pr? It is it ready for merging?

@gcushen any further updates to pr? It is it ready for merging?

Appears to be no response yet on a couple of review comments:

https://github.com/wowchemy/wowchemy-hugo-modules/pull/2018/files#r550933338
https://github.com/wowchemy/wowchemy-hugo-modules/pull/2018#discussion_r554145985

Also, it's best to install Yarn and Node and run the following JS lint and format commands (https://github.com/wowchemy/wowchemy-hugo-modules/blob/master/.github/contributing.md#where-to-start):
cd wowchemy
yarn install
yarn run lint
yarn run format

@gcushen any further updates to pr? It is it ready for merging?

Appears to be no response yet on a couple of review comments:

https://github.com/wowchemy/wowchemy-hugo-modules/pull/2018/files#r550933338
https://github.com/wowchemy/wowchemy-hugo-modules/pull/2018#discussion_r554145985

Also, it's best to install Yarn and Node and run the following JS lint and format commands (https://github.com/wowchemy/wowchemy-hugo-modules/blob/master/.github/contributing.md#where-to-start):
cd wowchemy
yarn install
yarn run lint
yarn run format

Thanks for contributing to upgrade the slides capability @malmarz :)

Thanks for contributing to upgrade the slides capability @malmarz :)

Closes #1802

Closes #1802

Was a pleasure
Keep up the good work
Next pr will likely be smoother .. I hope :)

On 20 Jan 2021, at 2:24 AM, Geo<EMAIL_ADDRESS>wrote:

----==_mimepart_60076a322d2f6_d591a0493882
Content-Type: text/plain;
charset=UTF-8
Content-Transfer-Encoding: 7bit
Thanks for contributing to upgrade the slides capability @malmarz :)
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/wowchemy/wowchemy-hugo-modules/pull/2018#issuecomment-763207556
----==_mimepart_60076a322d2f6_d591a0493882
Content-Type: text/html;
charset=UTF-8
Content-Transfer-Encoding: 7bit

Thanks for contributing to upgrade the slides capability @malmarz :)
—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.

----==_mimepart_60076a322d2f6_d591a0493882--

Was a pleasure
Keep up the good work
Next pr will likely be smoother .. I hope :)

On 20 Jan 2021, at 2:24 AM, Geo<EMAIL_ADDRESS>wrote:

----==_mimepart_60076a322d2f6_d591a0493882
Content-Type: text/plain;
charset=UTF-8
Content-Transfer-Encoding: 7bit
Thanks for contributing to upgrade the slides capability @malmarz :)
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/wowchemy/wowchemy-hugo-modules/pull/2018#issuecomment-763207556
----==_mimepart_60076a322d2f6_d591a0493882
Content-Type: text/html;
charset=UTF-8
Content-Transfer-Encoding: 7bit

Thanks for contributing to upgrade the slides capability @malmarz :)
—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.

----==_mimepart_60076a322d2f6_d591a0493882--


Theme format decision

The CSS-like parser done by Binary1248 is definitely a nice theme loader, but doesn't compile without Boost Spirit. Do you guys think another minimal theme format would be nice to have?

Or I could just throw out spirit and use ell: http://code.google.com/p/ell/ and include it's hand full of headers (14, it is also header only and doesn't abuse templates and meta programming nearly as much as spirit does). It's syntax is almost identical to boost's except it leaves out the more complex features that I didn't use anyway. Because of that porting the code now to ell shouldn't take more than a few hours. Remember, the style parser is purely optional (maybe even disable it by default in CMake), so people won't mind the extra compile time if they choose to use it. I find a CSS-like theme makes sense for SFGUI. It describes the hierarchy and property system perfectly.

Good idea IMHO. Still better than including<PHONE_NUMBER>4 Boost headers IMHO.

@binary1248: Do you feel like implementing the ell stuff? ;)

Yeah.. after I'm done fixing up the pull requests.

Starting work on it now.


Bugfixes

This pull request fixes the following issues:

#54
#46
#55
#53
#41

What the heck is this bot? Suggesting a bunch of nonsensical indentation changes and then saying my code fails because the bot "doesn't support import/export declarations"

Ignore it, I hadnt set it up properly yet.


No makefile for Sparkfun Edge

Dear Team,
It is the first time I am using the TFLM and the Sparkfun Edge board. I am following the tutorial on Sparkfun and the Colab
tutorial : https://codelabs.developers.google.com/codelabs/sparkfun-tensorflow/#2
I can compile and link an example for the Sparkfun Edge using the Ambiq SDK.
I am trying now to generate an example with TFLM that I want to run on the Sparkfun Edge to check if the workflow is properly set up.
I installed all the tools needed on Windows.
When I launch make, the sparkfun_edge_makefile.inc is not found because the makefile is not in the GitHub repo.
$ make -f tensorflow/lite/micro/tools/make/Makefile TARGET=sparkfun_edge micro_speech_bin
tensorflow/lite/micro/tools/make/downloads/flatbuffers already exists, skipping the download.
tensorflow/lite/micro/tools/make/downloads/pigweed already exists, skipping the download.
tensorflow/lite/micro/tools/make/Makefile:497: tensorflow/lite/micro/tools/make/targets/sparkfun_edge_makefile.inc: No such file or directory
make: *** No rule to make target 'tensorflow/lite/micro/tools/make/targets/sparkfun_edge_makefile.inc'.  Stop.
Any recommandation or alternative ?
Many thanks in advance for your kind help
ClaudeZo

We are looking for someone to maintain the Sparkfun port of TFLM, see https://github.com/tensorflow/tflite-micro/issues/407#issuecomment-898660589 for additional context.
For now, you can try https://github.com/advaitjain/tflite-micro-sparkfun-edge-examples, with the caveat that we are not able to provide any support.
If you would be interested in being a maintainer for TFLM on Sparkfun Edge, then please let us know.

We are looking for someone to maintain the Sparkfun port of TFLM, see #407 (comment) for additional context.
For now, you can try https://github.com/advaitjain/tflite-micro-sparkfun-edge-examples, with the caveat that we are not able to provide any support.
If you would be interested in being a maintainer for TFLM on Sparkfun Edge, then please let us know.

What is the make command to compile the micro-speech for sparkFun-edge, please?


Analyzers

fixes parts of #170
(.Targets update still missing)
Includes support for the built-in rules and the list of rules here: https://github.com/tcartwright/SqlServer.Rules/blob/master/docs/table_of_contents.md
and here: https://github.com/davebally/TSQL-Smells
Includes ability to supress rules and groups of rules

@jmezach I will add a test project, just need to .Targets in place, maybe you can help?

@jmezach I tried and failed, as you can see from a previous commit, maybe you can guide me?

You'll probably want to add a condition on <CodeAnalysisRulesArgument> so that it is only set if certain rules are ignored. Right now you're always passing the -ar command line option, but without any arguments.

@jmezach > You'll probably want to add a condition on
😄 - yeah, makes sense, but I have no clue how to do that - sorry!

Something like <CodeAnalysisRulesArgument Condition="'$(CodeAnalysisRules)'!=''">-ar $(CodeAnalysisRules)</CodeAnalysisRulesArgument>


add support for placeholder templates that display while the template…

… is loading via ajax and is replaced once it is loaded fully
My use case is to have some placeholder that takes up space and shows something like a loading spinner while the content is loading, so it's not just blank while the user waits.

Hey, tbh, I accidentally opened this pull request to your fork, when I meant to just do it to my fork's master. But if you think it's something useful as well, you can merge it. Or close it if it's not interesting to you. 🙂
Overall, cool project!

This change is worth considering. My initial idea was that all spinning circles or whatever else indicators could be added to the placeholder by CSS. But maybe your way to solve this covers more edge cases.

@zgalant For your information, I have just created a new version with your changes plus tests and documentation. Note that I suggest to pass the placeholder_template_name by keyword like this, because this way it is clearer than positional arguments when a new developer first reads this code:
{% include_by_ajax "gallery/includes/latest_pictures.html" placeholder_template_name="utils/loading.html" %}


Refactor ResourceOptions.version and pluginDownloadUrl

ResourceOptions.version and pluginDownloadUrl should be Option[String] rather than String. We should not be misusing Scala's type system by passing empty strings.

review in context of #355

implemented via Input.Optional[String]


Create ca.yml for Catalan Language by majordom_alfred from Fetlife

ca.yml for Catalan speakers

Sorry, bad procedure.  First time


Specimen resource

Make sure the ValueSet for this code ($SCT#395058002) is defined in the terminology file.
Is there a need for identifier in this profile?

DiagnosticReport - Profiles - must include encounter data element.
Task - Profiles - requester and owner data elements must have cardinality specified or have a reference specified.
ServiceRequest - Profiles - doNotPerform data element must be listed under intent data element as per the base profile.

($SCT#395058002 is in the valueset VSSpecimenType

Ahh thanks, fixed diagnostic report and Task :)
However, doNotPerform is a standalone element in the root of ServiceRequest


How to finetune GLM-4V-9B with LoRA?

Feature request  / 功能建议
Add tutorial for fine-tuning GLM-4V-9B using LoRA in https://github.com/THUDM/GLM-4/tree/main/finetune_demo.
Motivation / 动机
I want to apply GLM-4V-9B to a special field and solve industry problems through graphic question-and-answer. I need to fine-tune the model based on the data in this field to improve the performance of the model. However, our computing resources are very prioritized, so we need to use technologies like LoRA to fine-tune the model. I hope you can provide some guidance. Thank you!
Your contribution / 您的贡献
https://github.com/THUDM/GLM-4/blob/main/finetune_demo/README.md

请问能不能直接拿cogvlm2的finetune_demo直接调glm4v

同问同问，期待4v的sft功能解锁

现在可以先试用swift框架微调，我们暂时没有那么高的优先级处理GLM-4b-9B的微调，但是这个任务正在我们的计划中

请问能不能直接拿cogvlm2的finetune_demo直接调glm4v

这个代码没有微调视觉模块，是因为cogvlm2使用的是数据并行，会超过80G显存，GLM-4v-9B可以自行尝试，结构相似

Would it be possible to fine tune the GLM-4V-9B (vision) model with consumer hardware such as one or two rtx3090? I mean using techniques such as LORA etc.
A small example guide for this case would be fantastic and would give a lot of visibility to this model, as it happens with pali gemma.
Congratulations to the authors!

我这里有一份实现的代码可以供参考：
https://github.com/XuRui314/GLM4v-Finetune


support for ASOF joins

What's up?
I am starting to look at PRQL and I was wondering if there is the possibility to support ASOF joins such as in duckdb   I understand this type of support is not in standard sql but It would help to have it translated.

Yes I think this makes sense, and would be quite easy — maybe as easy as adding it to https://github.com/PRQL/prql/blob/04b5174cba01bcc79837e69f0697486210d3d6dd/crates/prql-compiler/src/ir/pl/extra/expr.rs#L111-L112 and catching any places where that's matched against...

It looks depending on sqlparser crate here. So does it mean need to add support in sqlparser crate first?

It looks depending on sqlparser crate here. So does it mean need to add support in sqlparser crate first?

Coming back to this issue — this is correct... I think that would likely be a welcome contribution for them, and not that much code...

FYI this is now released upstream! So could be a reasonable contribution to PRQL
https://github.com/sqlparser-rs/sqlparser-rs/pull/1288


Wrap JS code around a document.body loaded function

On IE11 we often fail to run JS code because when this line of code gets executed the document.body is not loaded yet.
This PR then wraps the code around a document ready VanillaJS function.

Hi @cintamani,
Do you have an example of the sorts of errors you're seeing, and/or a URL where you're seeing this behaviour that I can use to try and reproduce this?
This code has been in place for ~6 years, and we are using the same approach in GOV.UK Frontend, but I'm not aware of this issue having come up before.
As I understand it, at the point the <script> runs, the <body> tag should have been parsed by the browser and be part of the DOM, and so document.body should be defined?
This makes me think there might be something more to this – is there any other JavaScript running on your page that is trying to modify the classes on the body tag? Could there be some sort of race condition going on?

Hi @36degrees! So you can reproduce the issue using this live service we went live with recently: https://wasteexemptions.service.gov.uk/
Feel free to inspect. It doesn't happen all the times, so you will need to refresh until you see it, while keeping the console open.
The error is the following:

From the console is easy to see the line of code causing it, and that's the one I am trying to fix in this PR.
We use this template through this gem: https://github.com/alphagov/govuk_elements_rails
Let me know :)

I spent some time looking into this with @cintamani.
After a bit of digging, we think this might be being caused by their application using turbolinks which hijacks the native browser navigation:

When you follow a link, Turbolinks automatically fetches the page, swaps in its <body>, and merges its <head>, all without incurring the cost of a full page load.

I think this would be a breaking change, as other services may well have scripts that rely on the js-enabled class existing on the body before the DOMContentLoaded event is fired.
For that reason, I'm going to close this now. If anyone else is seeing this behaviour, and isn't using turbolinks, please open an issue.


Implement GetStorageClassClaimConfig RPC call

Added 4 commits for below

Add CephResources to the StorageClassClaim status
Added code to extract the ceph resources from the StorageClassClaim CR status and return back the required details in the
GetStorageClassClaimConfig RPC call.
Add consumerUUID and storageClassClaimName as annotation when creating the StorageClassClaim CR during FulfillStorageClassClaim RPC
Remove rbd/cephfs specific resources from GetStorageConfig as GetStorageConfig is used only to get global ceph resources.

/assign @umangachapagain
/assign @nb-ohad

/retest

@Madhu-1 In addition to the code review above I saw that a lot of unit tests got removed and you didn't write any other unit tests to replace them. Is there a reason?

@Madhu-1 In addition to the code review above I saw that a lot of unit tests got removed and you didn't write any other unit tests to replace them. Is there a reason?

I didn't add any new tests as none of the test cases were removed, As we are removing the cephfs and rbd specific resources from the GetStorageConfig i removed the code where we are creating the rb/dcephfs resources and validating the ceph (rbd/cephfs) Resources in GetStorageConfig. Am working on adding the unit testing for the new RPCs (FulfillStorageClassClaim, RevokeStorageClassClaim and GetStorageClassClaimConfig), will be done in the follow-up PR's.

Ok, thanks for clarifying

/lgtm

/hold cancel


Improve error reporting for bit-constraint based expression solving

The function AffineExpression::try_solve_through_constraints should report why it was not able to solve an expression.

@georgwiese do you know if this was done during your latest/current witgen refactoring?

I'm not doing anything there, but looking at try_solve_through_constraints , it looks like this has been resolved. The reason can be one of:

There is an unconstrained polynomial
Range constraints are overlapping
The expression is unsatisfiable (which is a hard error)


misc: allow empty files for create_feed_document api

Hi,
first of all thanks for this awesome library, it helps us a lot with our daily tasks :+1:
Regarding the /feeds/2021-06-30/documents endpoint, this library provides a very comfortable way of creating a feeds document.
However, the implementation of this endpoint in this library does more than the actual endpoint:
While the endpoints only specifies a response containing a presigned url, which can then be used to upload a file, this library enforces to provide a file.
In case you have the file, this is very convenient, but this is not always the case and does not work with the sandbox api.
So I suggest to only do the upload if a file is actually provided. By not changing the parameter set of the endpoint, this enforces users to explicitly set the file = None and does not break anything.
Any opinions on that approach are welcome.
Happy to hear from you @saleweaver

Hey, haven't worked with the feeds api for some time, there should be all Amazon provided endpoints in the file, plus an extra convenience method.
I'll have a look at the PR tonight, can you check if the other methods don't actually do what you want already?
Thanks!

Hey,
awesome - thanks for the fast response :+1:
I think the extra convenient method you are referring to is submit_feed, which combines create_feed_document and create_feed.
However, I am actually referring to one of those underlying functions (create_feed_document).
It has some convenience as well which makes the basic functionality (simply returning the presigned url) of the endpoint inaccessible, because of the file requirements.

I'll have a look at the PR tonight [...]

Thanks for looking into this :-)

Hey @saleweaver,
did you find some time to check this? If there is anything I can do to support you to make some progress here, please let me know ;-)
Kind regards
GRBurst


chore: working builds for Appveyor, use Node.js 8 and 10

Description

This PR tries to fix the failing Appveyor Windows builds.

Related: https://github.com/nodejs/node/issues/20325

Mh, I guess we can mark this as WIP until it is fixed in the dependencies and Node.js.


Flutter Driver support in AppCenter

Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

Closing as a duplicate of #67


Rework replicator tests to use environment-specific prefix

Use the dbPrefix of the environment, where the test runs as part of the dbPrefix for all replicator tests.
Also delete databases in replicator only, if the database is a backup of a db with the current prefix.
PG2#1075 started.

PG2#1075 is green


Fixed bug which would crash server

Only inspect res if there is no error. If there is an error, use LogCritical to panic and get captured by defer func which will establish a new apns.Client and try again.
NOTE: there probably are better ways to accomplish the same, but I just started coding in Go like 4 hours ago.

Thanks for the PR, this was fixed (indirectly) because of the refactor to push proxy.

Thanks. Testing out the latest release, there is still an issue: a notification is not sent if it gets the dreaded Client.Timeout error. The fix is to send again on this error since it indicates that the socket was closed by Apple.


Add a Submit Button to a Form

My issue is with:"Your submit button should only have the text "Submit"."
I have asked for help on the forum but not one can figure out why they code is being rejected.  I looked at others who have coded the cat app past my current spot and they have passed.
Love this website and I am grateful for this resource.
Challenge Add a Submit Button to a Form has an issue.
User Agent is: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/602.3.12 (KHTML, like Gecko) Version/10.0.2 Safari/602.3.12.
Please describe how to reproduce this issue, and include links to screenshots if possible.
My code:

<link href="https://fonts.googleapis.com/css?family=Lobster" rel="stylesheet" type="text/css">

CatPhotoApp

Click here for cat photos.

Things cats love:

cat nip

laser pointers

lasagna

Top 3 things cats hate:

flea treatment

thunder

other cats

<form action="/submit-cat-photo">
  <button type="submit">Submit</button>
  <input type="text" placeholder="cat photo URL">
</form>

Hi @wpr2004! Thank you for reporting this issue.
Your code works fine for me. Can you refresh the page and try again?
Do you see any errors in the console? I'm not sure how to open it in Safari, but I know it's Cmd + Shift + J in Firefox and Chrome.
Some other things to try:

Run in incognito mode
Try in a different browser

Please calrify more as requested.

The code ended up being fine.  I had to open in a new browser and create it there.  It worked in Chrome buy not on my safari browser.

I could pass the challenge on Safari as well. Personally I recommend Chrome over Safari, but maybe a reset and retry should work for you.


Implement and load test tensorflow serving request batching

https://research.googleblog.com/2016/02/running-your-models-in-production-with.html
https://github.com/tensorflow/serving/blob/master/tensorflow_serving/batching/README.md
https://github.com/tensorflow/serving/issues/96
http://stackoverflow.com/questions/35037360/how-to-deploy-and-serve-prediction-using-tensorflow-from-api
https://top-mob-games.us/game/running-your-models-in-production-with-tensorflow-serving/

we need to work around the following issue:  https://github.com/tensorflow/serving/issues/10
specifically, the part about UNKNOWN.  this happens at scale with TF Serving currently.

after upgrading to v5.0.0 - and making some resource-cleanup changes on our end - this appears to no longer be an issue under high load.


Modification of point cloud fusion method

First of all, I want to thank you for your great work and express my gratitude for your open-source code.
The paper mentioned that the baseline is MVSTER. After adding the epipolar transformer, the score on the DTU data set has been greatly improved. However, I found that the dypcd script used in your code is different from that of MVSTER. We know that even for the same depth map, using different fusion methods or fusion parameters, the results will be very different. In your ablation experiments, are you using the exact same fusion script as MVSTER? Would such an experiment be fair if different fusion scripts were used?
I tried deleting the ET module, and as long as I used the "updated" fusion script, I could still achieve similar scores. Here are some of my results. (ET means using fusion scripts from your code repository, and regular means using scripts from MVSTER and other similar methods.)

experiment
fusion method
accuracy
completeness
overall

MVSTER (original paper)
regular
0.350
0.276
0.313

MVSTER (ET paper)
regular
0.351
0.284
0.318

ET-MVSNet (ET paper)
ET
0.329
0.253
0.291

ET-MVSNet (my PC)
ET
0.3291
0.2567
0.2929

ET-MVSNet but delete ET
ET
0.3272
0.2590
0.2931

ET-MVSNet but delete ET
regular
0.3514
0.2832
0.3173

The changes to the fusion script played a big role in the model's performance on the DTU dataset.
I'm looking forward to your explanation about it, thank you!

Thank you for your interest in our work. First of all, fusion methods are not unified in the MVS field, and different MVS methods adopt different fusion methods. Regarding our revised fusion method, we have mentioned it in the supplementary material and released the code. Secondly, the baseline we use is not the original mvster, for example, we did not use its monocular depth estimation branch, and in our experiments, the ET module can improve the performance of depth and point cloud metrics on both dtu and TT datasets.

Thank you for your reply!
Yes, I understand that compared to MVSTER, your code has deleted the MDE module, which is why your baseline is worse than the original article. It is totally reasonable.
I know that different methods use different fusion strategies, such as gipupa, dypcd, etc. Have you applied this better fusion strategy to MVSTER in the ablation experiment?

Apologies for the late reply, as we have been occupied with other projects recently. Our experiments were conducted following the mvster approach. Different MVS methods employ various fusion techniques, as you mentioned, such as pcd, dypcd, gipuma,open3d,etc. In addition, there are also some customized fusion methods. Effectively fusing deep maps into point clouds is a significant challenge and key issue within the MVS field. It's hard to definitively assert which fusion method is universally best since performance can vary depending on the dataset and the specific methodology used. The fusion method we adopted is based on the phenomena observed in the depth output and point cloud generated by our method. If you are a researcher in the MVS field, you could consider trying out the fusion method we employed. Best wishes！

Thank you for your response, however, it does not address my concerns.
In the baseline results, it seems that you utilized a default fusion method, but for your proposed ET module, there appears to have been a switch to your updated fusion method. This change seems to have led to performance improvements being attributed to the ET module, which may not entirely reflect its true impact. The MVSTER method achieves comparable scores using the updated fusion method alone on DTU, without the need for the ET module. Therefore, attributing the performance gains solely to the Epipolar Transformer in the ablation study seems unjustified.


[WIP] MLTransform transform catalog snippets

Add snippets that will be used in the MLTransform catalog page.
These will be use in the md page coming in the next PR

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

We'll also need to add this to the table in the Overview page: https://beam.apache.org/documentation/transforms/python/overview/
And to the TOC: https://github.com/apache/beam/blob/3d501ee9dc208af2efef009daa98c49819b73ddc/website/www/site/layouts/partials/section-menu/en/documentation.html


Template links to missing file

In metric-template.md, it reads:

The first few sentences should match the description in the [metrics list](../activity-metrics-list.md).

That file no longer exists. Is there a new location for that information or can we offer a short description of the goal in that file?

Good catch @mbbroberg. I just put in a PR to fix this. Until then, the list is here:
https://github.com/chaoss/metrics/tree/master/activity-metrics

Hi @mbbroberg,
Thanks for reporting the error.
The list is now at: https://github.com/chaoss/metrics/#full-list-of-activity-metrics
You can change everything about the metric. The sentence just means we want it to match afterwards.

Thanks @GeorgLink! Closing as fixed by #175