Fortran wrapper for otter-serial with fibonnaci example

Hi Adam,
Ok, now I'm working from the correct base!
The current structure of the code is the following files:

include/otter/otter-serial.F90 - this is a set of preprocessor macros for using the defined functions. These need to be included in any fortran program using #include. Must be visible on the include path (i.e. -Iinclude/otter).
src/otter/events/serial/otter-serial.F90 - this contains the module to call the C code from Fortran. Rather than overlap the names, they have their own names in Fortran, which are fortran_otterBaseName. This file is added in the the add_library(otter-events-serial part of the CMakeLists, and the resulting output is a otter_serial.mod file which also needs to be visible on the -I path in any building from Fortran. It is used by use otter_serial from a Fortran Program/subroutine/etc.
src/otter/examples/otter-serial-fib.F90 - this contains the example fibonacci example, with no automatic compilation so far. On my machine (from the otter root directory), I can build it with: gfortran -std=f2003 src/otter/examples/otter-serial-fib.F90 -Iinclude/otter/ -Lbuild/lib/ -lotter-serial -Ibuild, and the output I get is as follows:

[achalk@glados otter]$ ./a.out 4
[i] [otterTraceInitialise_i          ] Otter environment variables:
[i] [otterTraceInitialise_i          ] host                           glados.dl.ac.uk
[i] [otterTraceInitialise_i          ] OTTER_TRACE_PATH               trace
[i] [otterTraceInitialise_i          ] OTTER_TRACE_NAME               otter_trace
[i] [otterTraceInitialise_i          ] OTTER_APPEND_HOSTNAME          No
Trace output path:             trace/otter_trace.3908223
[d] [otterTraceInitialise_i          ] src/otter/examples/otter-serial-fib.F90:47 in FORTRAN_FUNCTION
[d] [otterParallelBegin_i            ] src/otter/examples/otter-serial-fib.F90:50 in FORTRAN_FUNCTION
[d] [otterTaskSingleBegin_i          ] src/otter/examples/otter-serial-fib.F90:53 in FORTRAN_FUNCTION
[d] [otterTaskBegin_i                ] src/otter/examples/otter-serial-fib.F90:13 in FORTRAN_FUNCTION
[d] [otterTaskBegin_i                ] src/otter/examples/otter-serial-fib.F90:13 in FORTRAN_FUNCTION
[d] [otterTaskBegin_i                ] src/otter/examples/otter-serial-fib.F90:13 in FORTRAN_FUNCTION
[d] [otterTaskBegin_i                ] src/otter/examples/otter-serial-fib.F90:18 in FORTRAN_FUNCTION
[d] [otterSynchroniseChildTasks_i    ] src/otter/examples/otter-serial-fib.F90:23 in FORTRAN_FUNCTION
[d] [otterTaskBegin_i                ] src/otter/examples/otter-serial-fib.F90:18 in FORTRAN_FUNCTION
[d] [otterSynchroniseChildTasks_i    ] src/otter/examples/otter-serial-fib.F90:23 in FORTRAN_FUNCTION
[d] [otterTaskBegin_i                ] src/otter/examples/otter-serial-fib.F90:18 in FORTRAN_FUNCTION
[d] [otterTaskBegin_i                ] src/otter/examples/otter-serial-fib.F90:13 in FORTRAN_FUNCTION
[d] [otterTaskBegin_i                ] src/otter/examples/otter-serial-fib.F90:18 in FORTRAN_FUNCTION
[d] [otterSynchroniseChildTasks_i    ] src/otter/examples/otter-serial-fib.F90:23 in FORTRAN_FUNCTION
[d] [otterSynchroniseChildTasks_i    ] src/otter/examples/otter-serial-fib.F90:23 in FORTRAN_FUNCTION
           4           3
OTTER_TRACE_FOLDER=/home/achalk/otter/trace/otter_trace.3908223

I haven't inspected the trace/am not quite sure how to do that with the changes in Otter right now, so I'll focus on fixing up the commit and then we can double check the output.
Regarding the .mod file - I feel like there must be a way to have this appear in the include directory, but I am not good with CMake so I'm not sure how to do that.

I'm trying to analyse the output trace, but the OTF reader is breaking, which I'm assuming is due to the Frotran wrapper but not sure:
(otter_env) [achalk@glados otter-1]$ python3 -m otter trace/otter_trace.403377/otter_trace.403377.otf2 -o graph.dot
An unhandled python exception has occurred in an OTF2_GlobalDefReaderCallback:
Traceback (most recent call last):
  File "/home/achalk/.INSTALL/otf2/lib/python3.6/site-packages/_otf2/GlobalDefReaderCallbacks.py", line 46, in wrapper
    args = convert_args(*args)
  File "/home/achalk/.INSTALL/otf2/lib/python3.6/site-packages/_otf2/GlobalDefReaderCallbacks.py", line 203, in convert_string
    return ref, str_decode(name)
  File "/home/achalk/.INSTALL/otf2/lib/python3.6/site-packages/_otf2/Config.py", line 114, in str_decode
    return otf2_str.decode(encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd8 in position 2: unexpected end of data
Traceback (most recent call last):
  File "/home/achalk/.INSTALL/python-3.9.0/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/achalk/.INSTALL/python-3.9.0/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/achalk/pyotter/otter/__main__.py", line 14, in <module>
    with otf2.reader.open(args.anchorfile) as r:
  File "/home/achalk/.INSTALL/otf2/lib/python3.6/site-packages/otf2/reader.py", line 20, in open
    return closing(Reader(*args, **kwargs))
  File "/home/achalk/.INSTALL/otf2/lib/python3.6/site-packages/otf2/reader.py", line 52, in __init__
    _otf2.Reader_ReadAllGlobalDefinitions(self._handle, global_def_reader)
  File "/home/achalk/.INSTALL/otf2/lib/python3.6/site-packages/_otf2/Reader.py", line 222, in Reader_ReadAllGlobalDefinitions
    c_ReadAllGlobalDefinitions(reader, defReader, ctypes.byref(definitionsRead))
  File "/home/achalk/.INSTALL/otf2/lib/python3.6/site-packages/_otf2/ErrorCodes.py", line 166, in HandleErrorCode
    raise Error(errcode)
_otf2.ErrorCodes.Error: INTERRUPTED_BY_CALLBACK (92): Record reading interrupted by reader callback

Running otf2-print on the trace I get:
(otter_env) [achalk@glados otter-1]$ otf2-print trace/otter_trace.405166/otter_trace.405166.otf2

=== OTF2-PRINT ===
otf2-print: warning: out of order Region definition: "Parallel Region 0" <1>
otf2-print: warning: out of order Region definition: "implicit task 1" <2>
otf2-print: warning: out of order Region definition: "single_executor" <3>
otf2-print: warning: out of order Region definition: "explicit task 2" <4>
otf2-print: warning: out of order Region definition: "explicit task 3" <5>
otf2-print: warning: out of order Region definition: "explicit task 4" <6>
otf2-print: warning: out of order Region definition: "explicit task 5" <7>
otf2-print: warning: out of order Region definition: "taskwait" <8>
otf2-print: warning: out of order Region definition: "explicit task 6" <9>
otf2-print: warning: out of order Region definition: "taskwait" <10>
otf2-print: warning: out of order Region definition: "explicit task 7" <11>
otf2-print: warning: out of order Region definition: "explicit task 8" <12>
otf2-print: warning: out of order Region definition: "explicit task 9" <13>
otf2-print: warning: out of order Region definition: "taskwait" <14>
otf2-print: warning: out of order Region definition: "taskwait" <15>
otf2-print: warning: out of order Region definition: "initial task 0" <0>

I think the data in the strings is not being copied across/saved as we suspected it may not be:
ENTER                                      0<PHONE_NUMBER>808458  Region: "initial task 0" <0>
                                                                   ADDITIONAL ATTRIBUTES: ("cpu" <13>; INT32; 9), ("encountering_task_id" <3>; UINT64;<PHONE_NUMBER>3709551615), ("region_type" <24>; STRING; "initial_task" <84>), ("event_type" <12>; STRING; "task_enter" <77>), ("endpoint" <14>; STRING; "enter" <81>), ("unique_id" <0>; UINT64; 0), ("task_type" <15>; STRING; "initial_task" <84>), ("task_flags" <10>; UINT32; 1), ("parent_task_id" <9>; UINT64;<PHONE_NUMBER>3709551615), ("parent_task_type" <16>; STRING; "" <0>), ("task_has_dependences" <11>; UINT8; 0), ("task_is_undeferred" <17>; UINT8; 0), ("task_is_untied" <18>; UINT8; 0), ("task_is_final" <19>; UINT8; 0), ("task_is_mergeable" <20>; UINT8; 0), ("task_is_merged" <21>; UINT8; 0), ("prior_task_status" <25>; STRING; "" <0>), ("source_line_number" <26>; UINT32; 47), ("source_file_name" <27>; STRING; "P�'" <118>), ("source_func_name" <28>; STRING; "�'" <119>)

The source_line_number value of 47 is correct, but the source_file_name and source_func_name should be src/otter/examples/otter-serial-fib.F90 and FORTRAN_FUNCTION respectively (which is what Otter prints during the run), but are not valid results. I think we'll need to change the code to use strcpy in C instead of just storing the pointers unfortunately - I think this may be due to requiring me to add \0 onto the Fortran strings to be usable from C - resulting in local variables which are going out of scope.

I will also look to get transformations into PSyclone so we can use these, can I double check the use-case for a fewof the otter-serial functions?

otterTaskSingleBegin_i - Is this at the start of a single region (or similar?).
otterLoopBegin_i - Is this the start of a parallelisable loop? I.e. a Loop that would/could be parallelised using something like !$DIR omp do or #pragma omp for
otterLoopIterationBegin_i - not sure on this one
otterSynchroniseChildTasks_i - equivalent to taskwait? I'm assuming depend style dependencies are not yet supported by Otter?
otterSynchroniseDescendantTasksBegin_i - How does this differ from 4?

Aidan

Hi Aidan, thanks so much for this PR, it looks great! What you've said about the failure to read the trace makes sense, I agree that the error is probably due to the strings going out of scope before they are written to the trace. I'm working on an update that will copy the string contents instead of relying on the lifetime of the pointer passed by the caller.
Here's my to-do list to integrate your contributions and make sure they're working:

[ ] open an issue re. usage of const char* which will be resolved when the string contents are copied and the caller is free to release the resource immediately. This will be developed on a separate branch.
[ ] merge that update into this feature branch to verify this produces a readable trace with correct string definitions
[ ] integrate fortran wrapper into cmake build system
[ ] merge above into dev

Sound good to you?
Re. your questions on the API:

1. otterTaskSingleBegin_i - Is this at the start of a single region (or similar?).

Yes, this is supposed to mimic the effect of a #pragma omp single - after chatting with Tobias about this I'm probably going to remove it as it's too specific to OpenMP.

2. otterLoopBegin_i - Is this the start of a parallelisable loop? I.e. a Loop that would/could be parallelised using something like `!$DIR omp do` or `#pragma omp for`

That's right.

3. `otterLoopIterationBegin_i` - not sure on this one

This indicates the beginning of a single loop iteration e.g.
otterLoopBegin();
for(...)
{
    otterLoopIterationBegin();
    // ...
    otterLoopIterationEnd();
}
otterLoopEnd();

4. `otterSynchroniseChildTasks_i` - equivalent to taskwait? I'm assuming `depend` style dependencies are not yet supported by Otter?

Correct x2.

5. `otterSynchroniseDescendantTasksBegin_i` - How does this differ from 4?

This mimics a #pragma omp taskgroup which synchronises all descendant tasks within a structured block - this begin/end pair will be replaced with something like a otterSynchroniseDescendantTasks entrypoint.
There's quite a bit of documentation missing from otter-serial at the moment, expanding that is on my to-do list...
Also, I'm not sure whether I've made this clear, but I should say explicitly that otter-serial is single-threaded only for the moment - it is not thread-safe as it was written as a prototype event source to be used as direct instrumentation in a single-threaded target application. Having a multi-threaded version is the long-term goal but getting there will take some time.

That all looks good to me, thanks!
Once I get the go-ahead from Rupert, I'll start working on a proof-of-concept set of functionality either in PSyclone or a dependent library to get Otter able to work with the PSyclone benchmark.

Hi Aidan,
I've implemented a solution which doesn't depend on the lifetime of the callee's file and func arguments (see https://github.com/Otter-Taskification/otter/issues/4#issuecomment-1130561062).
Is it possible for you to merge dev-value-registry-template-with-tests into your feature branch to check whether this resolves the issue in your fortran fibonacci demonstrator? Let me know if this doesn't work and we'll figure out another way to check that this solution works.

Hi Adam,
I just merged this in and it seems to work!
I ran the example, and then ran the resulting trace through pyotter and can generate a graph:

Also checking otf2-print, I have:
"source_line_number" <26>; UINT32; 47), ("source_file_name" <27>; STRING; "src/otter/examples/otter-serial-fib.F90" <118>), ("source_func_name" <28>; STRING; "FORTRAN_FUNCTION" <119>)

which looks like what we'd expect, so it all seems to be working correctly.
I guess the next step is to work out how to resolve the CMake "issue".
Do the examples get built automatically in some way as well? I've not added the example into the build system anywhere yet.

Fantastic! So glad it works on the first attempt :fireworks: :smile:
There is a cmake option which includes the examples in the build, -DWITH_EXAMPLES=ON which includes the src/examples/ subdirectory in the build. There's a CMakeLists.txt file in there which generates the examples. You can try adding your fibonacci example in there so it gets built automatically - hopefully you can follow the other examples, but let me know if this causes any headaches.

As an aside, I'm not sure whether you've seen the latest version of pyotter which can output HTML reports which include the graph as an svg - check out the --report option:
❯ python3 -m otter --help
usage: python3 -m otter [-h] [-o OUTPUT] [-r REPORT] [-f] [-v] [-i] [-ns] [-d]
                        [--loglevel {DEBUG,INFO,WARN,ERROR}] [--logdir LOGDIR]
                        anchorfile

Convert an Otter OTF2 trace archive to its execution graph representation

positional arguments:
  anchorfile            OTF2 anchor file

options:
  -h, --help            show this help message and exit
  -o OUTPUT, --output OUTPUT
                        output file (default: None)
  -r REPORT, --report REPORT
                        report path (default: None)
  -f, --force           force overwrite if files already exist (default: False)
  -v, --verbose         print chunks as they are generated (default: False)
  -i, --interact        drop to an interactive shell upon completion (default: False)
  -ns, --no-style       do not apply any styling to the graph nodes (default: False)
  -d, --debug           step through the code with pdb.set_trace() (default: False)
  --loglevel {DEBUG,INFO,WARN,ERROR}
                        log level (default: WARN)
  --logdir LOGDIR       log directory (default: .)

Ok - I have a draft implementation of the stuff in PSyclone for the code as it exists in this branch (https://github.com/stfc/PSyclone/pull/1746). I think from the issues that perhaps some of the otter code has changed though, is there a brief summary of what I need to update for this branch anywhere? I'll try to take a look of the CMake stuff and then hopefully can get a NemoLite2D version up and running soon too.

I'm unclear a bit on the CMake stuff. I tried adding set( CMAKE_Fortran_MODULE_DIRECTORY ${PROJECT_BINARY_DIR}/include) (as my googling suggested, and as the CMake docs seem to suggest but that doesn't seem to be working.

Ok - I have a draft implementation of the stuff in PSyclone for the code as it exists in this branch (stfc/PSyclone#1746). I think from the issues that perhaps some of the otter code has changed though, is there a brief summary of what I need to update for this branch anywhere? I'll try to take a look of the CMake stuff and then hopefully can get a NemoLite2D version up and running soon too.

I haven't made any changes to the Otter API on this branch, although there are some changes in the pipeline. What issues are you encountering? I'm not sure I follow the conversation in (stfc/PSyclone#1746).

I'm unclear a bit on the CMake stuff. I tried adding set( CMAKE_Fortran_MODULE_DIRECTORY ${PROJECT_BINARY_DIR}/include) (as my googling suggested, and as the CMake docs seem to suggest but that doesn't seem to be working.

Can you share the exact commands you'd use to compile the Fortran module and link the Fortran example with that module and the Otter runtime? I'll try integrating that into CMake myself.

I looked in the report and I have outputs like:
alg.F90\x00, main\x00, 48
I think there may be some errors in my script too so I'll try to fix those while I investigate the weird "\x00". Its a bit odd that the null character is appearing now - maybe I don't need to add it in Fortran any more with strcpy? I will see.

How is the otter-serial.mod file generated? I'll add a build target for that file so that it can be installed somewhere in the build tree

Ok, I think I understand what your compilation command expects :

The file otter-serial.F90 to be in /home/achalk/otter-1/build which it will #include
The compiled fortran module otter-serial.mod to be in /home/achalk/otter-1/build/lib which it then links with -lotter-serial

If that's so, I need to add build targets for include/otter/otter-serial.F90 and src/otter/events/serial/otter-serial.F90 to make them available to other projects. This is easy to do for the .F90 header, but I need to know how to compile src/otter/events/serial/otter-serial.F90 into a .mod file

How is the otter_serial.mod file generated? I'll add a build target for that file so that it can be installed somewhere in the build tree

otter_serial.mod is already created as part of:
# Serial event source
add_library(otter-events-serial OBJECT
    ${CMAKE_CURRENT_SOURCE_DIR}/src/otter/events/serial/otter-serial.c
    ${CMAKE_CURRENT_SOURCE_DIR}/src/otter/events/serial/otter-serial.F90
    ${CMAKE_CURRENT_SOURCE_DIR}/src/otter/types/otter-structs.c
)
target_include_directories(otter-events-serial PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/include)
target_compile_options(otter-events-serial BEFORE PRIVATE ${CMAKE_C_FLAGS})
target_compile_definitions(otter-events-serial PRIVATE DEBUG_LEVEL=${OTTER_DEBUG_CORE})
set_property(TARGET otter-events-serial PROPERTY POSITION_INDEPENDENT_CODE ON)

From make verbose I get this command:
/home/achalk/.INSTALL/llvm/bin/f18 -DDEBUG_LEVEL=0 -I/home/achalk/otter-1/include -I/home/achalk/.INSTALL/otf2/include -fPIC -c /home/achalk/otter-1/src/otter/events/serial/otter-serial.F90 -o CMakeFiles/otter-events-serial.dir/src/otter/events/serial/otter-serial.F90.o

I pushed the updated CMakeLists.txt to your branch. It now does the following:

Compiles src/otter/events/serial/otter-serial.F90 into a static library
Installs the static library to lib/
Installs the include/otter/otter-serial.F90 header to include/otter

It looks like you're building Otter and then compiling against the build artefacts in the build tree. I'd instead recommend installing Otter somewhere and then building against the installed components. You can do this with a cmake command like:
cmake -DCMAKE_INSTALL_PREFIX=/home/adam/tmp ../ && cmake --build . && cmake --install .

This will install the library targets to /home/adam/tmp/lib and the public headers (as well as the .mod file) to /home/adam/tmp/include/otter. I think this should satisfy your build requirements, but let me know if there are any difficulties with this approach.

What is libotter-serial-f.a? I wasn't aware of this existing (or linking to it, the Fortran code doesn't "do anything" so it just links C library, its solely needed as Fortran can only use functions defined in modules).
I was just linking to the build dir as I didn't want to install it when I was rebuilding so regularly, but I can do to check if everything ends up in the appropriate place.

What is libotter-serial-f.a? I wasn't aware of this existing (or linking to it, the Fortran code doesn't "do anything" so it just links C library, its solely needed as Fortran can only use functions defined in modules). I was just linking to the build dir as I didn't want to install it when I was rebuilding so regularly, but I can do to check if everything ends up in the appropriate place.

It's the source file src/otter/events/serial/otter-serial.F90 compiled to a static library (just the .o file in an archive). I'm assuming that the code in there defines the functions declared in the .mod file, and that when you want to compile a fortran executable with Otter you need to link the definitions in src/otter/events/serial/otter-serial.F90 so that the fortran code can call the Otter functions through those fortran functions. I'm not familiar with the fortran compilation process, but I think you'd use it something like this (assuming you've told cmake to install Otter to OTTER_INSTALL):
gfortran [...] -o nemolite2d.exe  boundary_conditions_mod.o continuity_mod.o momentum_mod.o time_update_mod.o infrastructure_mod.o psy.o alg.o ../../common/nemolite2d_common.a libdl_timer.a lib_fd.a ${OTTER_INSTALL}/lib/ -fopenmp -L${OTTER_INSTALL}/lib -lotter-serial -lotter-serial-f -I${OTTER_INSTALL}/include

It now occurs to me that this addition you made had the effect of compiling the Fortran code directly into the otter-serial library didn't it? Does this then mean that you only need to #include the `.mod' file to make use of these Fortran entrypoints?

I've restored this change and removed the libotter-serial-f.a target which I'd added. If you now build & install Otter, you'll get these components:
$ ls lib/ include/otter/
include/otter/:
total 8.0K
-rw-r--r-- 1 adam adam 2.2K Jun  9 13:58 otter-serial.h
-rw-r--r-- 1 adam adam  938 Jun  9 17:22 otter_serial.mod

lib/:
total 728K
-rwxr-xr-x 1 adam adam 383K Jun  9 17:22 libotter-ompt.so
-rwxr-xr-x 1 adam adam 341K Jun  9 17:22 libotter-serial.so

libotter-serial.so includes the Fortran entrypoints, so I presume you can just include otter/otter_serial.mod and link -lotter-serial with your Fortran app to use them, right?
$ nm lib/libotter-serial.so | grep _MOD_
000000000000d70a T __otter_serial_MOD_fortran_otterloopbegin_i
000000000000d6fe T __otter_serial_MOD_fortran_otterloopend
000000000000d623 T __otter_serial_MOD_fortran_otterloopiterationbegin_i
000000000000d617 T __otter_serial_MOD_fortran_otterloopiterationend
000000000000d9bf T __otter_serial_MOD_fortran_otterparallelbegin_i
000000000000d9b3 T __otter_serial_MOD_fortran_otterparallelend
000000000000d53c T __otter_serial_MOD_fortran_ottersynchronisechildtasks_i
000000000000d461 T __otter_serial_MOD_fortran_ottersynchronisedescendanttasksbegin_i
000000000000d455 T __otter_serial_MOD_fortran_ottersynchronisedescendanttasksend
000000000000d8d8 T __otter_serial_MOD_fortran_ottertaskbegin_i
000000000000d8cc T __otter_serial_MOD_fortran_ottertaskend
000000000000d7f1 T __otter_serial_MOD_fortran_ottertasksinglebegin_i
000000000000d7e5 T __otter_serial_MOD_fortran_ottertasksingleend
000000000000da9a T __otter_serial_MOD_fortran_ottertracefinalise
000000000000daa6 T __otter_serial_MOD_fortran_ottertraceinitialise_i
000000000000d449 T __otter_serial_MOD_fortran_ottertracestart
000000000000d43d T __otter_serial_MOD_fortran_ottertracestop

Yes, for the most part. You can use otter-serial and make use of the Fortran entry points while linking to libotter-serial.so. The .F90 file in the include directory contains macros to use otter which would be #included to use the macros (as macro's are not imported as part of a module in Fortran).

It sounds like this build process should produce what you need, then. Please can you try building (and maybe installing) Otter to see whether it works and let me know how you get on.

Hi Adam, I built the library correct, but make install doesn't seem to work for me:
Install the project...
-- Install configuration: "Debug"
-- Installing: /home/achalk/.INSTALL/otter/lib/libotter-ompt.so
-- Installing: /home/achalk/.INSTALL/otter/lib/libotter-serial.so
-- Installing: /home/achalk/.INSTALL/otter/include/otter/otter-serial.h
CMake Error at cmake_install.cmake:88 (file):
  file INSTALL cannot find
  "/home/achalk/otter-1/build/lib/fmod/otter_serial.mod": No such file or
  directory.

make: *** [Makefile:83: install] Error

For reference, this is the current structure of the directories after make:
[achalk@glados build]$ ls
CMakeCache.txt  CMakeFiles  Makefile  cmake_install.cmake  lib  otter_serial.mod

I'm assuming this is a similar issue to what I had with set(CMAKE_Fortran_MODULE_DIRECTORY ${CMAKE_BINARY_DIR}/lib/fmod) seemingly being ignored.

Hi Adam,
I have this now setup with Otter, but I'm failing to be able to generate a report now with pyotter.
I had assumed otterSynchroniseChildTasks_i was approximately equal to #omp taskwait, so I designed the system to make use of that. However, I wonder if something has changed, as while this trace works and otter doesn't complain, pyotter fails to generate a report.
Snippet of code:
      DO j_out_var = va%internal%ystart, va%internal%ystop, 32
        j_el_inner = MIN(j_out_var + (32 - 1), va%internal%ystop)
        CALL fortran_otterTaskBegin_i(__FILE__, 'invoke_0', __LINE__)
        DO j = j_out_var, j_el_inner, 1
          DO i = va%internal%xstart, va%internal%xstop, 1
            CALL momentum_v_code(i, j, va%data, un%data, vn%data, hu%data, hv%data, ht%data, ssha_v%data, sshn_t%data, &
&sshn_u%data, sshn_v%data, un%grid%tmask, un%grid%dx_v, un%grid%dx_t, un%grid%dy_u, un%grid%dy_v, un%grid%dy_t, un%grid%area_v, &
&un%grid%gphiv)
          END DO
        END DO
        CALL fortran_otterTaskEnd
      END DO
      CALL fortran_otterSynchroniseChildTasks_i(__FILE__, 'invoke_0', __LINE__)

Snipper of code run otter debug info:
[d] [otterTaskBegin_i                ] psy.F90:139 in invoke_0
[d] [otterTaskBegin_i                ] psy.F90:139 in invoke_0
[d] [otterTaskBegin_i                ] psy.F90:139 in invoke_0
[d] [otterSynchroniseChildTasks_i    ] psy.F90:147 in invoke_0
[d] [otterTaskBegin_i                ] psy.F90:150 in invoke_0
[d] [otterTaskBegin_i                ] psy.F90:150 in invoke_0
[d] [otterTaskBegin_i                ] psy.F90:150 in invoke_0
[d] [otterTaskBegin_i                ] psy.F90:150 in invoke_0

But pyotter fails as I think it expects there should be 2 events for this node?
(otter_env) [achalk@glados ~]$ python3 -m otter /home/achalk/PSycloneBenchClean/benchmarks/nemo/nemolite2d/psykal/nemolite2d_otter/trace/otter_trace.388054/otter_trace.388
054.otf2  -r report
Traceback (most recent call last):
  File "/home/achalk/.INSTALL/python-3.9.0/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/achalk/.INSTALL/python-3.9.0/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/achalk/pyotter/otter/__main__.py", line 213, in <module>
    for event in vertex['event']: # now guaranteed to have exactly 2 event instances per vertex (tw-begin+end)
TypeError: 'SyncBegin' object is not iterable

Do i need an earlier commit/version of Pyotter or has something changed that prevents this from working?

This is a very curious bug as everything seems to build & install fine for me.
What version of cmake do you have?
Can you check that you get the bug when you re-trace exactly these steps:
$ rm -rf /home/achalk/.INSTALL/otter # to clear out any existing installed Otter artefacts
$ rm -rf /home/achalk/otter-1/build/* # to clear existing build files
$ cd /home/achalk/otter-1/build
$ cmake -DCMAKE_INSTALL_PREFIX=/home/achalk/.INCLUDE/otter ../
$ cmake --build .
$ cmake --install .

When I do this on my machine, my build dir looks like this afterwards, and this is where otter_serial.mod is found:
$ ls -l
total 68K
drwxr-xr-x 11 adam adam 4.0K Jun 10 16:16 CMakeFiles
drwxr-xr-x  3 adam adam 4.0K Jun 10 16:16 lib
-rw-r--r--  1 adam adam  16K Jun 10 16:14 CMakeCache.txt
-rw-r--r--  1 adam adam 4.0K Jun 10 16:14 cmake_install.cmake
-rw-r--r--  1 adam adam  163 Jun 10 16:16 install_manifest.txt
-rw-r--r--  1 adam adam  35K Jun 10 16:14 Makefile
$ find . -name "*.mod"
./lib/fmod/otter_serial.mod

The install dir looks like this for me:
$ find /home/adam/tmp -type f
/home/adam/tmp/lib/libotter-serial.so
/home/adam/tmp/lib/libotter-ompt.so
/home/adam/tmp/include/otter/otter_serial.mod
/home/adam/tmp/include/otter/otter-serial.h

For the python error, can you check that you are using the latest version of pyotter from https://github.com/Otter-Taskification/pyotter/commits/dev

Hi Adam,
I just did redid the steps as you said (Only difference is I need to specify -DOTF2_INSTALL_DIR) and I'm still having the same problem.
[achalk@glados build]$ cmake --version
cmake version 3.20.2

git log shows this as the last commit
commit 449633c64bcff4f60b4bdb6ee7bedd36e15ae9b2 (HEAD -> dev-fortran-otter-serial-interface, origin/dev-fortran-otter-serial-interface)
Author: Adam<EMAIL_ADDRESS>Date:   Thu Jun 9 17:30:35 2022 +0100

    Restore src/otter/events/serial/otter-serial.F90 in the otter-events-serial target, remove spurious otter-serial-f target

otter_serial.mod is still in the build/. directory:
[achalk@glados build]$ find . -name "*.mod"
./otter_serial.mod

Re: Pyotter I am at 40de2aa6bc9ad91c7cc343a4050c26459a744f8d which matches up with the HEAD of the dev branch yeah. This could be an error in my code maybe - I've attached the trace output to this reply.
trace.tar.gz

What a strange CMake bug. What Fortran compiler version are you using? I have GNU Fortran (GCC) 12.1.0 and I could imagine that an older version might not output .mod files to the expected location...
Re. the python error, I'll have to take a look on Monday. The traceback suggests that there's a bug in my code.

Ah, I'm using llvm's 11 - f18 perhaps thats the issue.
Looks like if I use gfortran from gcc to compile the Fortran code and LLVM to compile C code (and provide OMPT) it compiles and builds successfully, so I guess this is a flang issue maybe.
No worries.

I believe I've fixed the bug with pyotter with the latest commits: https://github.com/Otter-Taskification/pyotter/tree/dev - I've now been able to get a sensible graph out of the trace you provided. Let me know if it also works for you.

Yeah, seems to work for me too.

So for the testcase:
https://github.com/stfc/PSyclone - branch: #1744_otter_transformations - install with -e with pip.
https://github.com/stfc/PSycloneBench - branch: psykal_otter
In PSycloneBench:
source compiler_setup/gnu.sh
cd benchmarks/nemo/nemolite2d/psykal
make nemolite2d_otter
This will eventually fail, but will make all the files in nemolite2d_otter
It creates a Makefile and alg.F90 which are not valid, and need replacing by the Makefile.bak and alg.F90.bak in the current directory:
cp Makefile.bak nemolite2d_otter/Makefile
cp alg.F90.bak nemolite2d_otter/alg.F90
cd nemolite2d_otter && make
Final thing you can do (and I probably would for the demo) is adjust the namelist file, changing jpiglo and jpjglo to be smaller (I used 256), and reducing nitend (I used 10).

Thanks, I'll have a go at building the test case with the Otter additions in this branch. Once that works I think we can merge this PR into Otter.


lucene version 2.4

hi,
first just amazing work..
I would like to use an old lucene index and not with java, so I was just trying to load my index but have faced many issue while reading it despite lucene index structure documentation.
my try https://github.com/darul75/moo/blob/master/src/solr.go a shame.. ;)
can you drive me a little I was looking at your code and codex package but I can not figure out how to change segmentsInfos.go to handle that kind of index version, mine is -7 (2.4 version)

Hi Julien, it's hard to say,as you may know, lucene is gradually evolving its index structure. I don't have much knowkedge of the old format, either. Let me have a check and get back to later.


Remove VTK dependency

Scalismo currently relies on the VTK library for many computations. This dependency is the main reason that Scalismo does not easily run in a container environment.
To make that work, VTK itself also needs to be installed in the container (which makes the containers a lot larger).
The VTK library is also often far from fast - probably due to converting to/from the vtk format.
I would therefore suggest to eventually have the VTK dependency removed from Scalismo.
This would have implications on:

Mesh decimation
Mesh I/O
Tetrahedral mesh

And a lot of:

Image I/O and operations

Maybe some of the things are not needed to be supported in the future, which is why I opened this ticket as a discussion as well to hear more view points on this "problem".

@madsendennis Thanks a lot for all your efforts for removing vtk. I think with what you have provided so far we should bring us very close. I don't worry so much about the image part. We have a pure java nifti reader/writer. Image operations such as Gaussian smoothing are, as far as I know, not often used. We could just move them into a separate module. The module could, in essence just provide a converter between scalismo and vtk objects. In that way, a user could use all of vtk from Scalismo, without us having to invest a lot of effort in providing a tight integration.

I discussed another possible move with @Andreas-Forster .
To move everything (non-vtk) to a new library scalismo-core. Scalismo would then depend on this library + still include all the current VTK functionality. In this way, normal users will not see a difference in using scalismo, but we would then have a scalismo-core with a much smaller footprint, more appropriate for production use.
The main disadvantage as I see is that every update to scalismo-core will also trigger an update to scalismo and then scalismo-ui and scalismo-faces.

This is another good possibility. Maybe its even just a difference in naming. The difficulty with the transitive updates seems manageable.
I have a preference for making Scalismo itself simpler rather than carving out the core. After the work you have now contributed, all essential functionality that are needed to write full shape modelling applications is there. If we name this Scalismo, we can tell users that they can get more functionality by combining it with the VTK library (at the expense of a couple of hundred MB footprint and reduced portability).  This feels to me different than offering a library called Scalismo, which has a footprint of 300 MB, but you can reduce it to a few MB by using only its core. I guess at this level it's just psychology.
It might however, make a practical difference in terms of maintenance. Maintaining Scalismo (withouth VTK and other native libraries) is a very manageable task. Adding natives adds a lot of overhead. If at some point we feel overwhelmed by the complexity of maintaining VTK libraries, we can independently stop supporting it, while still providing full maintenance for Scalismo. Especially now that web alternatives to scalismo-ui start to emerge, this could be a likely trajectory in the next few years.
@Andreas-Forster What do you think?

After discussing the "problem" offline with @marcelluethi  and @Andreas-Forster , the path forward is to try and make scalismo into a lightweight native scala library.
All VTK dependent functionality will be moved to another repo scalismo-vtk. The main implication is that support for all .vtk files is dropped in scalismo and moved to scalismo-vtk. This means that until a substitute format is found, formats such as: ScalarMeshField, LineMesh, TetrahedralMesh, ScalarVolumeMeshField and .VTK reader/writer for e.g. TriangleMesh is not included in the scalismo library.

In the future scalismo will not contain any vtk components. These components have been moved to scalismo-vtk, which depends on scalismo. Furthermore, all Active Shape modelling (ASM) aspects have been moved to scalismo-asm (as it also had some vtk dependencies and is rarely used).
A release branch has been created with native components merged in and vtk removed: https://github.com/unibas-gravis/scalismo/tree/release-1.0
When this branch is stable, it will be the foundation for a 1.0.0 scalismo release.


State of project?

Hope you are doing fine! Any idea whether you will continue to maintain the project in the future? It's so valueable!

Okay, thanks a lot for your feedback and good luck with your graduation 😊


Http handler

Work Done So far :

noVNC integrated and tested for KVM environment.
shifted entire consoleProxy server to jetty server on port 80.
added SSL support not tested currently

@sachinnitw1317 can you resolve conflicts and make sure your branch cleanly builds. I see some issues with some pom files.


Android Studio: Problem with cmake/host and Protobuf of selected version (3.0.0-p1)

I've read Brief overview section and do understand basic concepts. Yes
I've read F.A.Q. section and there is no solution to my problem there. Yes
I've read Code of Conduct, I promise to be polite and will do my best at being constructive. Yes
I've read Reporting bugs section carefully. Yes
I've checked that all the hunter_add_package/find_package API used by me in example is the same as in documentation. Yes
I'm using latest Hunter URL/SHA1. Yes

I've created SSCCE GitHub repository to reproduce the issue: https://github.com/bezigon/android-studio-with-hunter

Here is the log until first error reported by Hunter, option HUNTER_STATUS_DEBUG is ON:

Terminal Saved Output.log

I've checked that the first error in logs IS NOT external.build.failed. Yes

I'm building on OSX in Android Studio/Gradle.
I'm using brewed CMake release (brew install cmake)
Android Studio version: 3.2.1
Gradle version: 5.0
CMake version: 3.13.2

I'm not using toolchain

I'm using next command line on generate step:
gradle assembleDebug --info

Problem with cmake/host and Protobuf

What is the problem exactly? I don't see any Hunter/CMake/build errors

In fact, the problem is that the process does not go beyond after specific line Scanning dependencies of target Protobuf-Release in the near end of the log below.
[== "CMake Server" ==[

{"cookie":"","inReplyTo":"configure","message":"[hunter] \n************************* !!! WARNING !!! *************************\n\nYou're using experimental feature. This may not work well\nand can be removed anytime. It's a workaround for existing\nfeature request:\n\nhttps://github.com/ruslo/hunter/issues/495\n\n*******************************************************************","type":"message"}

]== "CMake Server" ==]

[hunter]
************************* !!! WARNING !!! *************************

You're using experimental feature. This may not work well
and can be removed anytime. It's a workaround for existing
feature request:

https://github.com/ruslo/hunter/issues/495

*******************************************************************
[hunter]
************************* !!! WARNING !!! *************************

You're using experimental feature. This may not work well
and can be removed anytime. It's a workaround for existing
feature request:

https://github.com/ruslo/hunter/issues/495

*******************************************************************

[== "CMake Server" ==[

{"cookie":"","inReplyTo":"configure","message":"[hunter] Building host project: /Users/bezigon/Documents/GitHub/android-studio-with-hunter-bezigon/cmake/host","type":"message"}

]== "CMake Server" ==]

[hunter] Building host project: /Users/bezigon/Documents/GitHub/android-studio-with-hunter-bezigon/cmake/host
[hunter] Building host project: /Users/bezigon/Documents/GitHub/android-studio-with-hunter-bezigon/cmake/host

[== "CMake Server" ==[

{"cookie":"","inReplyTo":"configure","message":"[hunter *** DEBUG *** 2018-12-19T12:22:55] Using default host generator","type":"message"}

]== "CMake Server" ==]

[hunter *** DEBUG *** 2018-12-19T12:22:55] Using default host generator
[hunter *** DEBUG *** 2018-12-19T12:22:55] Using default host generator
-- [hunter] Calculating Toolchain-SHA1

-- [hunter] Calculating Toolchain-SHA1
-- [hunter] Calculating Config-SHA1

-- [hunter] Calculating Config-SHA1
-- [hunter] HUNTER_ROOT: /Users/bezigon/.hunter

-- [hunter] HUNTER_ROOT: /Users/bezigon/.hunter
-- [hunter] [ Hunter-ID: c7b6099 | Toolchain-ID: 814ec5f | Config-ID: c9a4aa4 ]

-- [hunter] [ Hunter-ID: c7b6099 | Toolchain-ID: 814ec5f | Config-ID: c9a4aa4 ]
-- [hunter] PROTOBUF_ROOT: /Users/bezigon/.hunter/_Base/c7b6099/814ec5f/c9a4aa4/Install (ver.: 3.0.0-p1)

-- [hunter] PROTOBUF_ROOT: /Users/bezigon/.hunter/_Base/c7b6099/814ec5f/c9a4aa4/Install (ver.: 3.0.0-p1)
-- [hunter] Building Protobuf

-- [hunter] Building Protobuf
loading initial cache file /Users/bezigon/.hunter/_Base/c7b6099/814ec5f/c9a4aa4/cache.cmake

loading initial cache file /Users/bezigon/.hunter/_Base/c7b6099/814ec5f/c9a4aa4/cache.cmake
loading initial cache file /Users/bezigon/.hunter/_Base/c7b6099/814ec5f/c9a4aa4/Build/Protobuf/args.cmake

loading initial cache file /Users/bezigon/.hunter/_Base/c7b6099/814ec5f/c9a4aa4/Build/Protobuf/args.cmake
-- The C compiler identification is AppleClang <IP_ADDRESS>01145

-- The C compiler identification is AppleClang <IP_ADDRESS>01145
-- The CXX compiler identification is AppleClang <IP_ADDRESS>01145

-- The CXX compiler identification is AppleClang <IP_ADDRESS>01145
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc

-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- works

-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- works
-- Detecting C compile features

-- Detecting C compile features
-- Detecting C compile features - done

-- Detecting C compile features - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++

-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works

-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works
-- Detecting CXX compile features

-- Detecting CXX compile features
-- Detecting CXX compile features - done

-- Detecting CXX compile features - done
-- Configuring done

-- Configuring done
-- Generating done

-- Generating done
-- Build files have been written to: /Users/bezigon/.hunter/_Base/c7b6099/814ec5f/c9a4aa4/Build/Protobuf/Build

-- Build files have been written to: /Users/bezigon/.hunter/_Base/c7b6099/814ec5f/c9a4aa4/Build/Protobuf/Build
Scanning dependencies of target Protobuf-Release

Scanning dependencies of target Protobuf-Release

> Task :app:generateJsonModelDebug FAILED
:app:generateJsonModelDebug (Thread[Execution worker for ':',5,main]) completed. Took 23.305 secs.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:generateJsonModelDebug'.
> Conversion = ']'

* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.0/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 25s
16 actionable tasks: 1 executed, 15 up-to-date
bezigon@bezigons-iMac ~/D/G/a/android-studio>

Of course, I deleted the ~/.hunter/_Base directory for the purity of the experiment.

In the project protobuf-toolchains-mix the assembly on pure CMake/OS X of selected version of Protobuf (3.0.0-p1) passes without problems.

In fact, the problem is that the process does not go beyond after specific line Scanning dependencies of target Protobuf-Release in the near end of the log below.

It doesn't mean that problem is in Hunter, it may be a Gradle + CMake issue.
Try to use workaround from documentation:

https://docs.hunter.sh/en/latest/faq/android-studio.html#example

...
In this case you can add return() command right after first hunter_add_package call
...

Another example: https://github.com/elucideye/drishti#android-studio-workarounds

Already did that feature using return(), but no success.
[0/1] /usr/local/Cellar/cmake/3.13.2/bin/cmake -S/Users/bezigon/Documents/GitHub/android-studio-with-hunter-bezigon -B/Users/bezigon/Documents/GitHub/android-studio-with-hunter-bezigon/android-studio/app/.externalNativeBuild/cmake/debug/arm64-v8a
-- [hunter *** DEBUG *** 2018-12-19T13:02:12] HUNTER_ROOT set using HOME environment variable
-- [hunter *** DEBUG *** 2018-12-19T13:02:12] HUNTER_ROOT: /Users/bezigon/.hunter
-- [hunter *** DEBUG *** 2018-12-19T13:02:12] Settings (initialize):
-- [hunter *** DEBUG *** 2018-12-19T13:02:12]   HunterGate done (NO)
-- [hunter *** DEBUG *** 2018-12-19T13:02:12]   Cache init (YES)
-- [hunter *** DEBUG *** 2018-12-19T13:02:12] Single-configuration generator
-- [hunter *** DEBUG *** 2018-12-19T13:02:12] Using default cache server
-- [hunter *** DEBUG *** 2018-12-19T13:02:12] List of cache servers:
-- [hunter *** DEBUG *** 2018-12-19T13:02:12]   * https://github.com/ingenue/hunter-cache
-- [hunter *** DEBUG *** 2018-12-19T13:02:12] Settings (finalize):
-- [hunter *** DEBUG *** 2018-12-19T13:02:12]   HunterGate done (NO)
-- [hunter *** DEBUG *** 2018-12-19T13:02:12]   Cache init (YES)
-- [hunter *** DEBUG *** 2018-12-19T13:02:12] Variables from HunterGate:
-- [hunter *** DEBUG *** 2018-12-19T13:02:12]   HUNTER_GATE_ROOT: /Users/bezigon/.hunter
-- [hunter *** DEBUG *** 2018-12-19T13:02:12]   HUNTER_GATE_VERSION: 0.23.76
-- [hunter *** DEBUG *** 2018-12-19T13:02:12]   HUNTER_GATE_SHA1: c7b60993e841850e2c449afd454f5d5aa4ec04e4
-- [hunter] Calculating Toolchain-SHA1
-- [hunter *** DEBUG *** 2018-12-19T13:02:12] [/Users/bezigon/Documents/GitHub/android-studio-with-hunter-bezigon/android-studio/app/.externalNativeBuild/cmake/debug/arm64-v8a/_3rdParty/Hunter/toolchain] > "/usr/local/Cellar/cmake/3.13.2/bin/cmake" "-DTOOLCHAIN_INFO_FILE=/Users/bezigon/Documents/GitHub/android-studio-with-hunter-bezigon/android-studio/app/.externalNativeBuild/cmake/debug/arm64-v8a/_3rdParty/Hunter/toolchain/toolchain.info.NOLF" "-DCMAKE_TOOLCHAIN_FILE=/Users/bezigon/Documents/GitHub/android-studio-with-hunter-bezigon/android-studio/app/.externalNativeBuild/cmake/debug/arm64-v8a/generated/toolchain.cmake" "-DHUNTER_SELF=/Users/bezigon/.hunter/_Base/Download/Hunter/0.23.76/c7b6099/Unpacked" "-GNinja" "-H/Users/bezigon/Documents/GitHub/android-studio-with-hunter-bezigon/android-studio/app/.externalNativeBuild/cmake/debug/arm64-v8a/_3rdParty/Hunter/toolchain" "-B/Users/bezigon/Documents/GitHub/android-studio-with-hunter-bezigon/android-studio/app/.externalNativeBuild/cmake/debug/arm64-v8a/_3rdParty/Hunter/toolchain/_builds" "-DCMAKE_MAKE_PROGRAM=/usr/local/bin/ninja" "-DCMAKE_RELEASE_POSTFIX=" "-DCMAKE_DEBUG_POSTFIX=d"
-- Android: Targeting API '24' with architecture 'arm64', ABI 'arm64-v8a', and processor 'aarch64'
-- Android: Selected Clang toolchain 'aarch64-linux-android-clang' with GCC toolchain 'aarch64-linux-android-4.9'
-- Check for working C compiler: /Users/bezigon/Library/Android/sdk/ndk-bundle/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang
-- Check for working C compiler: /Users/bezigon/Library/Android/sdk/ndk-bundle/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /Users/bezigon/Library/Android/sdk/ndk-bundle/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang++
-- Check for working CXX compiler: /Users/bezigon/Library/Android/sdk/ndk-bundle/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/bezigon/Documents/GitHub/android-studio-with-hunter-bezigon/android-studio/app/.externalNativeBuild/cmake/debug/arm64-v8a/_3rdParty/Hunter/toolchain/_builds
-- [hunter *** DEBUG *** 2018-12-19T13:02:18] Already exists: /Users/bezigon/.hunter/_Base/c7b6099/4e02b95/toolchain.info
-- [hunter] Calculating Config-SHA1
-- [hunter *** DEBUG *** 2018-12-19T13:02:18] Cache data is up-to-date
-- [hunter] HUNTER_ROOT: /Users/bezigon/.hunter
-- [hunter *** DEBUG *** 2018-12-19T13:02:18] HUNTER_TOOLCHAIN_ID_PATH: /Users/bezigon/.hunter/_Base/c7b6099/4e02b95
-- [hunter *** DEBUG *** 2018-12-19T13:02:18] HUNTER_CONFIGURATION_TYPES: Release;Debug
-- [hunter *** DEBUG *** 2018-12-19T13:02:18] HUNTER_BUILD_SHARED_LIBS:
-- [hunter] [ Hunter-ID: c7b6099 | Toolchain-ID: 4e02b95 | Config-ID: 733cc58 ]
-- [hunter *** DEBUG *** 2018-12-19T13:02:18] load: /Users/bezigon/.hunter/_Base/Download/Hunter/0.23.76/c7b6099/Unpacked/cmake/projects/md5/hunter.cmake
-- [hunter *** DEBUG *** 2018-12-19T13:02:18] md5 versions available: [1.6]
-- [hunter *** DEBUG *** 2018-12-19T13:02:18] Package 'md5' CONFIGURATION_TYPES: Release;Debug
-- [hunter *** DEBUG *** 2018-12-19T13:02:18] Package 'md5' is cacheable: YES
-- [hunter *** DEBUG *** 2018-12-19T13:02:18] Install to: /Users/bezigon/.hunter/_Base/c7b6099/4e02b95/733cc58/Install
-- [hunter] MD5_ROOT: /Users/bezigon/.hunter/_Base/c7b6099/4e02b95/733cc58/Install (ver.: 1.6)
-- [hunter *** DEBUG *** 2018-12-19T13:02:18] Package already installed: md5
-- [hunter *** DEBUG *** 2018-12-19T13:02:18] load: /Users/bezigon/.hunter/_Base/Download/Hunter/0.23.76/c7b6099/Unpacked/cmake/projects/md5/hunter.cmake ... end
-- [hunter]
************************* !!! WARNING !!! *************************

You're using experimental feature. This may not work well
and can be removed anytime. It's a workaround for existing
feature request:

https://github.com/ruslo/hunter/issues/495

*******************************************************************
-- [hunter] Building host project: /Users/bezigon/Documents/GitHub/android-studio-with-hunter-bezigon/cmake/host
-- [hunter *** DEBUG *** 2018-12-19T13:02:18] Using default host generator
-- [hunter] Calculating Toolchain-SHA1
-- [hunter] Calculating Config-SHA1
-- [hunter] HUNTER_ROOT: /Users/bezigon/.hunter
-- [hunter] [ Hunter-ID: c7b6099 | Toolchain-ID: 814ec5f | Config-ID: c9a4aa4 ]
-- [hunter] PROTOBUF_ROOT: /Users/bezigon/.hunter/_Base/c7b6099/814ec5f/c9a4aa4/Install (ver.: 3.0.0-p1)
^Cninja: error: rebuilding 'build.ninja': interrupted by user```

-- [hunter] Calculating Toolchain-SHA1
-- [hunter] Calculating Config-SHA1
-- [hunter] HUNTER_ROOT: /Users/bezigon/.hunter
-- [hunter] [ Hunter-ID: c7b6099 | Toolchain-ID: 814ec5f | Config-ID: c9a4aa4 ]
-- [hunter] PROTOBUF_ROOT: /Users/bezigon/.hunter/_Base/c7b6099/814ec5f/c9a4aa4/Install (ver.: 3.0.0-p1)

HUNTER_STATUS_DEBUG=ON is missing on host level, can you please add option(HUNTER_STATUS_DEBUG "..." ON) to host CMakeLists.txt?

tl;dr I suspect that something is going wrong

Added exit() feature strictly after hunter_add_package(Protobuf)

Since Protobuf is problematic part I do recommend you (at least for debugging for now) to add something plain simple, e.g. md5.

Performed gradle assembleDebug -Parch=arm64-v8a --info target
Waited several time on
and forcefully exited

Part with "early exit" should be completed succesfully using Gradle, if it's not competed then something went wrong and workaround is not working.
-- [hunter] Calculating Toolchain-SHA1
-- [hunter] Calculating Config-SHA1
-- [hunter] HUNTER_ROOT: /Users/bezigon/.hunter
-- [hunter] [ Hunter-ID: c7b6099 | Toolchain-ID: 814ec5f | Config-ID: c9a4aa4 ]
-- [hunter] PROTOBUF_ROOT: /Users/bezigon/.hunter/_Base/c7b6099/814ec5f/c9a4aa4/Install (ver.: 3.0.0-p1)

I don't see HUNTER_STATUS_DEBUG=ON was applied here...

I already used md5 in parallel with Protobuf, please look at https://github.com/bezigon/android-studio-with-hunter
The part with md5 passes without problems.
It's interesting that the protobuf of the latest version (3.5.2) runs without problems, though I had to wait a while.
I will try to use the latest version, although there are times when a specific protobuf version is required... Thanks anyway for your support.

I already used md5 in parallel with Protobuf, please look at https://github.com/bezigon/android-studio-with-hunter

That's not what I see:

Waited several time on

-- [hunter] PROTOBUF_ROOT: /Users/bezigon/.hunter/_Base/c7b6099/814ec5f/c9a4aa4/Install (ver.: 3.0.0-p1)
<=<======-------> 50% EXECUTING [1m 28s]

:app:generateJsonModelDebug
^C?

and forcefully exited

Meaning you have hunter_add_package(protobuf) call, I do expect that we exit before. Should be:
hunter_add_package(md5)
return() # workaround

hunter_add_package(protobuf) # not used in workaround

So I'm still not sure that you're using workaround correctly.

It's interesting that the protobuf of the latest version (3.5.2) runs without problems, though I had to wait a while.

Sources are here: https://github.com/hunter-packages/protobuf feel free to check

I understood you. Will retry with workaround on first occurence of hunter_add_package(...).


fix #269 and #270

Issue #, if available: #269 and #270
Description of changes:

Make macros used in ion_collection.h accessible.
Fix ion_catalog's usage of _ion_collection_remove.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

pump


Updating mbed-os to mbed-os-5.6.1

Please test this PR
If successful then merge, otherwise provide a known issue.
Once you get notification of the release being made public then tag Master with mbed-os-5.6.1 .

Build fails because Windows has this maximum path length of 260 characters and now the file it failed to create is:
<EMAIL_ADDRESS>which is 261 character long.
Related to https://github.com/ARMmbed/mbed-os/issues/3847

I'll re-test this using experimental Linux version of IAR compiler.

Works OK using Linux IAR: https://jenkins-internal.mbed.com/job/ARMmbed/job/mbed-os-example-mesh-minimal/job/iar_linux/2/console


Cannot set the dropdown's value to an unavailable option.

PXT Bug - FieldUserEnum
Describe the bug
Console message:

Cannot set the dropdown's value to an unavailable option.
Block type: radioMessageCode, Field name: MEMBER, Value: add1

The radio send block's dropdown item is unselected!

To Reproduce
Steps to reproduce the behavior:

Create new project on MakeCode for micro:bit
Select radio-broadcast extention
radio send block
See console log

Expected behavior
message1 is selected and there is no warning message.
Screenshots

Edit source code on JavaScript

When convert to Blocks, item is unselected.

transfered:
https://github.com/microsoft/pxt-arcade/issues/6037


Resolve rest dynamic url path

resolve non static path(like request param or path variable in spring framework)

@goesang


Fix DMA changes post 5.18.*

Adjust DMA to support changes post 5.18.*

@flumm Don't know if you're still interesting in this Repo but I've made a number of fixes to get gim to compile on 6.8.x.x kernel.

great that you did this, but it seems AMD moved on from this approach (gim driver + sriov cards; at least readily available ones) and they're over 8 years old at this point (which is rather ancient for GPUs). I won't merge this pull request though as I can't vet or test it at the moment, but I'll leave it open if i have some time in the future. Also I'd recommend opening a pull request at the original upstream repository, since there it'll get more visibility i guess.

Haha thanks!I'm aware of the age, but there is a weirdly bigger than you'd expect group of people trying to get this to work still, primary for usage within things like proxmox.Hence why I did it! I will probably take what you did in the 5.11 branch, plus what I've done and add it against the main repo too.I doubt anyone is going to merge that one either, as the repo is inactive.On 29 Jul 2024 08:40, flumm @.***> wrote:
great that you did this, but it seems AMD moved on from this approach (gim driver + sriov cards; at least readily available ones) and they're over 8 years old at this point (which is rather ancient for GPUs). I won't merge this pull request though as I can't vet or test it at the moment, but I'll leave it open if i have some time in the future. Also I'd recommend opening a pull request at the original upstream repository, since there it'll get more visibility i guess.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>


Feature/stop spin when hidden

Found another case where Image and Font would spin even if they were invisible.

@charri, and chance of merging this? I'm trying to get my app's GPU usage down to zero when nothing is actually happening. My hidden load spinners are keeping the GPU busy for no reason.

thanks, lgtm

@charri any chance of getting this released on NuGet? This bit us again today. Happy to help if needed.


Finaldifftool

@gamerson
The output isnt as pretty as before, but its completely legible just has one extra [] and extra , everywhere.
sample diff

Closing see comment, thanks!

Hey @gamerson, the exclusions are all there. I just alphabetized them with special chars first. Should I change the ordering?

Oh i see, nevermind then! this looks good.

Just started reviewing :) :octocat: Sent from GH.

Sent https://github.com/gamerson/liferay-blade-samples/pull/35


Analytics

Waiting documentation from customer on what and how to setup https://adobe-dx-support.slack.com/archives/C056VBDUWH1/p1694434483824349

Docs provided by Raul
this site has a simple analytics:

that’s enough!
thanks


docs/dev server: Add explicit vagrant restart command

A naïve vagrant halt + vagrant up [--provision] doesn't suffice.

Thanks! Merged.


Question/Feature request: Multiple documentations

Is there a way to use this for multiple different API's in the same project?

Hey @bernhardh !
So the way this works is that a view is rendered on a route. You could register your own routes that render the same view dynamically setting the config('stoplight.openapi.path') before rendering which would allow you to implement this

Thanks for the feedback!


Migrate three tests to sky/unit/test

Hot. Lgtm
On Friday, August 21, 2015, Adam Barth<EMAIL_ADDRESS>wrote:

You can view, comment on, or merge this pull request online at:
https://github.com/domokit/sky_engine/pull/737
Commit Summary

Migrate three tests to sky/unit/test

File Changes

D sky/tests/painting/canvas-expected.txt
https://github.com/domokit/sky_engine/pull/737/files#diff-0 (6)
D sky/tests/raw/color_bounds-expected.txt
https://github.com/domokit/sky_engine/pull/737/files#diff-1 (8)
D sky/tests/raw/null_text_crash-expected.txt
https://github.com/domokit/sky_engine/pull/737/files#diff-2 (6)
D sky/tests/raw/null_text_crash.dart
https://github.com/domokit/sky_engine/pull/737/files#diff-3 (17)
R sky/unit/test/engine/canvas_test.dart
https://github.com/domokit/sky_engine/pull/737/files#diff-4 (12)
R sky/unit/test/engine/color_bounds_test.dart
https://github.com/domokit/sky_engine/pull/737/files#diff-5 (5)
A sky/unit/test/engine/null_text_crash_test.dart
https://github.com/domokit/sky_engine/pull/737/files#diff-6 (10)

Patch Links:

https://github.com/domokit/sky_engine/pull/737.patch
https://github.com/domokit/sky_engine/pull/737.diff

—
Reply to this email directly or view it on GitHub
https://github.com/domokit/sky_engine/pull/737.


Update docs with cloud versioning info

Report
Docs to be updated:

[ ] https://dvc.org/doc/use-cases/data-registry/tutorial
[x] https://dvc.org/doc/user-guide/data-management (see this PR; may need a new page)
[ ] https://dvc.org/doc/user-guide/project-structure/dvc-files (add version_id and push fields)
[ ] https://dvc.org/doc/command-reference/config
[ ] https://dvc.org/doc/command-reference/push
[ ] https://dvc.org/doc/command-reference/remote/modify
[ ] https://dvc.org/doc/command-reference/import-url

Do we have a timeline for this feature release? Expected DVC version? Thanks

We are aiming for release around the start of December. I don't have an expected DVC version. The changes aren't breaking for now since users have to configure flags to use cloud versioning, so I don't think the version number matters too much. The plan is to put something out soon and start getting feedback rather than wait for a major release.

@pmrowla Should we add version_id to https://dvc.org/doc/user-guide/project-structure/dvc-files?

I think we can close this one after we address that (we can address adding more high-level content into use cases, get started, etc. in separate issues).

Also the config and push refs?

Also the config and push refs?

For config, the remote section just refers users to the dvc remote ref, so I don't think there's actually an updated needed.
For push, I'm not sure what needs to be added. Maybe an example?

Actually, the https://dvc.org/doc/command-reference/import-url synopsis is missing the --version-aware option, so that also needs to be done.

especially for GCS since their defaults are very strict (cc @daavoo)

@daavoo Do you remember where you saw this?

@daavoo Do you remember where you saw this?

In the GCS page when you create a new bucket with versioning enabled

@dberenbaum do we plan to do any other updates within this epic?

I hope to at least make some progress on the other checkboxes so we can close this one.


Convert STEP cost model reference costs to inputs

The hardcoded values need to be converted to input variables. This is required to run uncertainty quantification tools on the STEP cost model.

While it would be useful that this is done, the UQ tools can be used with step_ref in its current form so isn't an immediate priority. Closing as this is captured in the STEP cost model repo.


#trivial SDK doesn't need to close the popup

Summary

addressing https://github.com/coinbase/coinbase-wallet-sdk/pull/1279#discussion_r1594823480
no code change. only comments / folder reorg.
How did you test your changes?

🟡
Heimdall Review Status

Item
Status
More Info

Minimum Required Reviews

🟡
0/1

Show calculation

Show calculations

1 if user is bot
0

1 if user is external
0

From .codeflow.yml

1

Additional review requirements

Show calculations

Max
0

0

From CODEOWNERS

0

Global minimum
0

Max

1

1

1 if commit is unverified
0

Sum
1

📝 Heimdall docs
·
🕵 Advanced details
·
♻ Refresh Heimdall
·
#⃣ #ask-heimdall

Im ready
man. 24. jun. 2024 kl. 20:38 skrev Jungho Bang @.***>:

Merged #1280 https://github.com/coinbase/coinbase-wallet-sdk/pull/1280
into master.
—
Reply to this email directly, view it on GitHub
https://github.com/coinbase/coinbase-wallet-sdk/pull/1280#event-13270334076,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A6QTYORWYSX47FMSVY3GR43ZJBRS7AVCNFSM6AAAAABHN5DLAWVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJTGI3TAMZTGQYDONQ
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***
.com>


Feature Request: Support for Input via stdin and Output via stdout

I’m using your tool in a somewhat unconventional way: I’m working on extracting my company's data out of an old proprietary WORM (Write Once Read Many) document storage system into an open-source solution. I wouldn’t have been able to do this without binwalk, because it's locating and extracting the content (and, by subtraction, also locating the metadata). Thank you for making this possible!
My issue is that binwalk doesn’t accept input via stdin. As a result, I’m forced to write to disk nearly half a million times instead of processing everything in-memory, which drastically slows down my pipeline.
I realize this use case might be somewhat niche, but I believe that supporting stdin for input and stdout for output could also benefit many regular users. For instance, it would streamline workflows involving Bash one-liners and other command-line pipelines.
Thank you again for this incredibly useful tool—not just for firmware reverse-engineering, but also for broader applications beyond the firmware space.
Cheers!

Supporting stdin for analysis only (no extraction) shouldn't be too difficult, and I can definitely see other use cases for this.
Extraction would be a bit more difficult, for (as I see it) two reasons:

Binwalk relies on many external utilities to perform extraction which expect to be provided file paths as both their input and output; this would necessitate some wrapper code that writes data to disk, runs the extraction utility, then reads in the output file and sends it to stdout. This could work, but you'd still be doing a lot of disk I/O, so it doesn't seem that it would really solve your problem.
Sending the extracted data to stdout becomes complicated when there are multiple files to extract (for example, if you feed Binwalk some data that contains both gzip compressed data and xz compressed data), or when a more complex data type is extracted (e.g., a SquashFS file system or data archive which contains multiple files). The data printed to stdout in this case would need to be represented as JSON (or some similar format) describing all extracted files and their included content.

So, I suppose a little more detail on your use case would be helpful; are you mostly interested in identifying file signatures in some blob of data passed via stdin, or do you also need those file types to be extracted? Do you need recursive extraction?

My bottleneck is mostly with stdin being missing. I mentioned stdout more for completeness. In my mind, I was picturing the extracted files as output, but from your explanation, I can see why that would be tricky to implement.
In my use case, I have an array of large blobs that include all kinds of data. Each entry in each blob is neatly separated by a header. I load each blob into memory, split it using the delimiter string, run binwalk on each entry to find where the header ends and extract the content (non-recursively), and grab some metadata from the header.
Everything before binwalk happens in memory, but because binwalk doesn’t accept stdin, I have to write each entry-blob to disk, run binwalk on it, and then delete it again. This extra I/O really slows things down.
If it helps, you can take a look at the script that handles part of the work here.

Got it! I should have some time to add stdin support over the weekend, I'll update this ticket when I do.
FYI, the latest mater branch currently has support for sending the JSON analysis results to stdout (specify a log file of -), so once stdin support is added you'll be able to pass your data to stdin and get the JSON results from stdout.

That’s amazing—thank you so much!
I appreciate the tip, and I can definitely use that feature in my script.
While I have your attention, I noticed the --threads flag. So far, I’ve just left it at the default, but would you recommend experimenting with it to squeeze out even more performance?

The default thread count will be equal to the number of available CPU cores on your system.
Binwalk will only analyze one file per thread, so having more than one thread only makes a difference when performing a recursive extraction where there is more than one file to analyze.
When you are performing recursive extraction, my experience is that adding more threads than available CPU cores doesn't really help much (and in fact, can actually slow down analysis time), since each thread will pretty much max out the CPU core at 100%. Trying to run, say, 100 threads that each want 100% of your CPU time on a system with only 8 CPU cores isn't very helpful. :)
On the other hand, reducing the number of threads during recursive extraction can be helpful if there are many very large files. Binwalk will read each file into memory in its entirety, so limiting the number of concurrent threads will reduce memory usage.

That’s helpful, thanks! In that case, I’ll focus on other parts of my pipeline first.

Support for reading from standard input via the --stdin argument has been merged into the master branch. You can now:
$ cat test.gz | binwalk --extract --stdin --quiet --log -
[
{
  "Analysis": {
    "file_path": "/home/craig/Downloads/extractions/stdin",
    "file_map": [
      {
        "offset": 0,
        "id": "0e868d95-41e5-42d0-ac3a-d5f20b948dfc",
        "size": 108096828,
        "name": "gzip",
        "confidence": 250,
        "description": "gzip compressed data, operating system: Unix, timestamp: 1970-01-01 00:00:00, total size: 108096828 bytes",
        "always_display": false,
        "extraction_declined": false
      }
    ],
    "extractions": {
      "0e868d95-41e5-42d0-ac3a-d5f20b948dfc": {
        "size": 108096810,
        "success": true,
        "extractor": "gzip_built_in",
        "do_not_recurse": false,
        "output_directory": "/home/craig/Downloads/extractions/stdin.extracted/0"
      }
    }
  }
}
]

Thank you again! I can’t believe you got it done within a day!


Browser back button with same route but different params

I have a route configured in my app for http://example.com/search which takes a query string parameter q=test . User can search for different keywords and can click browser back or forward buttons. So when then navigate from http://example.com/search?q=1 to http://example.com/search?q=2 using browser back button, I would like to update the page.
Here are more details about what I am trying to do:
https://stackoverflow.com/questions/57062320/reactjs-browser-back-button-to-refresh-the-same-component
I tried to add connected-react-router but I am not sure where and how to trigger the API calls when browser back/forward button is pressed.

I solved this with out using react-connected-router.


LoadBalancers

Add support for LoadBalancers
https://docs.hetzner.cloud/#load-balancers

sorry it's already there ...


Support various bundling systems

I keep seeing an error which says "ReactDOM" not found when using this module. Changing the module to "react-dom" seemed to fix it. So I think this change will overall make this module work irrespective of the bundling mechanism used. This will allow this component to be used by systems like nextjs.
Ive published @roopakv/sarif-web-component with this change, systems built on nextjs, etc work\
cc @jeffersonking

Thanks for the change @roopakv. We will publish to @microsoft/sarif-web-component soon.

@jeffersonking thank you! would love to move back to your version rather than the one i published temporarily :)

@roopakv It has been published. New version is 0.4.0.


perf: ⚡ Optimize query for saving of increased and decreased hits

Codecov Report

Merging #10 (2d0e40d) into main (84164ee) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##              main       #10   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           23        23
  Lines          499       495    -4
  Branches        42        42
=========================================
- Hits           499       495    -4

Impacted Files
Coverage Δ

hitcount/models/hits.py
100.00% <100.00%> (ø)

hitcount/mixins.py
100.00% <0.00%> (ø)

hitcount/templatetags/hitcount_tags.py
100.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 84164ee...2d0e40d. Read the comment docs.


Fail to bufferize tensor.pad op when using transform dialect for x86 GEMM CodeGen

What happened?
It seems that the GEMM op is tiled and padded as expected, but failed to do bufferzation (transform.iree.bufferize) due to tensor.pad op. the error log :
matmul.mlir:4:10: error: op was not bufferized
    %0 = "mhlo.dot"(%lhs, %rhs) : (tensor<1024x1024xf32>, tensor<1024x1024xf32>) -> tensor<1024x1024xf32>
         ^
matmul.mlir:1:1: note: called from
func.func @dot_1024x1024x256() -> tensor<1024x1024xf32> {
^
matmul.mlir:4:10: note: see current operation: %56 = "tensor.pad"(%51, %0, %0, %41, %0) ({
^bb0(%arg12: index, %arg13: index):
  "tensor.yield"(%8) : (f32) -> ()
}) {nofold, operand_segment_sizes = dense<[1, 2, 2]> : vector<3xi32>, static_high = [-1, -1], static_low = [-1, -1]} : (tensor<?x1xf32>, index, index, index, index) -> tensor<12x1xf32>
    %0 = "mhlo.dot"(%lhs, %rhs) : (tensor<1024x1024xf32>, tensor<1024x1024xf32>) -> tensor<1024x1024xf32>
         ^
codegen_spec.mlir:16:5: error: failed to bufferize ModuleOp:
"builtin.module"() ({
  "func.func"() ({
    %0 = "arith.constant"() {value = 0 : index} : () -> index
    %1 = "arith.constant"() {value = 1024 : index} : () -> index
    %2 = "arith.constant"() {value = 288 : index} : () -> index
    %3 = "arith.constant"() {value = 512 : index} : () -> index
    %4 = "arith.constant"() {value = 128 : index} : () -> index
    %5 = "arith.constant"() {value = 12 : index} : () -> index
    %6 = "arith.constant"() {value = 32 : index} : () -> index
    %7 = "arith.constant"() {value = 1 : index} : () -> index
    %8 = "arith.constant"() {value = 0.000000e+00 : f32} : () -> f32
    %9 = "hal.interface.binding.subspan"(%0) {alignment = 64 : index, binding = 0 : index, descriptor_type = 7 : i32, operand_segment_sizes = dense<[1, 0]> : vector<2xi32>, set = 0 : index} : (index) -> memref<1024x1024xf32>
    "memref.assume_alignment"(%9) {alignment = 64 : i32} : (memref<1024x1024xf32>) -> ()
    %10 = "hal.interface.binding.subspan"(%0) {alignment = 64 : index, binding = 0 : index, descriptor_type = 7 : i32, operand_segment_sizes = dense<[1, 0]> : vector<2xi32>, set = 0 : index} : (index) -> !flow.dispatch.tensor<readonly:1024x1024xf32>
    %11 = "hal.interface.binding.subspan"(%0) {alignment = 64 : index, binding = 1 : index, descriptor_type = 7 : i32, operand_segment_sizes = dense<[1, 0]> : vector<2xi32>, set = 0 : index} : (index) -> memref<1024x1024xf32>
... ....

Steps to reproduce your issue
GEMM test IR:
func.func @dot_1024x1024x256() -> tensor<1024x1024xf32> {
    %lhs = util.unfoldable_constant dense<1.0> : tensor<1024x1024xf32>
    %rhs = util.unfoldable_constant dense<1.0> : tensor<1024x1024xf32>
    %0 = "mhlo.dot"(%lhs, %rhs) : (tensor<1024x1024xf32>, tensor<1024x1024xf32>) -> tensor<1024x1024xf32>
    return %0: tensor<1024x1024xf32>
}

Transform IR for dispatch:
transform.with_pdl_patterns {
^bb0(%arg0: !pdl.operation):
  transform.structured.canonicalized_sequence %arg0 {
  ^bb1(%arg1: !pdl.operation):
    %0 = transform.structured.match ops{["linalg.matmul"]} in %arg1
    %foreach_op, %tiled_op = tile_to_foreach_thread_op %0 {num_threads = [1, 1]}
    %dispatch_op = transform.iree.foreach_thread_to_flow %foreach_op
  }
}

Transform IR for CODEGEN:
transform.with_pdl_patterns {
^bb0(%arg0: !pdl.operation):
  transform.structured.canonicalized_sequence %arg0 {
  ^bb1(%arg1: !pdl.operation):
    %0 = transform.structured.match ops{["linalg.matmul"]} in %arg1
    %1:4 = transform.structured.tile %0 [288, 128, 512] {interchange=[0, 2, 1]}
    %2:4 = transform.structured.tile %1#0 [12, 32, 1] {interchange=[0, 1, 2]}
    %3 = transform.structured.pad %2#0 {padding_values=[0.0 : f32, 0.0 : f32, 0.0 : f32], padding_dimensions=[0, 1, 2], pack_paddings=[1, 1, 0], hoist_paddings=[5, 6, 0], transpose_paddings=[[1, 0], [0, 1], [0, 1]]}
    transform.iree.bufferize
  }
}

What component(s) does this issue relate to?
No response
Version information
nightly-build: 20220811.232
Additional context
No response

@MaheshRavishankar Can you help take a look at this and triage?

Assigning to Nicolas/Matthias to take a look. Seems related to the transform dialect hook up. (Marking P1 for now)

see https://github.com/nicolasvasilache/iree/commit/ca76f997b379fb3514577f92003b13f15ebfedeb
We usually vectorize after pad to rewrite padding into the proper abstractions.
This should at least unblock you, if you do not want to vectorize before bufferization then we'd need to look deeper next week.

Bufferization is currently not implemented for tensor.pad. However, with some of the recent changes, we can lower it do alloc_tensor + fill + insert_slice during bufferization. The only tricky part is that we don't want to generate a linalg.fill to avoid a dependency of TensorTransforms on Linalg. In case of a constant, we generate an arith.constant + alloc_tensor (copy) for the moment, so that we have a path that always lowers. (Ideally, we would vectorize first as Nicolas said.)

This change provides a bufferization path if not vectorized: https://reviews.llvm.org/D132355


UPDATE syntax error on setup

When I run ctc setup the command ends with this error repeated many tinems
ARGS ('near "UPDATE": syntax error',)

and then when I try to invoke some methods I have the same problem
ctc abit xxxxxxxxx --json
ARGS ('near "UPDATE": syntax error',)

Any tip?
Thanks in advance

Hi, i encountered the same issue here. I was stuck with the sqlite_schema table not found before like #82. After i upgraded my sqlite version from 3.31.0 to 3.34.0, i ran into the same ARGS ('near "UPDATE": syntax error',) like you did. I tried to run ctc setup --debug then i found out that it was due to the following statement.
ARGS ('near "UPDATE": syntax error',)
ARGS ('near "UPDATE": syntax error',)
ARGS ('near "UPDATE": syntax error',)
ARGS ('near "UPDATE": syntax error',)
    return await future
           ^^^^^^^^^^^^
  File "/home/xxx/.local/lib/python3.11/site-packages/aiosqlite/core.py", line 110, in run
    result = function()
             ^^^^^^^^^^
sqlite3.OperationalError: near "UPDATE": syntax error

So i think it might be yet another issue with the sqlite version. I wonder which sqlite you were using in your env.


filterValues and filterKeys

supersedes https://github.com/purescript/purescript-maps/pull/80
Rebases and implements the suggestions in that PR.

Sorry for the bikeshedding 😁
cc @hdgarrood

Sorry for the bikeshedding 😁

Nothing to be sorry about, someone's gotta do it!

:+1: for filter and filterWithKey.

Thanks!


[ChroniquesOubliees] Sheet "Add" generates error on Roll20

Hello all,
If you try to add the "ChroniquesOubliees" sheet on a new game on roll20, the Sandbox will generate this error message.
"SyntaxError: Invalid or unexpected token" "SyntaxError: Invalid or unexpected token\n    at eval (<anonymous>)\n    at messageHandler (evalmachine.<anonymous>:284:6)\n    at process.<anonymous> (/home/node/d20-api-server/node_modules/tiny-worker/lib/worker.js:65:55)\n    at emitTwo (events.js:106:13)\n    at process.emit (events.js:194:7)\n    at process.nextTick (internal/child_process.js:766:12)\n    at _combinedTickCallback (internal/process/next_tick.js:73:7)\n    at process._tickCallback (internal/process/next_tick.js:104:9)"
The sheet is still working. Could someone help us find what "Invalid or unexpected toke" the roll20 platform is talking about ?
It has to be said that if you instead, import a "custom sheet" and you copy paste the code from this Github, no error is throw.
FYI @Ulty

I'm not able to reproduce this, so I'm going to assume that this is fixed and close this issue. If you encounter this again, please create a new issue.

Just create a new game and try to add the script COFantasy from the library. You get
Your scripts are currently disabled due to an error that was detected. Please make appropriate changes to your script's code and click the "Save Script" button. We will then attempt to start running the scripts again. More info... If this script was installed from the Script Library, you might find help in the Community API Forum.
For reference, the error message generated was: SyntaxError: Invalid or unexpected token


Info about upgrading from version 1.0.x to 2.0 or later

Greetings!
Many thanks for your open-source effort!
I have a problem finding information about migrating from version 1.0.29 (currently in one of my working projects) to 2 or 3.
Can you give me a link, or say does it has some breaking changes or I just could do composer update without any risk ?

Hi, there is a docs page for this: https://flysystem.thephpleague.com/docs/upgrade-from-1.x/
hope this helps


[Android] Instant crash on Android 5.1.1 - NPE

Using on a device running Android 5.1.1, the application crashes a few seconds after launching.
The stack trace shows a null pointer exception at ReanimatedNativeHierarchyManager.java:266

Hi, looks like the crash is happening in React Native, the core framework that the app uses to publish to both iOS and Android. Unfortunately, there's no way for us to fix this crash. The only thing we can suggest is try upgrading your Android OS version to something more recent.

After some research, it seems that the issue was fixed in the react-native-reanimated 2.4.0 release. The app seems to use version 2.3.1. Updating the version number should be able to solve the issue.


Targets should support multiple-parameter wildcards.

Maybe something like {param1}-{param2} where parameters are resolved at the manifest-data level (i.e. can be used in values for certain keys, like output file / directory / dependency names).
They should also be resolve-able in command arguments.

This is implemented.


TiedEmbeddingTransformerDecoder: unit test and Gemma refactor

After the addition of Qwen2, we have multiple models using a transformer decoder class with head weights tied to embedding weights (Gemma and Qwen2). The class TiedEmbeddingTransformerDecoder is intended to handle these models and other similar such cases. But there are a couple follow-ups we need to address:
(1) Refactor Gemma to use TiedEmbeddingTransformerDecoder
(2) Add unit tests for TiedEmbeddingTransformerDecoder, as it is now in our core modules folder without testing.

@pbontrager I think technically speaking this should be doable too. Personally I kinda like having separate classes for this case though, then we can be very clear on the contract for stuff like FSDP wrapping or checkpointing, where weight tying changes the contract in a meaningful way.

where weight tying changes the contract in a meaningful way.

Can you expand on this point?

where weight tying changes the contract in a meaningful way.

Can you expand on this point?

I just mean that we have an extra param floating around in the untied case that we do not have in the tied case. So e.g. for our memory_efficient_fsdp_wrap config.. this won't work for tied weight models, and so it is nice to have an explicit class we could gate on there (though admittedly we're only really using this wrapping logic for Llama3 models rn anyways). Code pointer


distribute a little browser (e.g. Midori) and a little VNC player with the package os that people don't have to install that?

add mac and linux installer as well

add mac and linux installer as well


Allow achievements to be edited with ease

Broke Add/Edit achievements into a directive and uses the sidebar to allow editing Achievements.
Add new achievements:

Edit previous achievement:

fixes #655

I am merging this because it is a needed fix and doesn't seem to be getting any attention right now.


Fixed Kingdra’s attack cost

https://forum.tcgone.net/t/br-kingdra-ul-85-dragon-steam-is-currently-coded-to-cost-a-w/9707?u=mt.gufo

 KINGDRA_85


[Mention The Feature]: Adding the icons to the navbar for Better  UI !!!

Is your feature request related to a problem? Please describe.

There are no icons for the items in the  Navbar !! The current navbar lacks visual appeal and functionality. Icons can help users quickly recognize different sections and improve overall navigation, enhancing the user experience.
Describe the solution you'd like.
The navbar currently only contains text links. By adding relevant icons (home, profile, cart, etc.), the UI will look more modern and intuitive. Icons will make it easier for users to navigate the website at a glance.
Describe alternatives you've considered.
Integrating icons next to each text link in the navbar. The icons should be responsive, accessible, and should enhance clarity. For instance, using Font Awesome or Material Icons to display home, account, and cart icons beside the respective links.
Additional context.
An alternative could be using a hover effect that changes the text styling. However, icons will provide a more immediate, user-friendly enhancement compared to text changes.
Show us the magic with screenshots
Here’s an example of how the navbar might look with icons:

Checklist

[X] I have checked the Existing Issues
[X] I have read the Contributing Guidelines
[X] I want to work on this issue. (optional)

please assign me for this issue !!! I will add icons for all the pages.

@aditya-bhaumik , I'd like to work on the issue of adding icons to the navbar for better UI. Could you please assign it to me? Thank you!"


TSS-1580: Add enhanced support for wpf property bindings and expression properties

Soon to come...

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Jesper Kimmer Jørgensen seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


RG.PYEXECUTE no longer working from RedisGears

From the latest docker version the RG.PYEXECUTE command is no longer known whilst other commands are working, has this been changed or removed in the latest updates (4 months ago)
<IP_ADDRESS>:6379> RG.PYEXECUTE
(error) ERR unknown command `RG.PYEXECUTE`, with args beginning with:
<IP_ADDRESS>:6379>

For reference the RG.DUMPEXECUTIONS works so its assumed the instance is running correctly
<IP_ADDRESS>:6379> RG.DUMPEXECUTIONS
(empty array)

Can you tell me what docker command you used to create the container ? To Run the container with RedisGear module loaded and Python as a plugin, you type the following command
$ docker run -p 6379:6379 redislabs/redismod --loadmodule /usr/lib/redis/modules/redisgears.so Plugin /var/opt/redislabs/modules/rg/plugin/gears_python.so


Investigate auth in Canvas Panel

see discussion at https://github.com/digirati-co-uk/iiif-canvas-panel/discussions/54

This has been partially implemented for RIV (dependency of canvas panel)
Pull request:
https://github.com/digirati-co-uk/react-iiif-vault/pull/170
Demo:
https://deploy-preview-170--react-iiif-vault-demo.netlify.app/#?manifest=https://iiif-auth2-server.herokuapp.com/manifest/01_Icarus_Breughel.jpg
Likely need to decide wether to support Auth1 and its prior derivatives, but Auth2 is a much better fit conceptually. There are still bugs with the current implementation with refreshing things (like thumbnails) and maintaining the state. There are a few quirks that need ironed out.


No Symbols

Hi, i have no function names.
http://www.directupload.net/file/d/4165/i97jyax2_jpg.htm

I would recommend contacting Microsoft, they supply the debug symbols for
Microsoft programs.
On Nov 8, 2015 3:29 AM, "DocMAX"<EMAIL_ADDRESS>wrote:

Hi, i have no function names.
https://photos-1.dropbox.com/t/2/AABWDYsrj0D-jZF7644-PNlOMK5t8H9v2HViP5c6t85kVQ/12/2787358/png/32x32/1/_/1/2/2015-11-08 11_21_05-IDA - D__TEMP_termsrv2.i64 (termsrv2.dll).png/ENHRtvsDGAIgBygH/Tnd9vDJL6dJr8u1fPhsRgEcNnDHVuseb3Q-s6pV8L8o?size=1600x1200&size_mode=2
—
Reply to this email directly or view it on GitHub
https://github.com/binarymaster/rdpwrap/issues/64.

I had to install VC 2010 runtime.
I get symbols now, except for termsrv.dll from latest 10586 build.

Not a RDP Wrapper issue, closed.
Related issue here: https://github.com/binarymaster/rdpwrap/issues/65


iFiles support missing

Missing iFiles support. Structure for this support nearly identical to data-group, so I'll work on this next, didn't want to fix on same feature branch.

I whole-heartedly approve of smaller feature branches.

This has been merged, closing issue.


Need to be clearer on the first two statements

The first two statements in this doc state that load testing isn't permitted, and then state that it's strongly discouraged - these are two very different things. I'd suggest changing the "strongly discouraged" wording to "isn't permitted" so there is consistency of messaging.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d2ecd3e9-2961-7bfc-b7e1-6f99e9b20658
Version Independent ID: 5fd3e877-c745-a683-92d3-92bd3a86dfbe
Content: Capacity planning and load testing SharePoint Online
Content Source: Enterprise/capacity-planning-and-load-testing-sharepoint-online.md
Service: o365-administration
GitHub Login: @KCCross
Microsoft Alias: krowley

@mealiffe Thank you for this suggestion
I will get this wording suggestion over to the Enterprise writing team for investigation.
Thank you for reporting and making the docs better. Much appreciated.
I made a note to request the team to update this when the work is complete.


Decide on whether to test update action during CI run

Is this a BUG REPORT or FEATURE REQUEST?: feature request, but more of a research/decision issue

K2cli has the ability to update nodes per the update action. Should we test this during CI?
Pros: It is a feature we offer; we should test it.
Cons: It can take a very long time to test this action (roughly ten minutes per node being updated). As well, since update happens via a change in the config file, CI needs a way to alter its own config file.

Yes.


gif image filetype

In files object, gifs do not receive type: image (there is no type)

Fixed this in hypha :)


Ensure destination directory exists before handing off to ImageMagick

This is yet another fix for those unable to open image errors. Instead of filtering out non-existent directories I'm ensuring those directories exist before handing over to ImageMagick.
Fixes #31
Substitutes #43

Let's try this and see how it goes, thanks :+1:

Okay, nice one. :smile:


devd misconfiguration causes devd startup failure

Describe the bug
There is missing closing quote on line 28 of /usr/local/etc/devd/trident-media.conf
This causes start-up of devd daemon.
To Reproduce
Steps to reproduce the behavior:

Boot Trident
Check /var/log/messages for "devd.*syntax error"

Expected behavior

devd starts up
Error is gone

Log
Oct  6 21:29:07  devd[80762]: line 29: device: syntax error
OS Version:
Trident Beta 3
Additional context
N/A

Fixed via PR #25


OSX contextual program menus lacks details

should contain a link to program settings and the other good stuff.

Brainstorming here:
chatterino
  |-- About Chatterino
  |-- *separator*
  |-- Preferences
  |-- *default stuff added by macOS*

View
  |-- Zoom In
  |-- Zoom Out
  |-- Actual Size

Window:
  |-- Minimise
  |-- Zoom
  |-- *separator*
  |-- Bring all to front

Help
  |-- Chatterino Wiki
  |-- Chatterino GitHub
  |-- Chatterino Discord

I'm hesitant to add chatterino features like "New Split," "Close Tab" etc. for now unless someone has a good way to tie them to our keybinds in Window.cpp and be able to control context actions like "Cut," "Paste" where text fields don't exist.

I think this is completed since #5266


Scatter matrix grid alignment with rotated X labels

When I rotate my X labels in a scatter matrix, the lower left panel does not seem to align correctly.
samples = 40
hvplot.scatter_matrix(pd.DataFrame({
    'a':np.random.normal(0.5, 1, size=samples),
    'b':np.random.normal(0.0,2.0, size=samples),
    'c':np.random.normal(1e10,0.5e10,size=samples)}), xrotation=90)

hvplot = 0.7.3
holoviews = 1.14.5
bokeh = 2.3.3

But it seems that it cannot rotate the Count one. Does there are any method to rotate it too? Thanks

try this
import holoviews as hv
hv.extension('matplotlib')


How to make the editor read only?

Is there a way to make the editor read only?
I can see the readOnly option in code mirror but not sure about how to use it using this library.

Yeah, just include this in your CodeMirror extensions, alongside the PromQL extension:
EditorView.editable.of(false)

Closing the issue here since it's not related to the extension directly, but hoping it solves your question!


Ch 8 exercise issue

The 3rd exercise asks you to update restricted.html to use the base template. Don't believe we get to creating that file till later. Awesome book btw.

Hi Adam,
Thanks, glad you're enjoying the book - we re-jigged a few things around from the last version. I'll add that to the todo list to sort out (hopefully this weekend).
let us know if you spot anything else, and how you get on.
Cheers
Leif

Hi Adam,
Thanks for this - excellent spot. The issue still remains. I moved the exercise to Chapter 9. in Chapter 9, you create the restricted view - as you quite rightly pointed out. So by moving the exercise to the end of Ch9, everything is now consistent. I'll close this now - but thanks for the spot!

PS I added you to the acknowledgements page. Cheers!


concurrent.futures._base.TimeoutError in GET /clan/LL8RQ8CU.json

Error in ClashLeaders
concurrent.futures._base.TimeoutError in GET /clan/LL8RQ8CU.json
View on Bugsnag
Stacktrace
clashleaders/clash/api.py:30 - __fetch_with_session
clashleaders/clash/api.py:36 - __fetch_all
clashleaders/clash/api.py:84 - fetch_all_players
clashleaders/model/clan.py:61 - fetch_and_save
clashleaders/views/clan.py:118 - clan_from_days_ago
clashleaders/views/clan.py:40 - clan_detail_json

View full stacktrace
Created automatically via Bugsnag

An error linked to this issue has been marked as fixed in Bugsnag
concurrent.futures._base.TimeoutError in GET /clan/LL8RQ8CU.json

An error linked to this issue has been reopened in Bugsnag
concurrent.futures._base.TimeoutError in GET /clan/devastation-8yyrgg29.xlsx

An error linked to this issue has been marked as fixed in Bugsnag
concurrent.futures._base.TimeoutError in GET /clan/devastation-8yyrgg29.xlsx

An error linked to this issue has been reopened in Bugsnag
concurrent.futures._base.TimeoutError in GET /clan/Y002UJ8V.json

An error linked to this issue has been marked as fixed in Bugsnag
concurrent.futures._base.TimeoutError in GET /clan/ri-bu-luo-wu-tuo-bang-guo-du-pcy2crp2.xlsx

An error linked to this issue has been reopened in Bugsnag
concurrent.futures._base.TimeoutError in GET /clan/9G2GCP9R.json

An error linked to this issue has been marked as fixed in Bugsnag
concurrent.futures._base.TimeoutError in GET /clan/an-jiao-xin-ling-yu-y9gq0rvg.xlsx

An error linked to this issue has been reopened in Bugsnag
concurrent.futures._base.TimeoutError in GET /clan/P82RLLQR.json

An error linked to this issue has been marked as fixed in Bugsnag
concurrent.futures._base.TimeoutError in GET /clan/i-love-coc-qqvqcccp.xlsx

An error linked to this issue has been reopened in Bugsnag
concurrent.futures._base.TimeoutError in GET /clan/coc-oaxaca-prqcj22p.xlsx


Errored Out Run UI

Similar to the canceled UI, the thank you page will show a "failed to complete" message when the run errors out.  It needs styling and copy:

No Zeplin design that I'm aware of.
HTML in errored.jsx: https://github.com/Autodesk/molecular-design-applications/blob/master/client/public/js/components/errored.jsx
No Sass, but create an errored.scss if needed.

Addressed by: https://github.com/Autodesk/molecular-design-applications/pull/30


version bump

Checklist

[ ] Used a personal fork of the feedstock to propose changes
[ ] Bumped the build number (if the version is unchanged)
[ ] Reset the build number to 0 (if the version changed)
[ ] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[ ] Ensured the license file is being packaged.

@conda-forge-admin, please rerender

Hi! This is the friendly automated conda-forge-linting service.
I wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found some lint.
Here's what I've got...
For recipe/meta.yaml:

When defining a source/url please add a sha256, sha1 or md5 checksum (sha256 preferably).
The source section contained an unexpected subsection name. sha-256 is not a valid subsection name.
noarch packages can't have skips with selectors. If the selectors are necessary, please remove noarch: python.

Hi! This is the friendly automated conda-forge-webservice.
I tried to rerender for you, but it looks like there was nothing to do.
This message was generated by GitHub actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/11520959332.