1
0
Fork 0
mirror of https://github.com/ruby/ruby.git synced 2022-11-09 12:17:21 -05:00
ruby--ruby/tool/transform_mjit_header.rb

282 lines
9.1 KiB
Ruby
Raw Normal View History

common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
# Copyright (C) 2017 Vladimir Makarov, <vmakarov@redhat.com>
# This is a script to transform functions to static inline.
# Usage: transform_mjit_header.rb <c-compiler> <header file> <out>
require 'fileutils'
require 'tempfile'
PROGRAM = File.basename($0, ".*")
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
module MJITHeader
ATTR_VALUE_REGEXP = /[^()]|\([^()]*\)/
ATTR_REGEXP = /__attribute__\s*\(\(#{ATTR_VALUE_REGEXP}*\)\)/
# Example:
# VALUE foo(int bar)
# VALUE __attribute__ ((foo)) bar(int baz)
# __attribute__ ((foo)) VALUE bar(int baz)
FUNC_HEADER_REGEXP = /\A[^\[{(]*(\s*#{ATTR_REGEXP})*[^\[{(]*\((#{ATTR_REGEXP}|[^()])*\)(\s*#{ATTR_REGEXP})*\s*/
TARGET_NAME_REGEXP = /\A(rb|ruby|vm|insn|attr|Init)_/
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
# Predefined macros for compilers which are already supported by MJIT.
# We're going to support cl.exe too (WIP) but `cl.exe -E` can't produce macro.
SUPPORTED_CC_MACROS = [
'__GNUC__', # gcc
'__clang__', # clang
]
# These macros are relied on this script's transformation
PREFIXED_MACROS = [
'ALWAYS_INLINE',
]
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
# For MinGW's ras.h. Those macros have its name in its definition and can't be preprocessed multiple times.
RECURSIVE_MACROS = %w[
RASCTRYINFO
RASIPADDR
]
IGNORED_FUNCTIONS = [
'rb_vm_search_method_slowpath', # This increases the time to compile when inlined. So we use it as external function.
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
'rb_equal_opt', # Not used from VM and not compilable
]
ALWAYS_INLINED_FUNCTIONS = [
'vm_opt_plus',
'vm_opt_minus',
'vm_opt_mult',
'vm_opt_div',
'vm_opt_mod',
'vm_opt_neq',
'vm_opt_lt',
'vm_opt_le',
'vm_opt_gt',
'vm_opt_ge',
'vm_opt_ltlt',
'vm_opt_aref',
'vm_opt_aset',
'vm_opt_aref_with',
'vm_opt_aset_with',
'vm_opt_not',
'vm_getinstancevariable',
'vm_setinstancevariable',
'vm_setivar',
]
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
# Return start..stop of last decl in CODE ending STOP
def self.find_decl(code, stop)
level = 0
i = stop
while i = code.rindex(/[;{}]/, i)
if level == 0 && stop != i && decl_found?($&, i)
return decl_start($&, i)..stop
end
case $&
when '}'
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
level += 1
when '{'
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
level -= 1
end
i -= 1
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
end
nil
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
end
def self.decl_found?(code, i)
i == 0 || code == ';' || code == '}'
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
end
def self.decl_start(code, i)
if i == 0 && code != ';' && code != '}'
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
0
else
i + 1
end
end
# Given DECL return the name of it, nil if failed
def self.decl_name_of(decl)
ident_regex = /\w+/
decl = decl.gsub(/^#.+$/, '') # remove macros
reduced_decl = decl.gsub(ATTR_REGEXP, '') # remove attributes
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
su1_regex = /{[^{}]*}/
su2_regex = /{([^{}]|#{su1_regex})*}/
su3_regex = /{([^{}]|#{su2_regex})*}/ # 3 nested structs/unions is probably enough
reduced_decl.gsub!(su3_regex, '') # remove structs/unions in the header
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
id_seq_regex = /\s*(#{ident_regex}(\s+|\s*[*]+\s*))*/
# Process function header:
match = /\A#{id_seq_regex}(?<name>#{ident_regex})\s*\(/.match(reduced_decl)
return match[:name] if match
# Process non-function declaration:
reduced_decl.gsub!(/\s*=[^;]+(?=;)/, '') # remove initialization
match = /#{id_seq_regex}(?<name>#{ident_regex})/.match(reduced_decl);
return match[:name] if match
nil
end
# Return true if CC with CFLAGS compiles successfully the current code.
# Use STAGE in the message in case of a compilation failure
def self.check_code!(code, cc, cflags, stage)
with_code(code) do |path|
cmd = "#{cc} #{cflags} #{path}"
out = IO.popen(cmd, err: [:child, :out], &:read)
unless $?.success?
STDERR.puts "error in #{stage} header file:\n#{out}"
exit false
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
end
end
end
# Remove unpreprocessable macros
def self.remove_harmful_macros!(code)
code.gsub!(/^#define #{Regexp.union(RECURSIVE_MACROS)} .*$/, '')
end
# -dD outputs those macros, and it produces redefinition warnings or errors
# This assumes common.mk passes `-DMJIT_HEADER` first when it creates rb_mjit_header.h.
def self.remove_predefined_macros!(code)
code.sub!(/\A(#define [^\n]+|\n)*(#define MJIT_HEADER 1\n)/, '\2')
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
end
# Return [macro, others]. But others include PREFIXED_MACROS to be used in code.
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
def self.separate_macro_and_code(code)
code.lines.partition do |l|
l.start_with?('#') && PREFIXED_MACROS.all? { |m| !l.start_with?("#define #{m}") }
end.map! { |lines| lines.join('') }
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
end
def self.write(code, out)
FileUtils.mkdir_p(File.dirname(out))
File.binwrite("#{out}.new", code)
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
FileUtils.mv("#{out}.new", out)
end
# Note that this checks runruby. This conservatively covers platform names.
def self.windows?
RUBY_PLATFORM =~ /mswin|mingw|msys/
end
def self.cl_exe?(cc)
cc =~ /\Acl(\z| |\.exe)/
end
# If code has macro which only supported compilers predefine, return true.
def self.supported_header?(code)
SUPPORTED_CC_MACROS.any? { |macro| code =~ /^#\s*define\s+#{Regexp.escape(macro)}\b/ }
end
# This checks if syntax check outputs one of the following messages.
# "error: conflicting types for 'restrict'"
# "error: redefinition of parameter 'restrict'"
# If it's true, this script regards platform as AIX or Solaris and adds -std=c99 as workaround.
def self.conflicting_types?(code, cc, cflags)
with_code(code) do |path|
cmd = "#{cc} #{cflags} #{path}"
out = IO.popen(cmd, err: [:child, :out], &:read)
!$?.success? &&
(out.match?(/error: conflicting types for '[^']+'/) ||
out.match?(/error: redefinition of parameter '[^']+'/))
end
end
def self.with_code(code)
Tempfile.open(['', '.c'], mode: File::BINARY) do |f|
f.puts code
f.close
return yield(f.path)
end
end
private_class_method :with_code
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
end
if ARGV.size != 3
abort "Usage: #{$0} <c-compiler> <header file> <out>"
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
end
cc = ARGV[0]
code = File.binread(ARGV[1]) # Current version of the header file.
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
outfile = ARGV[2]
if MJITHeader.cl_exe?(cc)
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
cflags = '-DMJIT_HEADER -Zs'
else
cflags = '-S -DMJIT_HEADER -fsyntax-only -Werror=implicit-function-declaration -Werror=implicit-int -Wfatal-errors'
end
if !MJITHeader.cl_exe?(cc) && !MJITHeader.supported_header?(code)
puts "This compiler (#{cc}) looks not supported for MJIT. Giving up to generate MJIT header."
MJITHeader.write("#error MJIT does not support '#{cc}' yet", outfile)
exit
end
MJITHeader.remove_predefined_macros!(code)
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
if MJITHeader.windows? # transformation is broken with Windows headers for now
MJITHeader.remove_harmful_macros!(code)
MJITHeader.check_code!(code, cc, cflags, 'initial')
puts "\nSkipped transforming external functions to static on Windows."
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
MJITHeader.write(code, outfile)
exit
end
macro, code = MJITHeader.separate_macro_and_code(code) # note: this does not work on MinGW
code_to_check = "#{code}#{macro}" # macro should not affect code again
if MJITHeader.conflicting_types?(code_to_check, cc, cflags)
cflags = "#{cflags} -std=c99" # For AIX gcc
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
end
# Check initial file correctness in the manner of final output.
MJITHeader.check_code!(code_to_check, cc, cflags, 'initial')
puts "\nTransforming external functions to static:"
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
stop_pos = -1
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
extern_names = []
# This loop changes function declarations to static inline.
while (decl_range = MJITHeader.find_decl(code, stop_pos))
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
stop_pos = decl_range.begin - 1
decl = code[decl_range]
decl_name = MJITHeader.decl_name_of(decl)
if MJITHeader::IGNORED_FUNCTIONS.include?(decl_name) && /#{MJITHeader::FUNC_HEADER_REGEXP}{/.match(decl)
puts "#{PROGRAM}: changing definition of '#{decl_name}' to declaration"
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
code[decl_range] = decl.sub(/{.+}/m, ';')
elsif MJITHeader::ALWAYS_INLINED_FUNCTIONS.include?(decl_name) && match = /#{MJITHeader::FUNC_HEADER_REGEXP}{/.match(decl)
header = match[0].sub(/{\z/, '').strip
header = "static inline #{header.sub(/\A((static|inline) )+/, '')}"
decl[match.begin(0)...match.end(0)] = '{' # remove header
code[decl_range] = "\nALWAYS_INLINE(#{header});\n#{header} #{decl}"
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
elsif extern_names.include?(decl_name) && (decl =~ /#{MJITHeader::FUNC_HEADER_REGEXP};/)
decl.sub!(/(extern|static|inline) /, ' ')
unless decl_name =~ /\Aattr_\w+_\w+\z/ # skip too-many false-positive warnings in insns_info.inc.
puts "#{PROGRAM}: making declaration of '#{decl_name}' static inline"
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
end
code[decl_range] = "static inline #{decl}"
elsif (match = /#{MJITHeader::FUNC_HEADER_REGEXP}{/.match(decl)) && (header = match[0]) !~ /static/
unless decl_name.match(MJITHeader::TARGET_NAME_REGEXP)
puts "#{PROGRAM}: SKIPPED to transform #{decl_name}"
next
end
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
extern_names << decl_name
decl[match.begin(0)...match.end(0)] = ''
if decl =~ /\bstatic\b/
puts "warning: a static decl inside external definition of '#{decl_name}'"
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
end
header.sub!(/(extern|inline) /, ' ')
unless decl_name =~ /\Aattr_\w+_\w+\z/ # skip too-many false-positive warnings in insns_info.inc.
puts "#{PROGRAM}: making external definition of '#{decl_name}' static inline"
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
end
code[decl_range] = "static inline #{header}#{decl}"
end
end
code << macro
common.mk: install a single header file for JIT compilation which is created by transforming a preprocessed vm.c. This file will be used by JIT compiler's generated code which we are going to have from succeeding commits. Makefile.in: generate MJIT header for UNIX environments. win32/Makefile.sub: generate MJIT header for mswin environments. At initial merge, we're going to support only MinGW for Windows. So the header installed by this file won't be used for short term, but we'll add mswin support in a half year or so, for sure. tool/transform_mjit_header.rb: New. This script was originally written as minimize_mjit_header.rb by Vladimir N. Makarov <vmakarov@redhat.com> for Feature 12589. Then I refactored a little so that it can conform CodeClimate CI which is currently set for Ruby's GitHub repository, and fixed some bugs and ported it to work on Windows. Also, as original minimize_mjit_header.rb takes too long time to run, this is modified to skip minimization step because having *static* unused definitions does not waste compilation time on -O2 since compiler can skip to compile unused static functions. So this does no longer "minimize" the header and is renamed. This header installation does NOT include a header to automatically export symbols used by MJIT. That's because original MJIT code was failing to export symbols in the import header in macOS environment. But I would like to have the functionality for maintainability in the future. I'll manually export things but it would be just an intemediate solution. Patch by: Vladimir N. Makarov <vmakarov@redhat.com> Part of: Feature 12589 and 14235. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62187 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 00:49:21 -05:00
# Check the final file correctness
MJITHeader.check_code!(code, cc, cflags, 'final')
MJITHeader.write(code, outfile)