Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   C Programming (http://www.velocityreviews.com/forums/f42-c-programming.html)
-   -   Separate compilation and access to small integer data in other programs (http://www.velocityreviews.com/forums/t954764-separate-compilation-and-access-to-small-integer-data-in-other-programs.html)

James Harris 11-22-2012 09:27 PM

Separate compilation and access to small integer data in other programs
 
Having tried a few options I'm not sure how best to do the following
in C or how feasible this is. Any suggestions?

AIUI the normal way of working in C where one program, say AA, uses
integers (or bitfields) as part of its interface those integers are
defined in a header file, say AA.h. Then when program BB wants to
interface with AA it includes AA.h and uses those symbolic values when
talking to AA. This means that the values are compiled-in to BB.

By contrast, I want the two progams to be compiled separately. Program
AA is to be compiled with its integer constants wholly inside it.
Program BB is to refer to the values as externals. Example below. The
intention is that the constants not be resolved at compile time but at
link time or later.

Best I've come up with so far can be illustrated as follows.

File AA:
int AA_CONST1 = 1;
int AA_CONST2 = 4;
int AA_FLAGS1 = 0x8000
void AA() {}

File BB:
extern AA_CONST2;
void BB() {
x = AA_CONST2;
AA_function(x, AA_FLAGS1);
}

It SEEMS to work but are there any gotchas? Is there a better way to
do it?

It is slightly annoying in that #defines in existing header files
cannot be used directly but ints have to be defined to represent them.

James

Ben Bacarisse 11-22-2012 10:24 PM

Re: Separate compilation and access to small integer data in other programs
 
James Harris <james.harris.1@gmail.com> writes:

> Having tried a few options I'm not sure how best to do the following
> in C or how feasible this is. Any suggestions?
>
> AIUI the normal way of working in C where one program, say AA, uses
> integers (or bitfields) as part of its interface those integers are
> defined in a header file, say AA.h. Then when program BB wants to
> interface with AA it includes AA.h and uses those symbolic values when
> talking to AA. This means that the values are compiled-in to BB.
>
> By contrast, I want the two progams to be compiled separately. Program
> AA is to be compiled with its integer constants wholly inside it.
> Program BB is to refer to the values as externals. Example below. The
> intention is that the constants not be resolved at compile time but at
> link time or later.
>
> Best I've come up with so far can be illustrated as follows.
>
> File AA:
> int AA_CONST1 = 1;
> int AA_CONST2 = 4;
> int AA_FLAGS1 = 0x8000
> void AA() {}
>
> File BB:
> extern AA_CONST2;
> void BB() {
> x = AA_CONST2;
> AA_function(x, AA_FLAGS1);
> }


I'm having trouble squaring this with the words. Neither AA nor BB
seems to be a program. They just look like translation units that might
be linked into one single program (with some other bits of course).

> It SEEMS to work but are there any gotchas? Is there a better way to
> do it?


How many programs are there? Could you give short actually compileable
examples of them? It might then be clear why what you want can't be
done another way (or maybe it will become clear that it can be).

> It is slightly annoying in that #defines in existing header files
> cannot be used directly but ints have to be defined to represent them.


Maybe you could give an example of what the problem with macro constants
is. Is the problem extra-C? I get a feeling this is maybe a build
problem, not one of C structure, and there is some build constraint
that's not been described.

--
Ben.

BartC 11-22-2012 10:38 PM

Re: Separate compilation and access to small integer data in other programs
 
"James Harris" <james.harris.1@gmail.com> wrote in message
news:4beef828-17fc-4c18-ba8c-9b92733ae9bc@n5g2000vbk.googlegroups.com...
> Having tried a few options I'm not sure how best to do the following
> in C or how feasible this is. Any suggestions?
>
> AIUI the normal way of working in C where one program, say AA, uses
> integers (or bitfields) as part of its interface those integers are
> defined in a header file, say AA.h. Then when program BB wants to
> interface with AA it includes AA.h and uses those symbolic values when
> talking to AA. This means that the values are compiled-in to BB.
>
> By contrast, I want the two progams to be compiled separately. Program
> AA is to be compiled with its integer constants wholly inside it.
> Program BB is to refer to the values as externals. Example below. The
> intention is that the constants not be resolved at compile time but at
> link time or later.


> File AA:
> int AA_CONST1 = 1;
> int AA_CONST2 = 4;
> int AA_FLAGS1 = 0x8000
> void AA() {}
>
> File BB:
> extern AA_CONST2;
> void BB() {
> x = AA_CONST2;
> AA_function(x, AA_FLAGS1);
> }


It seems fine provided you realise that these are not actually constants
(ie. immediate data), but variables.

Immediate data I think can be imported across modules via a linker, but I
don't think there's a mechanism in C to achieve this (even an assembler
would have difficulty because it might need to know the field-size needed to
contain the value).

What's the problem with having a header that is common to both AA and BB? Is
it that you are obliged to recompile both if some of the values
change?

--
Bartc


James Harris 11-23-2012 12:51 PM

Re: Separate compilation and access to small integer data in other programs
 
On Nov 22, 10:24*pm, Ben Bacarisse <ben.use...@bsb.me.uk> wrote:
> James Harris <james.harri...@gmail.com> writes:
> > Having tried a few options I'm not sure how best to do the following
> > in C or how feasible this is. Any suggestions?

>
> > AIUI the normal way of working in C where one program, say AA, uses
> > integers (or bitfields) as part of its interface those integers are
> > defined in a header file, say AA.h. Then when program BB wants to
> > interface with AA it includes AA.h and uses those symbolic values when
> > talking to AA. This means that the values are compiled-in to BB.

>
> > By contrast, I want the two progams to be compiled separately. Program
> > AA is to be compiled with its integer constants wholly inside it.
> > Program BB is to refer to the values as externals. Example below. The
> > intention is that the constants not be resolved at compile time but at
> > link time or later.

>
> > Best I've come up with so far can be illustrated as follows.

>
> > File AA:
> > int AA_CONST1 = 1;
> > int AA_CONST2 = 4;
> > int AA_FLAGS1 = 0x8000
> > void AA() {}

>
> > File BB:
> > extern AA_CONST2;
> > void BB() {
> > * x = AA_CONST2;
> > * AA_function(x, AA_FLAGS1);
> > }

>
> I'm having trouble squaring this with the words. *Neither AA nor BB
> seems to be a program. *They just look like translation units that might
> be linked into one single program (with some other bits of course).


Possibly poor choice of term on my part. "Program" may be too generic.
Yes, you could regard both of them linked together as a program. I
called each a program as the idea is that they be written, compiled
and distributed separately. It is intended that they are only linked
together when one is executed. Be that as it may, yes, they are
separate translation units.

>
> > It SEEMS to work but are there any gotchas? Is there a better way to
> > do it?

>
> How many programs are there? *Could you give short actually compileable
> examples of them? *It might then be clear why what you want can't be
> done another way (or maybe it will become clear that it can be).


I'll avoid the term "program" for the reasons mentioned above but I
can give a succinct example in which there are two source files. The
first, extern-t1.c, is

/*
* Test reference to external small constants.
*/
/* Build with
* cc -c extern-t1.c
*/
extern int unix_sys_open;
#include <stdio.h>
int main(void) {
fprintf(stderr, "Symbol number is %i\n", unix_sys_open);
}

The second file, os_sys.c, is

/*
* Test linkage to a small integer defined here
*/
/* Compile with
* cc -c os_sys.c
*/
#include <sys/syscall.h>
const int unix_sys_open = SYS_open;

Of course these are tiny examples. I figure small examples that
illustrate the point are preferred but perhaps I should say how such
translation units are intended to be run. Both will be compiled to
object (.o) files and not linked. Once the first, extern-t1, is
invoked the loader will look for and load extern-t1.o. It will then
resolve symbol unix_sys_open by loading os_sys.o and linking the two
modules together. In the above case it will then start extern-t1's
main() routine.

So only extern-t1 is invoked. The 'system' under which it runs will be
responsible for resolving the reference to the external symbol.


> > It is slightly annoying in that #defines in existing header files
> > cannot be used directly but ints have to be defined to represent them.

>
> Maybe you could give an example of what the problem with macro constants
> is.


One issue for me is that changes to symbolic constants in one routine
require recompilation of all modules which are potentially dependent.
In most cases the symbolic constants may not change but all dependent
files still need to be recompiled just in case. It would be better if
dependent module need to be recompiled only when other more
substantive parts of the interface change - e.g. a new version of a
piece of server code *takes away* a feature that existed in earlier
versions or *changes* an interface to make it incompatible with
previous versions.

> Is the problem extra-C? I get a feeling this is maybe a build
> problem, not one of C structure, and there is some build constraint
> that's not been described.


It is (at least intended to be) completely in keeping with C standards
and to run with standard C though Bart has raised a point about making
the values read-only. Will reply to him on that.

James

James Harris 11-23-2012 01:14 PM

Re: Separate compilation and access to small integer data in other programs
 
On Nov 22, 10:39*pm, "BartC" <b...@freeuk.com> wrote:
> "James Harris" <james.harri...@gmail.com> wrote in message
>
> news:4beef828-17fc-4c18-ba8c-9b92733ae9bc@n5g2000vbk.googlegroups.com...
>
>
>
> > Having tried a few options I'm not sure how best to do the following
> > in C or how feasible this is. Any suggestions?

>
> > AIUI the normal way of working in C where one program, say AA, uses
> > integers (or bitfields) as part of its interface those integers are
> > defined in a header file, say AA.h. Then when program BB wants to
> > interface with AA it includes AA.h and uses those symbolic values when
> > talking to AA. This means that the values are compiled-in to BB.

>
> > By contrast, I want the two progams to be compiled separately. Program
> > AA is to be compiled with its integer constants wholly inside it.
> > Program BB is to refer to the values as externals. Example below. The
> > intention is that the constants not be resolved at compile time but at
> > link time or later.
> > File AA:
> > int AA_CONST1 = 1;
> > int AA_CONST2 = 4;
> > int AA_FLAGS1 = 0x8000
> > void AA() {}

>
> > File BB:
> > extern AA_CONST2;
> > void BB() {
> > *x = AA_CONST2;
> > *AA_function(x, AA_FLAGS1);
> > }

>
> It seems fine provided you realise that these are not actually constants
> (ie. immediate data), but variables.


In other words, I cannot protect the original values from being
altered? That may be a problem. Is there no way with C to make them
read-only?

I suppose ideally the loader would place any constants in a read-only
page. I can do that and it would protect them perfectly. It would also
allow any attempt to update them to be trapped and pinpointed.

The slight issue is that there seems to be no way in C to say which
symbols should be made available to other routines, i.e. which should
be exported, so if two programs are linked together each has access to
all the data in the other. I'd welcome correction if there is a
feature of C which allows this control.

> Immediate data I think can be imported across modules via a linker, but I
> don't think there's a mechanism in C to achieve this (even an assembler
> would have difficulty because it might need to know the field-size neededto
> contain the value).


Field sizes should be fixed (at least under a 32-bit OS; a change to a
64-bit OS would be another issue but them everything would probably
require a recompile).

Interesting idea about having the linker resolve immediate values.
That would be good. I can see that if full-size 32-bit forms of
immediate instructions were used the linker could patch them by its
standard mechanisms as that's the kind of thing it does anyway. Like
you I cannot think of a way to do that in C.

> What's the problem with having a header that is common to both AA and BB?Is
> it that you are obliged to recompile both if some of the values
> change?


Yes. In most setups it would be rare for such values to change but
even where there is the potential for them to have changed all
dependent modules also need to be recompiled.

James

Eric Sosman 11-23-2012 01:33 PM

Re: Separate compilation and access to small integer data in otherprograms
 
On 11/23/2012 7:51 AM, James Harris wrote:
> [...] a succinct example in which there are two source files. The
> first, extern-t1.c, is
>
> /*
> * Test reference to external small constants.
> */
> /* Build with
> * cc -c extern-t1.c
> */
> extern int unix_sys_open;


Add "const" here, or ...

> #include <stdio.h>
> int main(void) {
> fprintf(stderr, "Symbol number is %i\n", unix_sys_open);
> }
>
> The second file, os_sys.c, is
>
> /*
> * Test linkage to a small integer defined here
> */
> /* Compile with
> * cc -c os_sys.c
> */
> #include <sys/syscall.h>
> const int unix_sys_open = SYS_open;


... remove "const" here.

> Of course these are tiny examples. I figure small examples that
> illustrate the point are preferred but perhaps I should say how such
> translation units are intended to be run. Both will be compiled to
> object (.o) files and not linked. Once the first, extern-t1, is
> invoked the loader will look for and load extern-t1.o. It will then
> resolve symbol unix_sys_open by loading os_sys.o and linking the two
> modules together. In the above case it will then start extern-t1's
> main() routine.


Okay. The linkers and loaders I've used would need to be told
where to look for the module defining unix_sys_open, but assuming
it's found all will be well.

> So only extern-t1 is invoked. The 'system' under which it runs will be
> responsible for resolving the reference to the external symbol.


Now you've made me unsure of your intent. The unaided 'system'
probably can't find unix_sys_open without some help from you --
after all, there might be forty-two .o files lying around with
forty-two incompatible definitions of unix_sys_open, and someone
has to tell the system which of them to use. The 'system' will
(most likely) document the places it will search and the order in
which it will search them, and it'll be up to you to ensure that
the desired unix_sys_open is found in one of them. (If it's found
in more than one, the system presumably documents how ties are
broken.)

> One issue for me is that changes to symbolic constants in one routine
> require recompilation of all modules which are potentially dependent.
> In most cases the symbolic constants may not change but all dependent
> files still need to be recompiled just in case. It would be better if
> dependent module need to be recompiled only when other more
> substantive parts of the interface change - e.g. a new version of a
> piece of server code *takes away* a feature that existed in earlier
> versions or *changes* an interface to make it incompatible with
> previous versions.


Okay. Your use of an external-linkage variable initialized to
the desired value solves this. It has the drawback that the constant
can no longer be a "constant expression," so you couldn't (for
example) use `case unix_sys_open:' as a `switch' label. That may
or may not be important; it depends what you want to do with the
value.

--
Eric Sosman
esosman@comcast-dot-net.invalid

James Kuyper 11-23-2012 03:33 PM

Re: Separate compilation and access to small integer data in otherprograms
 
On 11/23/2012 08:14 AM, James Harris wrote:
> On Nov 22, 10:39�pm, "BartC" <b...@freeuk.com> wrote:
>> "James Harris" <james.harri...@gmail.com> wrote in message
>>
>> news:4beef828-17fc-4c18-ba8c-9b92733ae9bc@n5g2000vbk.googlegroups.com...
>>
>>
>>
>>> Having tried a few options I'm not sure how best to do the following
>>> in C or how feasible this is. Any suggestions?

>>
>>> AIUI the normal way of working in C where one program, say AA, uses
>>> integers (or bitfields) as part of its interface those integers are
>>> defined in a header file, say AA.h. Then when program BB wants to
>>> interface with AA it includes AA.h and uses those symbolic values when
>>> talking to AA. This means that the values are compiled-in to BB.

>>
>>> By contrast, I want the two progams to be compiled separately. Program
>>> AA is to be compiled with its integer constants wholly inside it.
>>> Program BB is to refer to the values as externals. Example below. The
>>> intention is that the constants not be resolved at compile time but at
>>> link time or later.
>>> File AA:
>>> int AA_CONST1 = 1;
>>> int AA_CONST2 = 4;
>>> int AA_FLAGS1 = 0x8000
>>> void AA() {}

>>
>>> File BB:
>>> extern AA_CONST2;
>>> void BB() {
>>> �x = AA_CONST2;
>>> �AA_function(x, AA_FLAGS1);
>>> }

>>
>> It seems fine provided you realise that these are not actually constants
>> (ie. immediate data), but variables.


The term "variable" is not defined in the standard, except as part of
the phrase "variable length array". From the way it's used in the
standard, I've inferred that when used as a noun, it means "named
object", and every use in the standard is consistent with that meaning.
However, every use in the standard is also consistent with a more
restricted meaning that some people consider more consistent with it's
use in ordinary English, as something that can vary: "named object whose
definition is not const-qualified". His comment is more relevant if he's
referring to the more restricted definition.

> In other words, I cannot protect the original values from being
> altered? That may be a problem. Is there no way with C to make them
> read-only?


Yes, you can add 'const' to the definition of such objects. That will
mean that diagnostics are mandatory if any part of the code makes a
naive attempt to modify them. The compiler has the option, after issuing
the mandatory diagnostic, of accepting your code and translating it into
a program; if you choose to run it, the behavior of that program is
undefined. The diagnostics can also be bypassed by use of a cast:

*(int*)AA_CONST = 3;

but such code also has undefined behavior.

Therefore, const-qualification doesn't actually guarantee that the
object can't be modified. Undefined behavior includes the possibility (
among infinitely many others) that the object's value could be modified.
However, mandatory diagnostics is the strongest guarantee C provides on
such matters; if it's not good enough, you'll have to find some other
language that provides a better guarantees.

> The slight issue is that there seems to be no way in C to say which
> symbols should be made available to other routines, i.e. which should
> be exported, so if two programs are linked together each has access to
> all the data in the other. I'd welcome correction if there is a
> feature of C which allows this control.


C does not allow two programs to be linked together, so the issue never
comes up. Operating systems sometimes provide ways for programs to share
memory, for example if you're using a Unix-like system, you can use
mmap(), but the use of that function is outside the scope of the C
standard, and you'll get better answers to questions about it on
comp.unix.programmer. Other operating systems have similar features.

C does allow translation units to be linked together. I suspect that you
may be confusing translation units with programs. There can be at most
one main() function in any program (on freestanding systems, the
equivalent of main() may have a different name), but it can contain
multiple translation units. I'm most familiar with Unix-like systems, so
I'll use them as an example of how this usually works: you use a
compiler to convert a translation unit into an object file with an
extension of .o. Object files can be collected together into libraries
(with an extension of .a or .so). A linker is used to link one or more
..o files, whether stand-alone or extracted from a library, into a single
program.
Does that give you a better idea of what "program" means?

C does provide features that allow you to control whether or not an
identifier is shared between TUs. Every identifier has either internal
linkage, external linkage, or no linkage. Linkage applies only to
identifiers that identify objects or functions. Identifiers with
internal linkage identify the thing that they identify only within the
same translation unit where they are declared. Identifiers with external
linkage can be declared in one translation unit, and used within the
scope of that declaration to refer to things that might be defined in a
different translation unit. If your program has multiple definitions of
an identifier with external linkage, it has undefined behavior. If your
program actually uses the think identified by such an identifier, it has
undefined behavior unless it includes exactly one definition of the
identifier.

When you define a function or a file scope object, the identifier has
external linkage unless the 'static' keyword is used. Declarations of
functions and objects that have the 'extern' keyword do not define
anything, but merely make the corresponding identifier usable within the
scope of the declaration to refer to something defined with external
linkage using the same identifier in some other part of the program.

As a general rule, every identifier with external linkage (except
possibly "main") in your program should be declared in a header file
(with the 'extern' keyword if it identifies an object). That header file
should be #included in every translation unit where that identifier is
referred to - INCLUDING the one where the thing it identifies is
defined. This is not something that's required by the standard, it's
merely a good way to ensure that the thing is declared consistently
where it is used. The behavior of your program can be undefined if the
declarations are inconsistent between different translation units.
--
James Kuyper

Ben Bacarisse 11-23-2012 05:38 PM

Re: Separate compilation and access to small integer data in other programs
 
James Kuyper <jameskuyper@verizon.net> writes:
<snip>
> However, mandatory diagnostics is the strongest guarantee C provides on
> such matters; if it's not good enough, you'll have to find some other
> language that provides a better guarantees.


It's probably worth pointing out that there is no mandatory diagnostic
if a const-qualified object is accessed via a non-const-qualified
declaration in another translation unit. The behaviour is undefined,
but there won't necessarily be a message, and there's not cast to alert
anyone that something fishy may be going on.

(Your strategy below -- snipped -- of putting all extern declarations in
a header file to be included in all files that use or declare the
objects in question avoids this problem.)

<snip>
--
Ben.

BartC 11-23-2012 06:26 PM

Re: Separate compilation and access to small integer data in other programs
 
"James Harris" <james.harris.1@gmail.com> wrote in message
news:01770426-6c82-4870-aa8c-98194b428fba@ez26g2000vbb.googlegroups.com...
> On Nov 22, 10:39 pm, "BartC" <b...@freeuk.com> wrote:
>> "James Harris" <james.harri...@gmail.com> wrote in message


>> > File AA:
>> > int AA_CONST1 = 1;
>> > int AA_CONST2 = 4;
>> > int AA_FLAGS1 = 0x8000
>> > void AA() {}

>>
>> > File BB:
>> > extern AA_CONST2;
>> > void BB() {
>> > x = AA_CONST2;
>> > AA_function(x, AA_FLAGS1);
>> > }

>>
>> It seems fine provided you realise that these are not actually constants
>> (ie. immediate data), but variables.


> In other words, I cannot protect the original values from being
> altered? That may be a problem. Is there no way with C to make them
> read-only?


After linking? No. There are 'const' attributes, meaning read-only, that can
be applied in the C source to a variable, but that doesn't protect against
accidental, deliberate or malicious modifications. Depends how much you
trust the other module.

> I suppose ideally the loader would place any constants in a read-only
> page. I can do that and it would protect them perfectly. It would also
> allow any attempt to update them to be trapped and pinpointed.


I think that in the general case, that's not practical to do (some
const-attribute variables are on the stack, some are on the heap (via a
pointer), and some programs might depend on being able to bypass the
protection).

But, if the values *are* changed in one module, the new values will be
tracked in the other, so will that really cause a problem? Eg. will there
lots of dependencies which will then be out of step?

> The slight issue is that there seems to be no way in C to say which
> symbols should be made available to other routines, i.e. which should
> be exported, so if two programs are linked together each has access to
> all the data in the other. I'd welcome correction if there is a
> feature of C which allows this control.


In C, every function and variable declaration outside of a function can be
assumed to have a 'global' attribute, meaning it is always exported. (In
fact I explicitly add such an attribute in my own code; defining it is easy:

#define global

This then makes it obvious.) If you don't want to export a name, use a
'static'
attribute.

The 'extern' attribute you already know about (it should match a 'global'
name in at most one other module. (But C also allows extern and global
declarations of the same name, in the same file - usually extern will be in
a header, shared with other files, and global in the main body of the file.)

>> Immediate data I think can be imported across modules via a linker, but I
>> don't think there's a mechanism in C to achieve this (even an assembler
>> would have difficulty because it might need to know the field-size needed
>> to
>> contain the value).

>
> Field sizes should be fixed (at least under a 32-bit OS; a change to a
> 64-bit OS would be another issue but them everything would probably
> require a recompile).


They can; but sometimes you might be wasting a 32-bit field when there might
be an instruction available with an 8-bit field.

(BTW I trust you understand what I mean by 'variable' and I don't have to
expand on it!)

--
Bartc


Jorgen Grahn 11-24-2012 10:36 AM

Re: Separate compilation and access to small integer data in otherprograms
 
On Fri, 2012-11-23, James Harris wrote:
> On Nov 22, 10:24*pm, Ben Bacarisse <ben.use...@bsb.me.uk> wrote:
>> James Harris <james.harri...@gmail.com> writes:
>> > Having tried a few options I'm not sure how best to do the following
>> > in C or how feasible this is. Any suggestions?

>>
>> > AIUI the normal way of working in C where one program, say AA, uses
>> > integers (or bitfields) as part of its interface those integers are
>> > defined in a header file, say AA.h. Then when program BB wants to
>> > interface with AA it includes AA.h and uses those symbolic values when

....
>> I'm having trouble squaring this with the words. *Neither AA nor BB
>> seems to be a program. *They just look like translation units that might
>> be linked into one single program (with some other bits of course).

>
> Possibly poor choice of term on my part. "Program" may be too generic.


No, it's simply the wrong term. A "program" is an executable which a
user can cause to run. It's a bit fuzzy around the edges, but the
word is never used the way you used it above.

/Jorgen

--
// Jorgen Grahn <grahn@ Oo o. . .
\X/ snipabacken.se> O o .


All times are GMT. The time now is 10:29 PM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.